Calibrating COCOMO II For Functional Size Metrics
Calibrating COCOMO II for Functional SizeMetricsAnandi HiraUniversity of Southern CaliforniaJoint Software and IT Cost Forum 2020
Motivation
Objective/Goal:Allow FPs and CFPs as size inputsCOCOMO II (Constructive Cost Model)Software size estimate(in SLOC (Source Lines ofCode))COCOMO Estimates:Effort Factors (project,product, environmental)Software reuse,maintenance, andincrement parameters ResourceEquivalent SizeReuse impactRe-Engineeringor conversionMaintenanceSoftware Project dataCOCOMO II is an open and free modelSoftwaredevelopment andmaintenance: Costs (effort) Scheduleestimates Distributed byphase,activity,incrementLocal calibration toorganizationβs data
2 Prominent Functional Size MethodsIFPUG Software Model forFunction Points (FPs)COSMIC Software Model forFunction Points (CFPs)
Challenges for Organizations/TeamsFPs vs Effort from Empirical StudiesCFPs vs Effort from Empirical echtMatson100Zheng30000Effort (hours)Effort (hours)KemererDiMartino 000200400IFPUG Function Points (FPs)80010001200CFPs vs Effort across 2 teams from same 00Team 12,000Team 2Effort (hours)7,0003,000Team 12,000Team 21,0001,0000600COSMIC Function Points (CFPs)FPs vs Effort across 2 teams from same organizationEffort (hours) Research papers andorganizations typicallydo not account foreο¬ort factors. Teams will not havemuch data initially. Sharing data acrossteams cause highvariance. Empirical researchdoesnβt proposegeneralized model. Public datasets variedwith high 00IFPUG Function Points (FPs)COSMIC Function Points (CFPs)140160180
COCOMO II Effort Estimation Modelππ π΄ Ο ππΉ )(π΅ πΆ πππ§π ΰ· πΈπExponent ranges from 0.9 to 1.2, with 1.0997 as defaultPMSizeAB, CSFEM Software development effort (in Person-months)Size in Thousand Source Lines of Code (KSLOC)Calibrated Productivity constant (KSLOC/PM)Calibrated Exponent constantsScale Factors β have exponential effectEffort Multipliers β have multiplicative effect
DatasetsUnified Code Count (UCC)Confidential Industry Maintenance projects New development, with somereuse from previous work Firmware and software interactswith hardware Command line program Implemented in C, Verilog, VHDL Data from 2 teams 18 data points Add new features (10) Modify existing features (23) Code metrics tool Command line program Implemented in C , Java Each project by new team 32 data points
Methodology
Research Question and Hypothesis1. Can calibrated COCOMO II for FPs and CFPs perform better than optionssuggested in research? Null Hypothesis (H0): Calibrated COCOMO II will not perform better than thecurrently available options.2. Do functional size metrics, using the calibrated COCOMO II model, performbetter on some types of projects compared to others? Null Hypothesis (H0): Functional size metrics perform equally well on all types ofprojects.
Calibrating COCOMO IIππ π΄ πππ§π (π΅ πΆ Ο ππΉ ) ΰ· πΈπ1. Productivity Rates New Development Enhancement β New Features Enhancement β Modify Existing Features2. Adjust factors that may have relationship withsize β Complexity (CPLX)3. Adjust Exponent constants to adjust the rate atwhich effort grows with respect to size
Regressions to Compare (Question 1)Compare ImprovementCalibration Comparisons1. Linear Regression2. Nonlinear Regression (logtransform)3. Convert FPs to SLOC using ratiospublished by Capers Jones4. Convert FPs (and CFPs) to SLOC withcustom conversion ratioStep-wise to determine significance1. Productivity Factor2. New Dev/Enh3. New Dev/Add/Mod5. Convert CFPs to FPs and use existingmodel (linear and nonlinear)4. Prod Factor & Complexity5. New Dev/Enh & Complexity6. New Dev/Add/Mod & Complexity
2-Step CalibrationP hrsStep 1Calibrate forProductivity Rates andComplexity (CPLX)factor.Compare R2 for best fitbefore moving to Step 2. π¨ πππ§π (π΅ πΆ Ο ππΉ ) ΰ· π¬π΄log(P hrs) log π¨ ΰ΅«π΅ πΆ Ο ππΉ ΰ΅― log πππ§π log Ο EM CPLX log CPLXlog(P hrs) log Ο EM CPLX log π¨ π΅ πΆ Ο ππΉ log πππ§π log CPLXA number for now β throw away. Will calibrate in Step 2
2-Step Calibrationlog(P hrs) log Ο EM CPLX log π΄ π© πͺ Ο ππΉ log πππ§π log CPLXStep 2Calibrate for Exponentconstants B & C.log(P hrs) log Ο EM CPLX log π΄ πππ CPLX π© πͺ Ο ππΉ log πππ§ππππ P hrs πππ Ο πΈπ πΆππΏπ πππ π΄ πππ πΆππΏππππ Size π© πͺ ππΉ
NormalizeEffortP hrs π΄ πππ§π (π΅ πΆ P hrsΟ πΈπWith Respect toEffort Factorsdefined byCOCOMO II π΄ πππ§π (π΅ πΆ Ο ππΉ ) ΰ· πΈπΟ ππΉ )P hrsΟ πΈπ πππ§π (πΆ π΄ πππ§πΟ ππΉ )π΅ πΆ Ο ππΉ (πΆ Ο ππΉ )P hrsΟ πΈπ πππ§π (πΆ Ο ππΉ ) π΄ πππ§π π΅
R2: how closely the regressioncurve fits the data csMMRE: Mean Magnitude ofRelative Error. Ideally 25%PRED(25): Percentage ofestimates within 25% of actuals.Ideally 75%
Types of Projects (Question 2)1. Low parsing projects β UCC: control and data managementoperations, 1-3 computational operations2. High parsing projects β UCC: control and data managementoperations, 3-5 computational operations3. Data transfers, interact with Hardware β Industry: control, datamanagement, device-dependent, and simple computations4. Record, Encrypt, Decrypt β Industry: control, data management,device-dependent, and complex computations5. Input and Outputs β Industry, UCC: control and data managementoperations, 0 computational operations. Industry also includessome device-dependent operations.
Details of the calibratedCOCOMO II modelCOCOMO II forFunctional SizeMetrics
Calibration Step 1 ResultsStep-wise process to determine significanceIFPUG FPs (FPs)COSMIC FPs (CFPs)Prod .777Prod Factor & CPLX0.9210.954New/Enh & CPLX0.9220.973New/Add/Mod & CPLX0.9570.975Perform Step 2 of Calibration (exponent) on last model.
Original vs Calibrated COCOMO II ModelP hrs π΄ πππ§π (π΅ πΆ Ο ππΉ ) ΰ· πΈπACOCOMO II (SLOC)446.88FP, New Development52.602FP, Add Features100.51FP, Modify Features43.84CFP, New Development166.94CFP, Add Features95.04CFP, Modify Feature76.32BC0.910.010.8330.0110.6290.014
Original vs Calibrated Exponentπ βππ π΄ πππ§π (π΅ πΆ Exponent Range:Ο ππΉ )Low ΰ· πΈπDefaultHighCOCOMO II (SLOC)0.911.09971.2262COCOMO II (FPs)0.8331.05111.1963COCOMO II (CFPs)0.6290.90151.0829
Original vs Calibrated Complexity Factorππ π΄ πππ§π (π΅ πΆ Complexity Range:Ο ππΉ )Very Low ΰ· πΈπNominalExtra HighCOCOMO II (SLOC)0.7311.74COCOMO II (FPs)0.5712.298COCOMO II (CFPs)0.5312.57
Does CalibratedCOCOMO II performbetter than optionssuggested or provided inresearch papers?CalibratedCOCOMO II vsOptions inResearch
IFPUG FPsStatisticLinearNonlinearJonesConversionto SLOCLocalConversionto SLOCCalibratedCOCOMO 14%2%68%Conclusion: Calibrated COCOMO II performed better than other options
COSMIC FPsConversionto FPs LinearConversionto FPs NonlinearCalibratedCOCOMO IIStatisticLinearNonlinearLocalConversionto )18%38%4%22%20%70%Conclusion: Calibrated COCOMO II performed better than other options
Do functional size metrics,with the calibratedCOCOMO II model,perform better on sometypes of projectscompared to others?Analysis byProject Type
1. Low parsing projects - UCCUCC Low Parsing - CFPs450500400450Normalized Effort (hrs)Normalized Effort (hrs)UCC Low Parsing - 46810IFPUG Function Points (FPs)1214012345COSMIC Function Points (CFPs)PRED(25)80PRED(25)75MMRE16.21MMRE19.27Corr Coeff0.378Corr Coeff0.328678
2. High parsing projects - UCCUCC High Parsing - FPsUCC High Parsing - CFPs700500450600Normalized Effort (hrs)Normalized Effort IFPUG Function Points (FPs)560246810COSMIC Function Points Corr Coeff0.693Corr Coeff0.8821214
3. Data Transfer, HW β IndustryData Transfers - CFPs7000350060003000Normalized Effort (hrs)Normalized Effort (hrs)Data Transfers - 150IFPUG Function Points (FPs)200250020406080100120140COSMIC Function Points (CFPs)PRED(25)50PRED(25)80MMRE31.85MMRE19.36Corr Coeff0.973Corr Coeff0.973160180
4. Record, Encrypt, Decrypt β IndustryRecord, En/Decrypt - CFPs14006001200500Normalized Effort (hrs)Normalized Effort (hrs)Record, En/Decrypt - FPs100080060040020040030020010000474849505152IFPUG Function Points (FPs)5354051015202530COSMIC Function Points rr Coeff0.866Corr Coeff0.8663540
5. Inputs/Outputs β Industry, UCCInputs/Outputs - FPsInputs/Outputs - CFPs1200800700Normalized Effort (hrs)Normalized Effort 5IFPUG Function Points (FPs)303540024681012COSMIC Function Points (CFPs)PRED(25)50PRED(25)37.5MMRE68.49MMRE28.22Corr Coeff0.683Corr Coeff0.8681416
Conclusions
Conclusions (1/2)Hypothesis 1 Calibrated COCOMO II will notimprove effort estimationaccuracy compared to othersuggested methods. False: Calibrated COCOMO IIperformed better for FPs andCFPs by a minimum PRED(25): 32% MMRE: 21.44%COCOMO II Calibration1. Different productivity rates forNew Development, AddModules, and ModifyingExisting Modules2. Complexity has stronger impact(in both low and highdirections) on effort comparedto SLOC.3. Effort grows at different ratewith respect to functional sizemetrics compared to SLOC
Conclusions (2/2)Hypothesis (2) Functional Size Metricsperform similarly well on alltypes of projects Performance varies See table to right where Red X means correlationcoefficient and/or accuracylow Green check meanscorrelation and accuracywithin acceptable rangesFPsCFPsLow Parse β UCC High Parse β UCC Data Transfer β Industry Record, Encrypt, Decryptβ IndustryInputs/Outputs β UCC,Industry
IFPUG vs COSMIC Function PointsIFPUG Function PointsCOSMIC Function Points New development tasks Large number of datatransferred Components with encryptionand decryption functionality Generally across multipledatasets Smaller/various changes in datatransferred Maintenance tasks Object-Oriented Design
Future Work Reuse factors in functional size (equivalent size like reuse model inCOCOMO II for SLOC) Separate the 5 types of Complexity as separate factors: Control, Computational, Device-Dependent, Data Management, and UserInterface Operations Each may have different impacts Size metric that accounts for changes in algorithms
1. Can calibrated COCOMO II for FPs and CFPs perform better than options suggested in research? Null Hypothesis (H 0): Calibrated COCOMO II will not perform better than the currently available options. 2. Do functional size metrics, using the calibrated COCOMO II model, perform better on some types of projects compared to others?
COCOMO -the βCOnstructive COst MOdelβ β COCOMO II is the update to COCOMO 1981 β ongoing research with annual calibrations made available Originally developed by Dr. Barry Boehm and published in 1981 book Software Engineering Economics COCOMO II described in new book Software Cost Estimation with COCOMO II COCOMO can be used as a .
The COCOMO software package is based upon the software cost and schedule estimation model: COnstructive COst MOdel version II (COCOMO II). This is the newly revised version of the original COnstructive COst MOdel (COCOMO) first published by Dr. Barry Boehm in his book Software Engineering Economics, Prentice-
COCOMO Model Basic (COCOMO I 1981) Menghitung dari estimasi jumlah FP dan LOC; FP suatu unit pengukuran untuk keterhubungan dan keterkaitan antar prosedur, fungsi dan lingkungan SW Intermediate (COCOMO II 1999) Menghitung dari besarnya program dan βcost driversβ (faktor-faktor yang berpengaruh langsung kepada
Bruno Hott 2 COCOMO II COCOMO II foi construΓdo em cima do COCOMO '81 para levar em consideração: β Novos processos de desenvolvimento (ex. espiral) β Aumentar a flexibilidade em desenvolvimento de software (ex. reuso, geração de cΓ³digo automΓ‘tico) β Necessidade de tomada de decisΓ£o com informaçáes incompletas β Novos dados de projetos (161 projects vs 61)
COCOMO II is also more accurate to solve the problems in software project estimation. In COCOMO II, there are two models, namely Early Design and Post Architecture model. Early Design model is a high-level model in COCOMO II. This model has function to explore alternatives in architectural or do the development strategies.
2. COCOMO II Model COnstructive COst MOdel II (COCOMO II) [4], which was developed in 1995, is a model that allows one to estimate the cost, e ort, and schedule when planning a new soft-ware development activity. It takes qualitative inputs and produces quantitative results. In COCOMO II, the e ort is represented as person-months (PMs). A .
Empirical Estimation of COCOMO I and COCOMO II Using a Case Study Muhammad M. Albakri1 M. 2Rizwan Jameel Qureshi 1-2Department of Information Technology, King Abdul-Aziz University, P.O. BOX 80221 Jeddah 21589, Saudi Arabia Abstract- There are several software estimation models such as Line of Code, Function Point and COnstructive
Neither A. Thomas Perhacs nor Velocity Group Publishing assumes any responsibility for the use or misuse of the concepts, methods and strategies contained in this book. The reader is warned that the use of some or all of the techniques in this book may result in legal consequences, civil and/or criminal. USE OF THIS BOOK IS DONE AT YOUR OWN RISK. (Updated Version, July 2008) As you begin to .
