Risk-based, Probabilistic Cost Estimating Methods - John Reilly Us
RISK-BASED, PROBABILISTIC COST ESTIMATING METHODSAuthors:John J. Reilly,John Reilly International, rds:A. Moergeli,moergeli consulting, llc., USAwww.moergeli.com,info@moergeli.comPhilip Sander,RiskConsult GmbH, Austria,www.riskcon.atsander@riskcon.at"Cost Optimization and Financing of Underground Structures "Risk, Management, Cost, Estimating, Probabilistic, Analysis1.2.SUMMARY. 2INTRODUCTION . 22.1COST ESTIMATING METHODS - DETERMINISTIC AND PROBABILISTIC.23. COST ESTIMATING - OVERVIEW. 33.1COST ESTIMATING MUST ADEQUATELY CONSIDER INHERENT UNCERTAINTY .33.2TYPES OF COST ESTIMATE METHODS .33.3COMPONENTS OF COST ESTIMATES – BASE COST, RISKS, AND OTHER UNCERTAINTIES .33.4REPRESENTATIVE COST ESTIMATING METHODS .43.5THE DETERMINISTIC METHOD.43.5.1Contingency Factors.43.6THE PROBABILISTIC METHOD .53.6.1Comparison of Deterministic and Probabilistic Cost Estimation Methods .63.6.2Key Considerations of Deterministic Versus Probabilistic Cost Estimates.73.7THE BANDWIDTH APPROACH .83.8THE SQUARE ROOT APPROACH TO AGGREGATE RISK VALUE .84. COST ESTIMATING FOR CONSTRUCTION. 94.1THE TRADITIONAL WAY – A DETERMINISTIC APPROACH .95. COMPARING COST ESTIMATING METHODS. 95.1COMPARING COST ESTIMATING METHODS – EXAMPLES .95.1.1Deterministic Approach.95.1.2Bandwidth Approach .95.1.3Square Root Approach.95.1.4Probabilistic Approach.105.2ASSESSMENT OF ESTIMATING METHODS .106. ADVANTAGES OF USING RISK-BASED, PROBABILISTIC COST-ESTIMATING. 116.1HOW A BETTER COST-RISK ASSESSMENT HELPS EVEN IN A “LOW-BID” ENVIRONMENT .116.2CONTRACTOR’S ADVANTAGE USING RISK-BASED, PROBABILISTIC ESTIMATING .116.3BENEFITS OF MORE COMPLETE RISK INFORMATION .116.4CONTRACTORS FOCUS ON MAKING A PROFIT DURING CONSTRUCTION .136.5OWNER’S ADVANTAGES USING A RISK-BASED, PROBABILISTIC COST ESTIMATING METHOD .136.5.1Owner’s Strategy for Budgeting and Bidding.136.5.2Owner’s Strategy in the Construction Phase.147. EXAMPLES OF PROBABILISTIC COST ESTIMATING METHODS . 147.1COST ESTIMATE VALIDATION PROCESS (CEVP ).147.2RIAAT (RISK ADMINISTRATION AND ANALYSIS TOOL) .158. INTERDEPENDENT AND CORRELATED RISKS. 169. KEY POINTS. 1610. CONSIDERATIONS. 1711. REFERENCES . 17ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 1 of 17
ITA WTC 2015 Dubrovnik, Croatia1.SummaryEvery cost estimate is uncertain. Underestimating construction costs, by owners in the planning or designphases or by contractors in the bidding phase, and with respect to low probability/high impact “black swan”events, can lead to disputes, claims, and litigation. A better understanding of potential costs can help ownersbudget and get authorization for projects with a reduced chance of cost overruns. A better understanding ofpotential costs can help contractors in determining an appropriate base cost and margin for bidding, strategiesto secure the work in a low-bid environment, and construction management strategies to maximize profit, toavoid loss, and to better manage, and recover costs of, construction changes and claims.This paper will address risk-based probabilistic cost estimating methods that can improve our appreciation ofthe cost of uncertainty and potential risk events. It will address the uncertainty inherent in predicting the valueof any future project element or process and the identification and characterization of potential risk (threats oropportunities) that can impact outcomes.2.IntroductionEstimating and managing the costs of complex infrastructure projects – in the planning, design, andconstruction phases, has been a challenge for decades for both owners and contractors. The more complex andtechnologically advanced the project, the greater the uncertainty and related challenges – including potentialrisks that are important to owners and contractors, such as: Cost risks to owners – meeting budget and schedule, maintaining public credibilityCost risks to contractors – profit, consequences of loss, impacts to reputation/future workThis concern has been addressed in various ways by the underground construction industry, for some time(Reilly 2001 et seq.). In particular, while significant advances have been made in cost estimating for theplanning and design phases (Reilly et al., 2004), which are important to agencies and political decisionmakers, it is not apparent that these advances have been widely included in construction phase cost estimates.The reasons for this seem to relate to “low-bid” considerations – any method that tends to increase thecontractor’s cost estimate by including risk costs may lead to an erosion of the contractor’s competitiveposition if others are not similarly required to include such costs.2.1Cost Estimating Methods - Deterministic and ProbabilisticThe probabilistic approach, compared to the simpler deterministic approach, fundamentally gives more usefulinformation with respect to the range of probable cost and cost drivers (risks, opportunities, variability,potential for loss or gain) as shown in Figure 1 following.Fig. 1 Deterministic Cost Compared to Probabilistic Profit–Loss Curve (Sander 2014)ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 2 of 172
In Figure 1, the results for deterministic and probabilistic cost estimates are shown related to the potentialprofit or loss for a typical project. As is evident in this example, there is significant potential for costs to berealized that are higher than the proposal bid value estimated using a deterministic approach. Using aprobabilistic approach, it is possible to better recognize this potential outcome in the bid phase and, as aconsequence, to:1. Change the budget (as an owner) or proposal-bid value (as a contractor) – if this is consistent with astrategic approach to win project funding (as an owner) or the bid (as a contractor), in order to realizea project within budget (as an owner) or a profit a the end of the job (as a contractor), or2. To stop the project (as an owner) or withdraw from the project (as a contractor) if a strategy to winthe bid and still realize a profit is not feasible.3.Cost Estimating - Overview3.1Cost Estimating Must Adequately Consider Inherent UncertaintyCost estimating must deal adequately with uncertainty, especially in the very early stages of projects whereneither the exact quantities nor the exact costs or prices are known, and quantities can only be addressed byreference to basic elements and a detailed analysis is not yet available due to a lack of precise information.With a deterministic approach, information about uncertainties and their characteristics – such as higher orlower values, ranges of quantities, and potential costs – cannot be easily taken into consideration although thisinformation is generally available or could be estimated. A probabilistic approach can more reasonablyaddress this type of uncertainty.3.2Types of Cost Estimate MethodsThere are several different methods of preparing a cost estimate depending on the purpose, level of planning,and/or design, as well as project type, size, complexity, circumstances, schedule, and location. These methodscan fall into categories such as: parametric, historical bid-based, unit cost/quantity based, range, andprobabilistic risk-based estimates. For a detailed discussion of cost estimating see Reilly 2010.References for best cost estimating practices include “Project Management Body of Knowledge,” Chapter 7,“Project Cost Management” (PMI 2004), and State Agency guidelines documents such as WSDOT’s “CostEstimating Manual for WSDOT Projects” (WSDOT 2009) and the AACEI Guidelines (AACEI 2003 et seq.).3.3Components of Cost Estimates – Base Cost, Risks, and Other UncertaintiesThe components of cost that need to be correctly addressed in the estimate include:1. Base cost – the cost that will result if “all goes according to plan” (Reilly 2004)2. Risk cost – the cost result of threat and opportunity events, if they should occur3. Escalation cost – cost resulting from normally expected inflation4. Other uncertain cost – cost that results from other events, normally external to the project team’scontrol, which may include unanticipated events, politically related changes, and “black swan” events(Talib 2007)In order to identify and address risk cost, all cost components should be considered. Each cost component willhave an individual uncertainty that should be considered using adequate evaluation methods. In particular, forlarger projects, individual line-item budgets should be created for every cost component to enable tracking ofdeviations and management of changes throughout project development, design, and construction.The method by which these components are evaluated, quantified (estimated), modelled, and combined iscritical to a valid result. Different methods treat each component differently – which can lead to differences inthe reliability and usefulness of the results. Additionally, uncertainty always plays a major role in estimates –for example, while basic cost elements may be reasonably well known, the quantities and prices associatedwith them are uncertain, leading to variability in these base costs.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 3 of 17
ITA WTC 2015 Dubrovnik, CroatiaÆ Will always occurÆ Exact price, quantity, or time uncertainÆ Has a probability of occurrenceÆ Consequences (cost, time, etc.) are uncertainFig. 2 Uncertainty in Base Cost and Risks3.4Representative Cost Estimating Methods1. Deterministic: Aggregated unit quantities multiplied by unit prices – usually with some degree ofconservatism built in – plus an added reserve or contingency2. Probabilistic: Range approach which characterizes cost information with probability distributions3. Bandwidth: Range approach where costs of each line element are characterized with parameters ofminimum, most likely, and maximum cost. The total cost is obtained by simply adding theseparameters for all line items4. Square Root: The Square Root approach delivers one single figure which is the sum of all base costsplus the square root of the sum of the squares of the risk contingencies.3.5The Deterministic MethodThe deterministic process is commonly used to create a budget or bid price. This involves estimating knownquantities (from plans) and unit prices (from contractors or suppliers) to get “line item costs” and adding acontingency to account for the incomplete nature of the design, project uncertainties, and the consequence offuture events.A more sophisticated deterministic approach adds line-item risk elements to the deterministic base cost, usinga risk-register and assigning a probability of occurrence and impact to each risk register line item. The resultis the expected value of risk impacts. If multiple risks are to be accounted for, the total risk is often computedas the mathematical sum of all single risks.Rtotal pi * I iFig. 3 Equation for Deterministic Aggregation of RisksHowever, such a simple summation for risks (or for ranges) should not be done by simple mathematicaladdition because it is statistically not valid. It is also necessary to add an overall contingency to account forother unknowns. That overall contingency is subject to bias since there may be no rational basis for how theunknowns are aggregated or how they are estimated.3.5.1Contingency FactorsThe uncertainty (and associated contingency) at various project phases can be classified by such techniques as“Estimate Class Levels” (AACEI 2003), used in deterministic cost estimates, in which the inherentuncertainty is reduced as the project advances through the phases of planning, design, bidding, and intoconstruction. The uncertainty is represented by “contingency factors” that are related to these phases.Contingency in the AACEI table can range from 5–75% depending on phase and circumstance. Alternatively,probabilistic cost-risk estimating recognizes that base costs and risk events have uncertainty in bothprobability and impact (positive or negative). This method is more detailed and analytically more complete.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 4 of 174
Applying contingency factors is a very broad approach, not very useful for identifying and developing astrategic management of risk or achieving a profit in construction. The contingency applied in thedeterministic method is often based solely on the cost estimator’s judgment or experience with a history ofsimilar projects, if available, but this is problematic for at least the following reasons:1. Estimators and project staff are generally optimistic in their approach to estimating costs.2. The “history of similar projects” varies with each contractor’s experience.3. The “history of similar projects” may be inadequate if applied to the current project.The contingency approach does not give useful information on the impact (probability and magnitude) ofuncertain events. This means that strategies such as risk avoidance, risk mitigation, or risk transfer cannot besufficiently evaluated.3.6The Probabilistic MethodIn the probabilistic method, the total cost is made up of base costs (quantities times unit prices, both withsome variability) plus risk events – including risks of delay with associated liquidated damages, risks ofescalation, and the cost impact of other higher-level (e.g., political) risks. Risk impacts are determined byestimating the probability of occurrence and the impact of specific risk events (normally in a workshop withproject staff and subject matter experts). Dependencies and correlations between specific risks are alsoelicited and used in modelling.Since empirical/historical data, as input to the risk analysis, is often not available, the risk probabilities can bedifficult and complex to estimate. The probabilistic method characterizes each risk with individual andspecific distributions – such as a large “bandwidth” for large uncertainties or a narrower “bandwidth” forsmaller uncertainties. Using this approach, the uncertainty contributing to a particular cost estimate can bemodelled more specifically and in greater detail than by use of a single-point deterministic estimate.Distribution Function (Impact in kUSD)6%100%90%80%70%4%60%50%3%40%2%Value at RiskRelative Frequency5%30%20%1%10%5045403530250%200%Fig. 4 Probability Distribution for an Individual Risk Using a Triangle Function (Sander 2014)Single, individual risks can be evaluated using probability distributions, one of which is shown in the figureabove. These individual distributions can be aggregated using simulation methods (Monte Carlo Simulationor Latin Hypercube Sampling) to determine a probability distribution that depicts the total risk potential.Value at Risk (VaR) defines, within a distribution, a certain amount (e.g., USD) that will not be exceededaccording to the corresponding probability. In the example below, VaR 70 means that a 5M budget wouldnot be exceeded in 70% of all simulated scenarios. However, even with such coverage, there remains a 30%probability that the 5M budget will be exceeded.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 5 of 17
ITA WTC 2015 Dubrovnik, CroatiaDistribution Function (Impact in kUSD)7%100%90%6%80%70%60%4%50%3%40%Value at RiskRelative 000%Fig. 5 Probability Distribution Showing Probable Cost with Value at Risk Data (Sander 2014)3.6.1Comparison of Deterministic and Probabilistic Cost Estimation MethodsTable 1 Comparison of Deterministic versus Probabilistic Cost Estimation MethodsElementDeterministicProbabilisticInputA single value for probability and asingle value for impact of each risk.One figure for the probability ofoccurrence and several values forthe impact (e.g., minimum, mostlikely, and maximum), thereforeincluding uncertainty.ResultA single value from a mathematicaladdition of the impacts of all risks(probability multiplied by impact).A “range of probable cost” with allproject risks included, based onthousands of coincidental butrealistic scenarios.QualificationResults are displayed as a single, sharpfigure, which, in itself, does not have aprobability.Results are displayed as probabilitydistributions.Treatment of riskRisk and uncertainty are added as alumped “contingency” based on theestimator’s historical experience andindustry guidelines (e.g., AACEI2003).Risk and uncertainty are explicitlyand quantitatively identified,characterized, modeled, andaggregated probabilistically.Risks are added probabilistically.Riskmanagement/responseRisk management is usually based on aseparate risk register, using historicalexperience.Risk management can be focusedon the higher level risks that areidentified and quantified by thismethod.Other high level risksFinancial, schedule, and other risks areidentified, characterized, andquantified “approximately.”Significant high-level risks may not beincluded or addressed.Financial, schedule, and other riskscan be explicitly identified,quantified, and prioritized for riskresponse.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 6 of 176
3.6.2Key Considerations of Deterministic Versus Probabilistic Cost EstimatesCost estimating using a deterministic process could significantly underestimate potential costs by:1. Misapplication of “contingency factors”2. Neglect of variability in prices and quantities3. Lack of appreciation of the impacts and probabilities of potential risk events4. Including additional (non-explicit) contingency in base costs and the overall contingency5. Overestimating the total cost of upper levels of ranges in the range-estimating approachA risk-based, probabilistic cost-estimating process inherently identifies more detail regarding risks andopportunities and can generate more useful information about the characteristics of uncertain events. Riskbased methods can better quantify the range of potential costs by more detailed characterization of risk andopportunity and the inclusion of conditional, dependent, and inter-related risk cost results. This can lead tobetter strategies in the bidding phase (to secure the project) and in the construction phase (to preserve profit).Risk-based, probabilistic methods can be more complex and time-consuming than deterministic methods,which are often based on a simple spreadsheet approach. The main reasons why a probabilistic approach isrecommended can be summarized as follows (Tecklenburg 2003).1. A deterministic method can produce the same value for risks that have a low probability ofoccurrence and high impact as those risks which have a high probability of occurrence and lowimpact by using a simple multiplication of probability and impact. This approach is probabilisticallyincorrect and can have a large impact on the accuracy of results and an appreciation of potentialconsequence.2. By multiplying the two elements of probability and impact, these values are no longer independent.Therefore, this method is not adequate for aggregation of risks where the characteristics of theelements need to be available. The only information that remains is the mean value of thecombination.3. The actual impact will definitely deviate from the deterministic value (i.e., the mean). Without theValue at Risk information, there is no way to determine how reliable the mean value is and howlikely it is that it might be exceeded.Bier summarizes the opportunities for probabilistic risk assessment as follows (Bier 1997):1. Probabilistic risk analysis allows reasonable modeling of deviations from normal (expected) valuesfor complex projects and systems.2. Probabilistic risk analysis can characterize any element or system performance, including theperformance of subsystems and their interactions.3. As a consequence, specific impacts from different interacting systems can be identified anddifferentiated.4. Probabilistic risk analysis delivers a quantitative risk estimation that can lead to better decisionmaking, risk response, and risk mitigation.5. Probabilistic risk analysis takes uncertainties into consideration. This is especially valuable ifstatistical data about potential impacts are sparsely available and large uncertainties dominate.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 7 of 17
ITA WTC 2015 Dubrovnik, Croatia3.7The Bandwidth ApproachThe bandwidth approach is one form of a range approach to address uncertainty. This method deals withuncertainty, but it is still an approximate method. In addition to “normal variability” in base costs, specificelements of the cost estimate can be characterized by ranges (plus and/or minus), expressed as percentages orvalues. The ranges are summed to indicate a range of probable cost. However, it is not accurate to do thisarithmetically as this will lead to aggregated maximum and minimum values that have an extremely lowprobability of occurrence and are therefore not realistic. This is because only the upper and lower bounds ofeach element are summed. Both these bounds are of extremely low probability when aggregated.The bandwidth approach is an approximate method. It can include all scenarios, however each will have thesame probability of occurrence. All probability information is missing. There are methods to address thisconcern, however they will only yield approximate results and rely on a sufficiently accurate determination ofthe interrelationships between the cost elements – something that probabilistic methods do explicitly, notapproximately.3.8The Square Root Approach to Aggregate Risk ValueThe Square Root approach is another form of range approach that results in one single figure – characterizedthe sum of all base costs plus the square root of the sum of the squares of the risk contingencies – as shown inthe following figure.Fig. 6 Approximate Treatment of Aggregated Risk Contngency (Sander 2014)ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 8 of 178
4.Cost Estimating for Construction4.1The Traditional Way – A Deterministic ApproachAs noted above, in the deterministic method, it is usual for cost estimators work up the cost for a definedproject (based on plans and specifications) using aggregated quantities multiplied by unit prices obtained fromhistorical projects using bid tab data, crew/activity analysis, and pricing data from current estimates or quotes.Usually there is a degree of conservatism (both implicit and explicit) built in to the quantities and prices,which can skew the result. To the aggregated units (summed quantities and prices) an overall contingency isadded to arrive at an “estimated construction cost” – a cost at a point in time with specific assumptions (oftennot explicitly or sufficiently stated or documented).5.Comparing Cost Estimating MethodsThe same input parameters from the following table are used, which shows inputs used for estimating the basecost of a simplified tunnel excavation and support element, in order to compare the above estimating methodsby means of a practical example. Quantities are used with a triangle distribution using a minimum (min), mostlikely (ml), and maximum (max) expectable value for each cost item.5.1Comparing Cost Estimating Methods – ExamplesTable 2 Deterministic Base Cost of an Excavation & Support Category Using Triangle UncertaintyDistributions (Sander 2014)DeterministicCost ItemQuantitymax unitminmlmaxCost/m eter oftunnel 20.71.225.91.633.747.1minmlShotcrete 10 cm, Top Heading13.8Steel Mesh AQ5013.8Unit Price [USD]Swellex 3.0 m, Top Heading1.715.41.8Shotcrete 5 cm - Bench5.25.86.6m²6.07.59.743.1Swellex 3.0 m - Bench0.40.50.5pc20.725.933.711.7ml 5.1.1"Mos t Likely Value"307.0Deterministic ApproachThe deterministic approach delivers a single figure (USD/ 307) that is the sum of the products of the mostlikely quantity multiplied by the most likely price.5.1.2Bandwidth ApproachThe Bandwidth Approach delivers three results as the sums of the following elements:1. Sum of all minimum quantities multiplied by all minimum prices2. Sum of all most likely quantities multiplied by all most likely prices3. Sum of all maximum quantities multiplied by all maximum prices5.1.3(USD/ 223.36)(USD/ 307.00)(USD/ 453.86)Square Root ApproachThe Square Root approach delivers one single figure which is the sum of all base costs plus the square root ofthe sum of the squares of the risk contingencies. Its value in this case is USD/ 376.ITA 2015, World Tunnel Conference, Cost-Risk PaperPage 9 of 17
ITA WTC 2015 Dubrovnik, Croatia5.1.4Probabilistic ApproachThe probabilistic approach combines base cost plus risk costs in a simulation. The result is a probabilitydistribution showing relative probability of a particular cost over a range.The following diagram, Figure 7, compares results of the above methods for the example given.Fig. 7 Visualized Result, Comparison of Discussed Estimating Methods (Sander 2014)5.2Assessment of Estimating MethodsTable 3 Assessment of Cost Estimating e single figureWell-known & acceptedQuickCan be performed “manually”Three values (minimum, mostlikely, and maximum) in a rangeQuickCan be performed “manually”Square RootOne single valueQuickCan be performed “manually”ProbabilisticFull probability informationITA 2015, World Tunnel Conference, Cost-Risk PaperConNo probability information for single valueNo VaR informationMore often than not on the unsafe side (high,unknown probability of cost overruns)No probability information for range valuesNo VaR informationMore often than not on the unsafe side (high,unknown probability of cost overruns)Range maximum and minimum very unlikelyNo probability information for single valueBandwidth information is lostNo VaR informationRange limits are extreme values and veryunlikelyNeeds probabilistic thinking & understandingNeeds (a little bit) more timeNeeds software supportPage 10 of 1710
6.Advantages of Using Risk-Based, Probabilistic Cost-Estimating6.1How a Better Cost-Risk Assessment Helps Even in a “Low-Bid” EnvironmentPrevious papers (Reilly 2008) have noted that in a “low-bid” environment each party enters a contract at theirown risk and the contractual environment is characterized by the ability of each party to treat the other partyas an adversary – for thei
The components of cost that need to be correctly addressed in the estimate include: 1. Base cost - the cost that will result if "all goes according to plan" (Reilly 2004) 2. Risk cost - the cost result of threat and opportunity events, if they should occur . 3. Escalation cost - cost resulting from normally expected inflation . 4.
Section 3, Cost Estimating Methods, discusses historical, conceptual, risk-based, and cost-based estimating methods and estimating software. Section 4, Cost Estimating Factors, discusses cost drivers and the impact that each has on the construction cost estimate throughout the project development process.
Electrical Construction Estimating Introduction to Electrical Construction Estimating Estimating activites will use the North State Electric estimating procedures. Estimating and the Estimator Estimating is the science and the art by which a person or organization determines in advance of t
GAO Cost Estimating and Assessment Guide -The "12-Steps" 1. Cost estimating (CE) purpose 2. Cost estimating plan 3. Define program characteristics 4. Determine estimating structure 5. Identify ground rules and assumptions 6. Obtain data 7. Develop point estimate; compare to ICE 8. Conduct sensitivity analysis 9. Conduct risk and .
3 cost estimating and management practices 6-7 cost management principles estimating formats 4 estimating requirements 8-17 general warm-lit shell versus tenant improvement (TI) cost estimates . b estimate tracking sheets 71 c uniformat project cost summary 72 d building cost analysis forms 74-75
OPERATING COST ESTIMATING GUIDELINE – WATER AND SEWER UNCONTROLLED IF PRINTED OR SAVED Document: QDS101 – Operating and Maintenance Cost Estimating Guidelines Revision: 2.0 Page 8 of 14 6 Cost Estimating Examples The following examples of Options Analysis – Cost Effectiveness Analysis have been included as
Estimating Time-Lines for Total Costs . 23 Estimating Time-Lines for Total Costs . 24 Estimating Annual Costs . 25 The cost model will accommodate costs in non- . worksheets as listed in this slide. Also indicated are the number of individual cost line items arranged in worksheets corresponding to SWBS Groups. These
deterministic polynomial-time algorithms. However, as argued next, we can gain a lot if we are willing to take a somewhat non-traditional step and allow probabilistic veriflcation procedures. In this primer, we shall survey three types of probabilistic proof systems, called interactive proofs, zero-knowledge proofs, and probabilistic checkable .
find on software development processes, which led me to Scrum and to Ken Schwaber’s early writings on it. In the years since my first Scrum proj ect, I have used Scrum on commercial products, software for internal use, consulting projects, projects with ISO 9001 requirements, and others. Each of these projects was unique, but what they had in common was urgency and criticality. Sc rum excels .
