Identifying Acquisition Patterns Of Failure Using Systems .
Identifying AcquisitionPatterns of Failure UsingSystems ArchetypesFinding the Root Causes ofAcquisition ProblemsApril 9, 2008Linda LevineBill Novak 2008 Carnegie Mellon University
Did You Ever Wonder, Why acquisition programs often violate the fundamental principle of spiraldevelopment by doing the riskiest development last? we continue to invest in failing acquisition programs long past thepoint that makes economic sense? “Win/Win” partnerships degenerate for no apparent reason?some of a program’s most critical risks or issues never make it theattention of the program manager? with all of the advanced cost estimation models that we have, large,critical programs frequently underestimate costs by up to 70%? NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University2
Purpose of this PresentationTo show how Acquisition Archetypes (based on Systems Thinkingconcepts) can help to avoid common counter-productive behaviors insoftware acquisition and development programsAgenda Systems ThinkingFeedback Loops and Causal Loop DiagramsSystems ArchetypesAcquisition ArchetypesSeeing the Bigger Picture and Breaking the PatternNext StepsNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University3
Acquisition DistilledFrom “I Want” to “I Got”OperationalNeedAcquirerAcquisition RFPPlanning Prep.Solicita- Source Program Leadership SystemTransitiontionSelection Insight / Oversight AcceptanceDeveloperPlanDesignDevelopIntegrate& TestDeliverNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University4
Why is Software-Intensive Acquisition Hard?Complex interactions between PMO, contractors, sponsors, and users Full chain of actions & their longer-term consequences are not clearHard to apply corrective actions when status is uncertainSignificant delays exist between applying changes and seeing results Difficult to control systems with long delays between cause & effectExample: Steering an aircraft carrierUnpredictable and unmanageable progress and results Limited visibility into real progress & statusComplexity of interdependencies has unintended consequencesUncontrolled escalation of situations despite best management efforts Misaligned goals can drive potentially conflicting behaviorsLinear partitioning is the standard approach to address large systems When systems have feedback between components that are partitioned, itmakes it difficult to see & address these interactionsExponential growth of interactions as size grows linearlyNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University5
Can Systems Trap Us into Behaviors?Inside a complex, dynamic system, people’s actions can be at the mercyof that system’s dynamics Housing: Fears of rising interest rates, falling home values, andforeclosures scare new home buyers away from adjustable mortgages. Asbuyers hold back, slower demand makes prices drop further. Moremortgages exceed homes’ values, so prices drop more—causing buyersto pull back even more.1 Politics: When a party comes to power, large donors try to influencelegislation, which corrupts lawmakers and produces biased laws.Perceiving that the party has become corrupt, the electorate abandonsthem, the opposition party comes to power, and the cycle continues 1from “Marketplace” report, American Public MediaNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University6
What is Systems Thinking?Systems Thinking is a method for analyzing complex systemsDeveloped by Jay W. Forrester at MIT modeling electrical feedback Also exists in economic, political, business, and organizational behaviorsUses feedback loops to analyze common system structures that eitherspin out of control, or regulate themselvesHelps identify a system’s underlying structure, and what actions willproduce which results (and when)Systems Thinking teaches us that:System behavior is greater than the sum of component behaviors “Quick fix” solutions usually have side-effects that make things worse True improvement comes from changing the underlying system structure NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University7
Time DelaysMuch instability and unpredictability of systems is due to time delaysTime delays obscure the connections in cause & effect relationships Side-by-side causes and effects would be “smoking gun” evidencePeople are inherently poor at controlling systems with substantial timedelays between cause & effectExamples: Over-steering a large ship that is slow to respond, so it weaves backand forth A thermostat controlling a low-BTU air conditioner that’s slow to cool,so the house temperature bounces between too hot and too cold Inability to determine which surface, handshake, sneeze, or coughresulted in an infectionNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University8
What are the Acquisition Archetypes?The Acquisition Archetypes depict the underlying structures of a set ofdynamic behaviors that occur throughout acquisition organizations Each diagram tells a familiar, recurring story Each describes the structure that causes the dynamicAcquisition Archetypes are used to: Identify failure patterns as they develop (recognition) Single out root causes (diagnosis) Engage in “big picture” thinking (avoid oversimplification) Promote shared understanding of problems (build consensus) Find interventions to break out of ongoing dynamics (recovery) Avoid future counter-productive behaviors (prevention)NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University9
Anatomy of an Archetype: Causal Loop DiagramsDepict qualitative “influencing” relationships (increasing or decreasing)and time delays between key variables that describe the systemShow relationship direction by labeling them Same ( ) or Opposite (-)to indicate how one variable behaves based on the previous variableConsist primarily of two types of feedback loops: Reinforcing – Changes to variables reinforce, X increases, Y increases Balancing – Changes to variables alternate, X increases, Y A Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University10
“Fixes That Fail” – Systems ArchetypeA quick Fix for a Problem Symptomhas immediate positive results, butalso has long-term UnintendedConsequences that, after a delay,worsen the original Problem Symptomas the Fix is used more xbased on “Fixes That Fail”“Firefighting” concept from “Past the Tipping Point”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University11
“Sacrificing Quality” – Acquisition ArchetypeOAs schedulepressureincreases andthe cycle Schedulerepeats PressureandOworsens. quality suffers Quality which reduceserrors. anderrorsOincrease ErrorsOBRAs schedule pressureincreases, processes areshortcut, quality suffers, anderrors increase—requiringmore re-work. However, rework consumes resources,which increases schedulepressure, and the cyclerepeats and worsens.S whichincreasesschedulepressure Rework requiringmorerework AvailableResources OHowever, reworkconsumes resources based on “Fixes That Fail”“Firefighting” concept from “Past the Tipping Point”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University12
“Firefighting” – Acquisition ArchetypeSEarlyDevelopmentActivities onNext ReleaseODesignProblems inCurrentReleaseORBResourcesDedicated toCurrentReleaseResourcesDedicated toNext ReleaseOToleranceforDesignProblemsOSProblemGapIf design problems in the currentrelease are higher than the tolerancefor them, more resources must bededicated to fix them. This reducesproblems, but now fewer resourcescan work on the next release. Thisundermines early developmentactivities which, after a delay,increases the number of designproblems in the next release.Sfrom “Past the Tipping Point”based on “Fixes That Fail”“Firefighting” concept from “Past the Tipping Point”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University13
“Shifting the Burden” – Systems ArchetypeImpatience makes the organizationchoose the Symptomatic Solution inthe first ySide-EffectA Symptomatic Solution temporarilysolves a Problem Symptom, whichlater recurs. Its repeated use over thelonger term has Side-Effects that makeit less and less feasible to use themore effective Fundamental Solution—trapping the organization into usingonly the Symptomatic Solution.B2SOFundamentalSolutionBased on “Shifting the Burden”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University14
“Bow Wave Effect” – Acquisition ArchetypeSSDevelopmentof chedulePressureR1OSystemRiskSystemR2 ModifiabilityTasks planned for an early spiral toreduce risk are postponed to a laterspiral, making near-term performancelook better. This increases risk insubsequent spirals by delayingrequired “risky” development for whichthere is now less available schedule toaddress potential issues, and lessflexibility in the system toaccommodate changes needed.B2ODevelopmentof ComplexFunctionalitySSAbility toIntegrateNewCapabilitySbased on “Shifting the Burden”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University15
Acquisition ArchetypesThere are many recurring patterns of behavior in software development andacquisition that have been modeled using Systems Archetypes and CLDs: Sacrificing Quality The 90% Syndrome Firefighting Requirements Scope Creep The “Bow Wave” Effect Feeding the Sacred Cow Underbidding the Contract Brooks’ Law Shooting the Messenger PMO vs. Contractor Hostility Robbing Peter to Pay Paul Staff Burnout and Turnover Longer Begets Bigger The Improvement Paradox.NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University16
Related Disciplines and ConceptsTragedy of theCommonsThe Acquisition Archetypesdraw on ideas and conceptsfrom a variety of differentdisciplines:Social anderEffectSocial ProofSocial TrapGame TheorySocial lScienceGame TheoryCounterReinforcerAcquisitionSocial alityFree RiderProblemPolitical ernalitySpiral ofSilenceNetworkEffectMoral HazardInformationAsymmetryAgency CostNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University17
The Bigger Picture/Breaking the PatternBy representing the underlying structure of a dynamic, AcquisitionArchetypes show where best to apply leverage to slow or stop it—forexample: Change negative dynamics into positive ones by running them backwards Slow the acceleration of unwanted reinforcing loops—“When you’re in a hole,stop digging”Each Acquisition Archetype has specific interventions for addressing itKnowing about these common counter-productive dynamics is the best wayto prevent themA clever person solves a problem.A wise person avoids it.-- EinsteinNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University18
Why Is This Approach Critical?Increasing complexity and acceleration in technical and organizational systemsLinear behaviors become nonlinear and unpredictable when combinedWe lack problem solving methods that serve a “whole systems” viewOur current tools and methods are well suited for handling detailedcomplexity—where there are many variables.Dynamic complexity refers to “situations where cause and effect are subtle,and where the effects over time of interventions are not obvious” (Senge, 1990,p. 71) When the same action has dramatically different effects in the short run & thelong run When an action has one set of consequences locally and very differentconsequences in a different part of the system, there is dynamic complexity. When obvious interventions produce nonobvious consequencesNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University19
The ChallengesPatterns & structural properties are hard to perceive & discern. Too much situationalflux; few are looking closely, broadly or over time.Problem solving strategies (for handling detail) are a poor match for handlingdynamic complexity, and provide false assurance requires a radical shift in point of view & new problem solving methodsWork-life values can run contrary to a systems view—with a focus on short term,bottom line, and stovepipes. Actions based purely on these values often result incounter productive behavior. We think we are doing the right thing, but our perspective istoo small or too short.Solutions that “sound good” but often backfire (insidious traps)—“results” focused—(tyranny of) consensus—low hanging fruitBalance tackling the fundamental solution and achieving results. Can you find “quick fixes” that contribute to the fundamental solution? Identify and address competing goals Align incentivesNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University20
Next Steps and Future DirectionsPattern Library of Acquisition Archetypes Eleven Acquisition Archetypes have been describedPlan to identify additional acquisition dynamics & root causesCollaborative Consulting Help customers identify program-specific, counter-productive behaviorsLearning Experiments Interactive “hands-on” exercises that demonstrate key dynamics insoftware acquisition programsAcquisition Archetypes Workshop “Improving Acquisition Practice and Avoiding Patterns of Failure”NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University21
Workshop: Improving Acquisition Practice andAvoiding Patterns of Failure2 day interactive workshop for acquisition practitionersGoals Introduce the systems thinking approachEmploy (example) acquisition archetypes to (1) convey commonacquisition patterns, (2) assist practitioners to apply andtranslate these archetypes into forms that illustrate failure patterns theysee occurring in their own program and context Elicit classic failure traps. Illustrate counter-productive behaviorsthrough short exercises with gaming and micro world managementsimulators Identify high-leverage “interventions” that can be used by a program torecognize, stop, and recover from the diagnosed acquisition failurepatterns NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University22
For Additional InformationUpcoming SEI TechnicalNote: Archetypal Patternsof Failure in the Acquisitionand Development ofSoftware-Intensive SystemsSEI -support/pof-intro.htmlNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University23
NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University24
Back-up Slides 2008 Carnegie Mellon University
Systems Archetypes -1Fixes that Fail A quick fix for a problem has immediate positive results, but itsunforeseen long-term consequences worsen the problem.Balancing Loop with Delay The current state of a system is moved toward the desired state thoughrepeated action, but the delay raises doubts about its effectiveness.Limits to Growth Initially rapid growth slows because of an inherent capacity limit in thesystem that worsens with growth.Shifting the Burden ("Addiction") An expedient solution temporarily solves a problem, but its repeateduse makes it harder to use the fundamental solution.Accidental Adversaries Two parties destroy their relationship through escalating retaliations forperceived injuries.NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University26
Systems Archetypes -2Escalation Two parties compete for superiority, with each escalating its actions toget ahead.Drifting Goals A gradual decline in performance or quality goals goes unnoticed,threatening the long-term future of the system.Growth and Underinvestment Investments in a growing area aren't made, so growth stalls, which thenrationalizes further underinvestment.Success to the Successful When two parties compete for a limited resource, the initially moresuccessful party receives more resources, increasing its success.Tragedy of the Commons A shared resource is depleted as each party abuses it for individualgain, ultimately hurting all who share it.NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University27
Reinforcing LoopDecreasesDecreasesNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University28
Reinforcing LoopIncreasesIncreasesNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University29
Reinforcing NDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University30
Balancing eworkNDIA Systems Engineering ConferenceBill Novak, October 24, 2007 2006 Carnegie Mellon University31
Acquisition Archetypes (based on Systems Thinking concepts) can help to avoid common counter-productive behaviors in software acquisition and development programs Agenda Systems Thinking Feedback Loops and Causal Loop Diagrams Systems Archetypes Acquisition Archetypes
LLinear Patterns: Representing Linear Functionsinear Patterns: Representing Linear Functions 1. What patterns do you see in this train? Describe as What patterns do you see in this train? Describe as mmany patterns as you can find.any patterns as you can find. 1. Use these patterns to create the next two figures in Use these patterns to .
1. Transport messages Channel Patterns 3. Route the message to Routing Patterns 2. Design messages Message Patterns the proper destination 4. Transform the message Transformation Patterns to the required format 5. Produce and consume Endpoint Patterns Application messages 6. Manage and Test the St Management Patterns System
4 WORKOUT A - UPPER BODY EXERCISE SETS REPS Pull Up 3 1-2 short of failure Push Up 13 -2 short of failure Inverted Row 3 1-2 short of failure Dip 3 1-2 short of failure Lateral Raise 3 1-2 short of failure One-Arm Shrug 2 per side 1-2 short of failure Biceps Curl 12 -2 short of failure Triceps Extension 2 1-2 short of failure Workout Notes:
Creational patterns This design patterns is all about class instantiation. This pattern can be further divided into class-creation patterns and object-creational patterns. While class-creation patterns use inheritance effectively in the instantiation process, object-creation patterns
Distributed Systems Stream Groups Local Patterns Global Patterns Figure 1: Distributed data mining architecture. local patterns (details in section 5). 3) From the global patterns, each autonomous system further refines/verifies their local patterns. There are two main options on where the global patterns are computed. First, all local patterns
Selection Defense Business Systems Middle Tier of Acquisition Acquisition of Services Major Capability Acquisition . Reference Source: DoDI 5000.80, Paragraph 1.2.b The MTA pathway is intended to fill a gap in the Defense Acquisition System (DAS) for those capabilities that have a level of maturity . Acquisition programs intended to be .
Case History #2. Failure Analysis of a Conveyor Drive Shaft Case History #3. Metallurgical Failure Analysis of A Welded Hydraulic Cylinder Case History #4. Aircraft Component Failure Analysis Case History #5. Cap Screw Assembly Failure Case History #6. Aircraft Engine Failure APPENDIX A: Summary of Fracture Mechanics Applications to Failure .
universiteti mesdhetar orari i gjeneruar:10/14/2019 asc timetables lidership b10 i. hebovija 3deget e qeverisjes 203 s. demaliaj e drejte fiskale 204 a.alsula histori e mnd 1 b10 n. rama administrim publik 207 g. veshaj tdqe 1
