BI-DIRECTIONAL GAP ANALYSES OF CMMI, AUTOMOTIVE

BI-DIRECTIONAL GAP ANALYSES OFCMMI, AUTOMOTIVE SPICE, AND INCOSETravis Foust, Matthew Ramirez, Regina DorowOctober 2 nd , 2017

BACKGROUNDCMMI for Development is a process reference model developed by the Software EngineeringInstitute (SEI) to be used by organizations developing systems (software or otherwise) tomeasure their maturity and/or capability.Automotive SPICE, or ASPICE, is a process reference model developed by organizationswithin the automotive industry to create a more automotive-focused reference model comparedto SPICE or CMMI. ASPICE is derived from SPICE.INCOSE Systems Engineering Handbook is a handbook developed by the InternationalCouncil of Systems Engineering to be a practical guide to Systems Engineering. INCOSEstandards are be used to evaluate individual engineers while CMMI and ASPICE are used toevaluate organizations. The handbook adheres to ISO15288.Objective - While CMMI and ASPICE are the two dominant sets of standards in the automotiveindustry, this study will attempt to understand the differences between all three of thesestandards. The results will be a valuable resource in determining what standard(s) should bemapped and adopted as best practice, required of suppliers, and/or used for internal appraisals.2

CMMIProject ManagementREQM (2)RequirementsManagementPP (2)ProjectPlanningSAM (2)Supplier AgreementManagementPMC (2)Project Monitoringand ControlIPM (3)Integrated ProjectManagementEngineeringRD (3)RequirementsDevelopmentTS (3)TechnicalSolutionVER (3)VerificationVAL (3)ValidationPI (3)ProductIntegrationQPM (4)Quantitative ProjectManagementRSKM (2)RiskManagementProcess ManagementMA (2)Measurementand AnalysisDAR (3)Decision Analysisand ResolutionPPQA (2)Process and ProductQuality Assurance3CM (2)ConfigurationManagementCAR (5)Causal Analysisand ResolutionSupportOPF (3)OrganizationalProcess FocusOT (3)OrganizationalTrainingOPD (3)OrganizationalProcess DefinitionOPP (4)OrganizationalProcess PerformanceOPM (5)OrganizationalPerformance Management

ASPICE4[1]

INCOSE5[2]

METHODOLOGY AND APPROACH6

THE PROCESS1.2.3.4.5.6.7.8.Become familiar with CMMI, ASPICE, and INCOSE.Conduct literature review to understand similar studies in the field.Define gap analysis scope and technique.Execute gap analyses.Review gap analyses.Analyze results.Determine and document conclusions.Develop appropriate proposed actions.7

ANALYSES VOLUME6 Analyses (bi-directional x3)ASPICEINCOSECMMI – 167 Specific PracticesASPICE – 261 Base PracticesCMMI8INCOSE – 110 Process Activities

EXAMPLE ANALYSIS – ASPICE TO CMMIASPICECMMISystem EngineeringProcess Group (SYS)EngineeringSYS.1 RequirementsElicitationSYS.1.BP1: Obtainstakeholderrequirements andrequests.9Search CMMIspecific practicesfor match toSYS.1.BP1.ASPICE SYS.1Requirements ElicitationASPICE BPCMMI SPRequirementsDevelopment (RD)SYS.1.BP1RD 1.1RD 1.1: Elicitstakeholder needs,expectations,constraints, andinterfaces for allphases of theproduct 1.BP6Coverage16.7%“ASPICE to CMMI” may also beinterpreted as CMMI coverage ofASPICE.

CHALLENGES1. Related practices may make assumptions and/or omit key actions INCOSEASPICEGapIdentify relevantregulations.Use expert to helpdetermine relevantregulations if needed.2. Similar practices may vary in specificity ASPICEDevelop software detaileddesign.GapMatchCMMIDevelop a design for theproduct or productcomponent.3. Identical practices may have different subjects. E.g. “Identify risks” in System Design does not map to“Identify risks” in Acquisition.10

RESULTS11

ASPICE TO CMMI Major Gap Areas Acquisition Supply Documentation Reuse Consistent gaps found in mostprocesses for ensuring consistencyof process and communicatingresults to impacted groups.12ASPICE to CMMI Gap AnalysisAcquisition Process Group Contract Agreement57.1%Supplier Monitoring100.0%Technical Requirements70.0%Legal and Administrative Requirements0.0%Project Requirements33.3%Request for Proposals0.0%Supplier Qualification60.0% 40.0%Supply Process GroupSupplier Tendering0.0%Product Release46.2% 28.6%System Engineering Process Requirements Elicitation66.7%GroupSystem Requirements Analysis62.5%System Architectural Design75.0%System Integration and Integration Test55.6%System Qualification Test71.4% 65.8%Software EngineeringSoftware Requirements Analysis62.5%Process GroupSoftware Architectural Design66.7%Software Detailed Design and Unit Construction 62.5%Software Unit Verification71.4%Software Integration and Integration Test55.6%Software Qualification Test71.4% 64.6%Supporting Process Group Quality Assurance100.0%Verification80.0%Joint Review50.0%Documentation12.5%Configuration Management88.9%Problem Resolution Management55.6%Change Request Management87.5% 66.0%Management ProcessProject Management90.0%GroupRisk Management100.0%Measurement81.8% 89.3%Reuse Process GroupReuse Program Management37.5% 37.5%Process ImprovementProcess ImprovementProcess Group87.5% 87.5%Total59.1%

CMMI TO ASPICE Major Gap Areas Project Monitoring, Integrated Project Management, and Quantitative ProjectManagement Verification and Validation Decision Analysis Process ManagementSome gaps in Engineering were found as a result of ASPICE’s higher level of specificity.CMMI to ASPICE Gap AnalysisMaturity Project ManagementLevel 2 Requirements ManagementSupplier Agreement ManagementProject PlanningProject Monitoring and ControlLevel 3 Integrated Project ManagementRisk ManagementLevel 4Level 5TotalQuantitative Project ManagementEngineering100.0%66.7%57.1%10.0%0.0% Requirements Development85.7% Technical SolutionProduct s ManagementMeasurement and Analysis62.5%Process and Product Quality Assurance 75.0%Configuration Management100.0%50.0% Decision Analysis and Resolution50.0%66.7%40.0%25.0%Causal Analysis and Resolution40.7%1347.5%0.0% Organizational Process FocusOrganizational Process DefinitionOrganizational TrainingOrganizational Process Performance100.0% Organizational Performance Management66.7%Total61.1%0.0% 29.1%0.0%0.0%0.0% 0.0%30.0% 53.3%7.9% 39.5%

INCOSE TO CMMI Major Gap Areas Architecture Definition System Analysis Operation Maintenance Disposal Supply Portfolio Management INCOSE to CMMI has the highestcoverage results in this study.14INCOSE to CMMI Gap AnalysisTechnical ProcessesBusiness or Mission AnalysisStakeholder Needs and Requirements DefinitionSystem Requirements DefinitionArchitecture DefinitionDesign DefinitionSystem al Management Project PlanningProcessesProject Assessment and ControlDecision ManagementRisk ManagementConfiguration ManagementInformation ManagementMeasurementQuality AssuranceAgreement ProcessesAcquisitionSupplyOrganizational Project- Life-Cycle Model ManagementEnabling ProcessesInfrastructure ManagementPortfolio ManagementHuman Resource ManagementQuality ManagementKnowledge 3.6%

CMMI TO INCOSE Major Gap Areas Project Monitoring, Integrated Project Management, and QuantitativeProject Management Causal Analysis and Resolution Organizational Process Performance and Performance Management CMMI to INCOSE gaps mostly involve self-evaluation and process improvement.CMMI to INCOSE Gap AnalysisMaturity Project ManagementLevel 2 Requirements ManagementSupplier Agreement ManagementProject PlanningProject Monitoring and ControlLevel 3 Integrated Project ManagementRisk ManagementLevel 4Level 5TotalQuantitative Project Management15EngineeringSupportProcess ManagementMeasurement and Analysis100.0%Process and Product Quality Assurance 100.0%Configuration Management100.0%80.0%100.0%50.0%30.0%20.0% Requirements Development 90.0% Decision Analysis and Resolution85.7% Technical Solution100.0%Product 8.6%Causal Analysis and Resolution50.8%85.0%100.0% Organizational Process FocusOrganizational Process DefinitionOrganizational TrainingTotal72.2%55.56% 69.8%57.14%42.86%Organizational Process Performance0.00% 16.7%0.0% Organizational Performance Management 30.00% 20.0%83.3%39.5% 62.3%

ASPICE TO INCOSE Major Gap Areas Acquisition and Supply Software Engineering Joint Review Problem Resolution Change Management Reuse Most of the ASPICE to INCOSEengineering gaps were found due toINCOSE not covering engineeringstandards specific to software.16ASPICE to INCOSE Gap AnalysisAcquisition Process Group Contract Agreement100.0%Supplier Monitoring80.0%Technical Requirements40.0%Legal and Administrative Requirements40.0%Project Requirements20.0%Request for Proposals25.0%Supplier Qualification80.0% 46.7%Supply Process GroupSupplier Tendering12.5%Product Release23.1% 19.0%System Engineering Process Requirements Elicitation83.3%GroupSystem Requirements Analysis87.5%System Architectural Design50.0%System Integration and Integration Test88.9%System Qualification Test71.4% 76.3%Software EngineeringSoftware Requirements Analysis62.5%Process GroupSoftware Architectural Design50.0%Software Detailed Design and Unit Construction0.0%Software Unit Verification0.0%Software Integration and Integration Test0.0%Software Qualification Test0.0% 19.1%Supporting Process Group Quality Assurance100.0%Verification100.0%Joint Review0.0%Documentation62.5%Configuration Management66.7%Problem Resolution Management0.0%Change Request Management0.0% 41.5%Management ProcessProject Management70.0%GroupRisk Management83.3%Measurement54.5% 66.7%Reuse Process GroupReuse Program Management12.5% 12.5%Process ImprovementProcess ImprovementProcess Group87.5% 87.5%Total45.0%

INCOSE TO ASPICE INCOSE to ASPICE has the lowestcoverage results of this study. The large quantity of major gapareas is mostly due to INCOSE usingbroad, systems engineeringstandards while ASPICE usesspecific, automotive engineeringstandards.17INCOSE to ASPICE Gap AnalysisTechnical ProcessesBusiness or Mission Analysis40.0%Stakeholder Needs and Requirements Definition 66.7%System Requirements Definition75.0%Architecture Definition33.3%Design Definition50.0%System n0.0%Maintenance0.0%Disposal0.0% 33.3%Technical Management Project Planning0.0%ProcessesProject Assessment and Control100.0%Decision Management0.0%Risk Management80.0%Configuration Management80.0%Information Management0.0%Measurement100.0%Quality Assurance80.0% 60.7%Agreement ProcessesAcquisition0.0%Supply0.0% 0.0%Organizational Project- Life-Cycle Model Management0.0%Enabling ProcessesInfrastructure Management0.0%Portfolio Management0.0%Human Resource Management0.0%Quality Management0.0%Knowledge Management0.0% 0.0%Total31.8%

RESULTS SUMMARY CMMI has the highest averagecoverage of the other standards(66.4%). ASPICE has the lowest coverage ofother standards due to its higherspecificity. INCOSE has weaker coverage ofother standards due its lack ofsoftware-specific processes.18Results Summary - Row to 0%INCOSE73.6%31.8%

CONCLUSIONS19

GAP ANALYSIS SUMMARY – CMMI, ASPICE, INCOSECMMI Unique GapsUnique Inclusions20 Supply Management(outgoing)Legal and AdministrativeRequirementsDocumentationManagement Process Performance Integrated ProjectManagement Quantitative ProjectManagementASPICEINCOSE Decision Analysis andResolution Life-Cycle ModelManagement Human ResourceManagement Knowledge Management Causal Analysis andResolution Joint Review Request for Proposals Reuse Program Management Software-specific engineering System AnalysisOperationMaintenanceDisposal

WHICH GAPS ARE IMPORTANT?Weighing the importance of each major gap identified will lead us to determiningwhich standard(s) should be mapped in Stages, adopted as best practice,required of suppliers, and/or used for internal appraisals.Some identified gaps may have little significance in our business objective whileothers are critical.Note that the weight of some gaps may differ for OEMs and suppliers. (E.g.supply management)21

NEXT STEPS1. Weigh the GapsWeigh each of the items identified in the Gap Analysis Summary. This shouldinclude all relevant engineering process leaders. These weights should be usedto help determine action items regarding which standard(s) should be mapped inStages, adopted as best practice, required of suppliers, and/or used for internalappraisals.22