Best Practice Guideline Software Release - ZVEI

Release process from the point of view ofthe OEM – time sequenceThe integration of software into the system consists of three different integration stages. Thefirst stage is the integration on the componentlevel, and then the integrations at the subsystem and system level are performed. Testingcan be started consecutively but run in parallel.Once the quick checks of an integration stagehave been completed successfully, the nexttest stage can be started. Additional deliveriesof software are only permitted for show stoppers. The system freeze is done at a previouslydefined point in time. From this point in timeonwards, no additional deliveries are permittedand the concluding tests are carried out (seefigure 5).Release process from the point of view ofthe OEM – TheoryIntegration stages with defined functionalextensions are planned by the OEM. At the startof the development, the intervals are longer,between two and four months. Shortly beforeSOP, the functional extensions are no longer ascomplex and come at intervals of 2 to 6 weeks(see figure 6).Release process from the point of view ofthe OEM – RealityBug fix loops are pushed between the integration stages through unplanned bug fix measures. This can be traced back to the fact thatbug fixes are delivered additionally, regardlessof the planned timeline. Bug fixes and functional extensions are often not separated inpractice.Capacities for further development and validation may be factored in for these unplannedbug fix loops. Unplanned loops may delay thefinal software release. At the same time thereis a probability that the comprehensibility andtransparency of the actions carried out, will suffer due to the loops which are slotted in (seefigure 7).Release process from the point of view ofthe OEM – Best practiceIn reality errors are to be expected at every integration stage. Therefore bug fix loops shouldbe planned right from the start. This ensures ahigher quality and transparency in the softwaredevelopment process. The same amount of timeshould be planned for the additional bug fixloops for each iteration (functional extension)(see figure 8).Additional delivery forshow stopperSF1SystemSRQCSystem function test with focus on the completesystem, side effects and characteristic Function testsComponent testtKey:SWLKFSF1SF2SRQCSoftware deliveryFreeze of componentsSystem freeze 1System freeze 2System releaseQuick checkAdditional delivery forshow stopperReleaseFigure 5: Release process: temporal sequence (picture source: ESG Elektroniksystem- und Logistik)8“System Release”

Functional extensionFunctional extensionFunctional extensionFigure 6: Release process from the point of view of the OEM: Theory (picture source: ESG Elektroniksystem- und Logistik)Functional extensionBugfixFunctional extensionBugfixFunctional extensionFigure 7: Release process from the point of view of the OEM: Reality (picture source: ESG Elektroniksystem- und Logistik)Functional extensionFunctional extension BugfixFunctional extension BugfixFigure 8: Release process from the point of view of the OEM: Best practice (picture source: ESG Elektroniksystem- und Logistik)9

2.4. Impact of externally developed softwareAutomotive software releases increasingly contain software from multiple suppliers, evenfrom a cascade of suppliers. This extends theclassic relationship between the OEM as customer and Tier-1 as supplier in several aspects.On the one hand, a supplier has to assume theperspective of an OEM, when he integrates software from a Tier-2 supplier into his own component. On the other hand, an OEM also hasto assume the supplier perspective, when heprovides software that a supplier integrates intoa software package or into hardware. For thesoftware release, there is an important difference whether the integration of external software is ordered by the customer or if it is a freedecision of the supplier. The responsibility forreleasing such external software parts should beclarified between customer and supplier beforethe first delivery.In a cascading sequence of software suppliers, ideally the software requirements relatedto quality, maturity, development processes,timing of delivery should be forwarded to eachsupplier on each level. In practice, this forwarding is limited. For example: Commercial-off-the-shelf (COTS) softwarecomponents, e. g. AUTOSAR basic softwareFor the most part, COTS software has to beintegrated as is. Quality assessments atthe supplier may not be possible. Softwarechanges can be difficult with regard to content, timing, or even in general. Desiredrelease documentation and artefacts (cf.chapter 3) may not be provided by the supplier in a format and with the content thatcan easily be integrated into the overall documentation of the Tier-1 supplier. In order toaddress these uncertainties and to mitigatecorresponding risks the integrator of suchsoftware has to transform the information ofthe COTS provider or even has to add appropriate quality assurance measures.10 Open source softwareAlthough for open source software the sourcecode is fully transparent to an integrator, theimplications are quite similar and even morelikely than with COTS software. Open sourcesoftware is often maintained by a community,so that the availability of any needed updatesis not assured. Even a reliable issue reporting is often not guaranteed. Additionally, anassessment of the development process of anopen source component usually is not feasible. Such implications should be clarified withthe customer, even if the customer requestedthe usage of this open source component andeven if it is the OEM. A very important issue isthe license type of open source software. If itis a strong copyleft license (GPL) for example,all code has to be shared. Due to this reason,open source license information is an important part of the release notes. Proprietary software (e. g. functionalsoftware from OEM)Such software often contains innovative functions and thus is linked to specific intellectual property. To protect this, an integratordoes not often have transparent insight intothe source code, e. g. if he is requested tointegrate pure object code. In that case, thecapability of the integrator and consequentlyhis responsibility is limited to the integration purpose. Depending on the scope of theintegrated software and its functionalities,the functional responsibility stays with thesupplier of the component. The distributionof these responsibilities should be clearlydefined in the contractual relationshipbetween customer and supplier. In practice,this is particularly important for an integrator if an OEM assumes the additional role ofa software supplier, beneath the role of thetop-level awarding authority (customer).

2.5. Impact of configurable softwareConfigurable software is a key principle to realise ever growing software content with reasonable effort and quality or to enable reuse. Butconfiguration of software brings along somedrawbacks that have to be addressed in the context of releasing and providing software.One example is AUTOSAR basic software withthousands of parameters including configuration parameters that significantly influence thebehaviour of the software. A similar case is aplatform software of a supplier, which is developed to be used in many projects for differentOEMs.Differentiate: Configuration by supplierPart of the configuration is done by thesupplier before delivering the software.This restricts the configuration freedom forthe integrator/customer – we could call it“pre-configuration”. Configuration by integratorPart of the configuration is done by the integrator/customer (OEM who integrates an ECUinto a vehicle variant or Tier-1 who integratessoftware components into an ECU).Configuration leads to functional variants andthe main challenge is to adequately test thevariants, because testing of all possible variantsmay not be possible with a reasonable effort.Different strategies can be used to meet thischallenge that are not discussed here. Regarding software releases at the interface of different organizations, maximal transparency is themajor goal. This will be dealt with in chapter 3.11

3. Documentation and ArtefactsThis chapter provides a description of supporting documents accompanying the delivery of arelease item.3.2. Detailed release documentation3.2.1. Delivery scopeThis document supplies a comprehensive overview of the software components or the controlunits. This includes an overview of variants e. g.in the form of a matrix, which contains all thenecessary information for the integration of thecomponent. The 3rd party software componentscontained in this delivery also have to be listed,independent of whether they are OEM standardsoftware, open source software, or any kind ofproprietary software.3.1. Overview documentPractice shows that an overview document provides the best access to release documentationfor the customer. The following aspects aresummarised in this document: A short description of the release item, purpose of use (construction phase, test run,intermediate release .). Using a clearlystructured nomenclature in the release designation enables the distinction between mainreleases and bug fix releases. In addition it isuseful to state in the nomenclature, whetherthe software has interface compatibility withits predecessor or not. Overview of the documents delivered, overview of the documentation. Project schedule with reference to the currentphase in the project (A, B, C sample) Short description of the agreed standardtimeline or the process model for a release(see figure 9).Network description n-19-18-17-16-15-14Preliminary freezeall requirements knownRelease n-1-13-12-11-10-9The information concerning software version,configuration files, flash bootloader, memorydata (RAM/ROM), interface description, HWetc. denoting a software release item must bestated clearly and in detail here. If required,parameter sets and diagnosis data inputs arealso described.Furthermore it is reasonable to describe theexact data of the “build environment” used,for instance compiler versions. In Table 1, thematrix description of the B1 sample stage ofa control unit is given as an example. Similarpresentations can also be used for pure software deliveries. In particular with 3rd partysoftware, it is important to also mention thelicense (see Table 1).Network description n 1-8-7Freezeall requirements discussedNo of weeks before delivery to OEM-6-5-4-3Basic functionalityapproved to OEMDelivery of the components-2Calibration120123456Ability to drive status to OEMProgramming / integration / testFigure 9: Example standard procedure software release (picture source: ZF Friedrichshafen)-1Release nJoint clearanceOEM/supplierSW test HILSW test vehicleTest OEM7

Sample statusVariantTier-1Part no.1. Flexray singleXY2. Flexray x4XY3. Flexray x8XYOEM LU no.(delivery scope)OEM ZB no.(assembly)IdentificationB1SWunique identification for delivered softwareversion (e. g. software part number)HWunique identification for delivered E/Ehardware version (e. g. E/E hardware partnumber)MECHunique identification for delivered mechanichardware version (e. g. number and index ofdrawing)DBC file OEM.ECU file name.Standard SW packageOEM.Flash bootloader – status.SW article codeExtended SW article code.Table 1 : Example of release information for a control unit:3.2.2. Subsystem descriptionIn addition to the unique identification of therelease item in the delivery scope, a clear reference to the subsystem or control unit levelis added for a software release item. This isespecially helpful for the customer if he wishesto check functionalities on a release item andwishes to have the corresponding frameworkconditions available quickly.The following information is part of the subsystem description for example: Circuit diagram of the control unit Interface description of the control unit orsubsystem Block wiring diagram Connector description3.2.3. Change logThe task of the change log is to give the customer an overview of the modifications of therelease item. A central element is a listing of thesoftware changes and bug fixes with reference tothe previous release item. The changes will ideally be exported from the workflow managementsystem in order to avoid consistency problemsof software and documentation. It is sensibleto reference the document “Delivery scope” inorder to document the compatibility of hardware, software and tools clearly.This important block of release documentationcan frequently be transferred from one releaseto the next release. Information on the compatibility of the software release item to varioushardware states must also be documented forchanges in the system or control units HW.13

It is useful to make a distinction between newor modified requirements and bug fixes whichare implemented. Tables with the following columns as categories are commonly used: Consecutive number Unique supplier change ID (reference to supplier‘s change management system) Headline of the change Change type (requirement or bug fix) Unique customer change ID (reference torequirements or problem management system of the customer)3.2.4. Function listThe functions to be implemented for a releaseare specified during release planning andstated in the function list. They are referencedto the relevant requirements documents. Further important information is whether theplanned function was implemented completely.If it was only implemented in part, a statementof the resulting limitations is added. If not allthe variants that can be activated by configuration parameters are implemented in a specificrelease, this is also documented.The trend towards ever finer, more granularreporting, is a challenge which can extend tothe level of individual requirements undercertain circumstances. An agreement with thecustomer about a suitable depth or an average granularity minimises time and effort. Areference to superordinate functions or function groups may thus be appropriate. The term“Feature” is used for this at many points.A requirement specification of good quality andstability, supports the creation of an informative function list. In case of a poor or volatilerequirement specification quality, the importance of the function list for obtaining an overview of the functionality increases.Bug fixes are already shown in the change log.The function list additionally shows faults whichare known but which have not yet been fixed(“known issues”). This increases transparencyand reinforces confidence.14It also makes sense to create metrics for thefunctional extensions between the releases.How does the number of requirements developover the project? Are the agreed functionalextensions achieved? How great are the deviations?Figure 10 shows the degree of implementationand fulfilment assumed for a notional projectprogression. Here a distinction is made betweenrequirements, which were planned and implemented for the release, and requirements whichwere planned but could not be implemented.3.2.5. Configuration parametersConfiguration parameters can either be parameter set at build time in a makefile configuration or a post build time calibration parameterset. A list of configuration parameters that areintended to be used by the integrator is documented. A detailed description of the effecton the functionality is not part of the releasedescription, but rather of the detailed technicalspecification. A list of the changes in comparison to former releases increases readability.3.2.6. Verification resultsThe purpose of this document is to make theverification results of the supplier available tothe customer in a suitable way. The customerand supplier agree which test methods and testend criteria are to be used in the project rightat the start of the project.It is necessary to agree to a suitable abstractionlevel for the verification outcomes. This minimises time and expense for the customer andthe supplier and safeguards know-how. Normally it is sufficient to report the test coveragewith reference to the customer requirements(see also section 3.2.4). The detailed test resultsare only then made available, if this is agreedon contractually. A good compromise ofteninvolves making it possible to view detailedresults without these being passed on.In case of configurable software, the supplierstates which sets of configuration parametershave been verified in the current release item.If the test level is different over the range ofvariants this is documented (e. g. full test forthe standard variant and only test of standardbehaviour – “Geradeauslauf” – for other variants). It could be useful to add informationabout those sets of configuration parameters

140012801200130013001100 tednot implementedplanned for 20D0ReleaseFigure 10: Implementation status requirements specifications (picture source: Leopold Kostal)that have been tested in former release items.For the software release, the expectations onthe verification test of configurable softwaredepend on the configuration time point. If thesoftware is configured by the supplier, completetest coverage for this configuration is expectedby the OEM. If the software is configured duringruntime by calibration values, the verificationhas to be performed by the integrator with thisspecial dataset.Notes: The defined document scope can deviate,depending on the project phase. In earlyphases the document scope may possibly beincomplete or adapted. In the event of several deliveries and recursion loops for a release item, it may be beneficial to carry out a delta analysis.3.2.7. MetricsMetrics assist in the all-round evaluation of asoftware release item. This makes them an element of quality management. In many casesit makes sense to document the same metricsover the project progression so that trend statements can be derived from them. Therefore, thecareful definition of the metrics at the start ofthe project is important. Any later modificationmay require recalculations and in any case,make statements on the long-term trend moredifficult.The following metrics are commonly used: Number of the function modifications implemented (“functional extensions“) Number of bug fixes Number of faults which have not been fixedor open points (“known issues”) Test coverage with reference to the requirements Test coverage with reference to the code created (e. g. “function coverage” or “code coverage”) Test coverage with reference to configurationparameters (e. g. coverage of functional variants) Metrics for evaluation of the product quality(e. g. MISRA or HIS) Maturity IndexThe maturity index is a very useful tool fordetermining the product quality or productmaturity during product development. Ittakes problems and modification wishes intoconsideration depending on their degree ofdifficulty and processing status. Details aboutthis can be found in the glossary. Resource usage (with regard to RAM, ROMand runtime, see figure 11)15

ROM usage (data section)600500Kbyte400Sum300Usable85 %2001000Figure 11: Example of resources consumption actual/target (picture source: ZF Friedrichshafen/ZVEI)3.2.8. Releasing SoftwareWith this document, the release process of theinvolved development departments is summarised in a multi-stage process (see figure 12).Releasing a software release item for a defineduse is declared by the authorized

2.1. Software release Software release means that the software is cleared to be passed on to the user or customer. With the release of software, the supplier gives a statement about the implemented functions and properties and hands them over to the cus-tomer within the definedframework for use. Release of the software typically results in the