'Leveled' Conformance Tests - A Must For Interoperability
'Leveled' Conformance Tests - A Must for Interoperabilityin Networked SystemsDr. Wolfhard Lawrenz, Frank Fischer, Karsten Hoffmeister, Maria ScheurerModular design and networked solutions are the basic answers to master complexityof today’s and future systems. Complexity will even grow dramatically as the desiredfunctionality by the end user is increasing drastically. A basically efficient approach toachieve proper interoperability in distributed systems is to apply standard 'layers'onto which the application itself is built, to specify standard layers as high as possiblewhile minimizing the individual application parts and to check the functionality ofthese layers by sufficiently efficient conformance tests. In automotive and otherapplications these 'standard' layers comprise – from the lowest to the highest level –the 'transceiver' layer, the 'network' layer, a 'device driver interface' layer and the'operating system' layer, which currently is the final interface to the application. Inautomotive applications the communication layers mostly are based on CAN protocolcomplemented by CAN software drivers interfacing the operating systemOSEK/VDX.This stack of 'standard' levels provides a powerful and neutral interface tothe application. But experience over the last years has shown that theimplementations of the individual levels differ in their behavior. This is due to severalreasons such as the specifications of the individual levels are not precise enough,they are ambiguous, they difficult to implement because of their complexity. Even ifthere formal and executable specifications for instance based on state machinelanguages, implementations show defects because the implementers try to optimizeand thus they violate unknowingly for instance the constraints related to localvariables.c&s group has worked in the field of conformance tests since more than 5 years andthus has contributed for instance to the development of the ISO CAN Conformancetest standard. As a complement to the ISO standard c&s has specified andimplemented conformance tests for the CAN register interface and for the related'higher layer' software drivers. Furthermore c&s is cooperating in a group of automanufacturers – Daimler Chrysler, BMW, Audi, Volkswagen, PSA – to specify andimplement conformance tests for a new generation of fault tolerant CAN transceivers.Tests for the communication and network management part of OSEK/VDX have beenspecified and implemented. The OSEK tests go beyond the 'normal' test methodswhich typically check all possible transitions from each state to its neighbor states,because this neglects some typical implementation 'faults', which are due to someoptimization side effects.Looking at the 'tests chain' comprising all the levels being involved to build the bridgefrom one application on one module to the other modules a high degree of confidencein the conformance of the whole chain is provided. The present paper discusses theconstraints given by the distributed system in conjunction with the philosophy how toderive tests on the various levels, how to implement the tests and typical testproblems and achieved results.
IntroductionComplex systems in cars grow at a veryrapid pace. Enhanced performance suchas lower fuel consumption, less pollution,better safety, enhanced comfort requiremore electronics. Even medium sized carscurrently have more than 30 electroniccontrol units – ECUs -, in high end carsthere are even more than 100 ECUs. Inpower train control they serve for enginecontrol, gear box control, brakes control,stability control, etc. In the body controlarea the ECUs do seat positioning, mirrorpositioning, power windows control, etc.The multi media applications support radiofunctionality, TV, telephone, CD player,navigation systems, etc. In current carsthere are multiple intelligent air bagcontrols. In the near future there will besteer by wire, brake by wire and otherintelligent functions.All this complex functionality requirescommunication between the various ECUsin order to optimize or even enable theoverall control loop of the car as a system.Communication typically is performedthrough multiple networks performingdifferent application oriented protocols.Obviously predictable and even the safeoperation of the complex 'system car'would require good specifications of thecomponents at the various levels as wellas sufficiently well specified tests. Theserequirements would be even more munication-links'betweenthe modules.But very often the specification of thecommunication components at the variouslevels is not precise nor complete. In mostcases the specification does not exist in aformal language. Therefore formal checkson completeness, in-ambiguity, etc. cannot be performed. Due to this there is noway to derive formal verification tests.Tests to check the conformance of suchstandard components are derived fromexperience, enhanced by systematicconsiderations to optimize the tests and tominimize the amount of tests to beexecuted. Very often the tests in turnbecomethespecificationofthecomponent – which may be the wrong wayto govern complexity of large systems.All this is due to the fact that systems growso rapidly. Because of the competitionrace there is no time in the predevelopment phase for systematic formalapproaches. Even worse, in order to haveadvantages in front of the competitionimportant parts of the specification ofstandard components are hidden and notpublished. Interoperability then can onlybe guaranteed by exhaustive conformancetesting, although this may be the wrongway.Subsequently the state of the art ofderiving conformance tests for aninsufficientlyspecifiedstandardcommunication component is given. Thetest cases were derived empirically andoptimized by systematic treatment. Thisconsequently would lead to a formal morecomplete specification of the componentitself. An iterative optimization approach isgiven balancing the tests and thespecification of the component.c&s masters the art of derivingconformance tests for an ent.Layered Communication ArchitecureProper function of complex systems incars is essentially depending on theconformance of multiply used standardcomponents. One of the mostly usedstandardcomponents/functionsiscommunication. Therefore interoperabilityof the communication modules beingsupplied by various implementers linkingthe ECUs into a system is a feature of veryhigh interest. Referring to the ISO-OSImodel a communication chain based one.g. CAN communication protocol [2], [3]couldconsistsofthefollowingmodules/layers:
OSI - COMMUNICATION - STACKConfigurationLayer 7OSEK / VDXOSOSEK / VDXNMApplicationOSEK / VDXCOM.Layer 3Layer 2"Protocol"Layer 1Fig. 1: car system archtitecture exchange them without an adaptation tothe higher levels like OSEK-COM. Ingeneral layer 2 shall provide services likeset up of data which shall be send,indicate received data, set transmitrequests and confirm transfer status. Thephysical layer is the interface between thesignal transmission over the physicalcommunication media and the digital'world' towards the higher OSI-layers.Layered Conformance TestsApplication, such as engine control orgear box control, an operating systemlikeOSEK/VDXOS,oranConfiguration VDXNetwork-ManagementOSI layer 6 . 3: may be voidOSI layer 2: CAN protocolOSI layer 1: CAN transceiver andphysical wiringA basically efficient approach to achieveproper interoperability in distributedsystems of high complexity is to applystandard 'layers' onto which the applicationitself is built, to specify standard layers ashigh as possible while minimizing theindividual application parts and to checkthe functionality of these layers bysufficiently efficient conformance tests.All layers provide their services to theupper layers and use services of the lowerlayers. The application, a configurationmanagement or an operating system maybe the 'final' user of the OSI layer 7. TheOSEK-Network Management of layer 7provide to the 'final' users only local(node-related) and global- (networkrelated) management methods (e.g. startup network or management of differentmechanisms for node monitoring), theOSEK-Communication provide the 'real'communication functionality for inter-ECUcommunication (communication withinelectronic control units) and also the ECUinternal communication (communicationwithin electronic control units). To providethis services the layer 7 uses the lowerlayer services which are provided directlyper the data link layer (layer 2) or per ahigher layer (e.g. layer3). The layer 3 to 6may be void but there can be anotheranother higher level protocol (e.g. sessionlayer of time triggered CAN). In any casean adaption of the communication layer(layer 2) to the higher layers must exist.The data link layer services are notstandardized in case of CAN. This is aproblem because CAN implementations ofvarious implementers are too different toApparently the communication link is acrucial factor for the proper function of thewhole system. Although there arestandardized communication protocols theinteroperability of the communicationmodules may be questioned as thespecification of the standards may bepartially not strong enough as to ensurethatdifferentimplementationsaresufficiently interoperable. The functionalityof a component is controlled by thespecification of the component. Thespecification is mostly given in naturallanguage, which may be ambiguous andnot precise and not complete enough.Often some 'formal' specification meansare applied partially for instance statediagrams. But for various reasons such asprotecting knowledge–intellectualproperty IP – in front of the competitorseven these formal description parts are notsufficiently deeply specified. This willinglytakes the risk or even the sureconsequence into account that differentimplementersoftheverysamespecification result into devices whichbehavedifferentlyundercertainconditions. This of course is crucial wheninteroperability of 'standard' components isassumed, being supplied by differentcomponent manufacturers.
Baring in mind the constraint ofinsufficiently specified components andthe requirement that these componentsshould be interoperable, one way out ofthis dilemma is to apply 'sufficientlyexhaustive' conformance tests to verify thedesired interoperability of components. Asno sufficiently detailed and formaldescription of the component exists a socalled black box test technique isapplicable. Test stimuli are applied fromoutside of the component – which isreferred to as Implementation under TestIUT – to the accessible inputs, theresponses are read from the outputs of theIUT and compared to what is understoodfrom the specification.In a first step in order to derive a firstguess of a sufficiently exhaustive set oftest cases an empiric driven selection of aset of test cases, representing practicalconditions under which a component lateron would work, is a recommendablepractical approach. In a second step theindividual elements of the above set areorganized in such a way that they eachwould result as the parameterized productof a set of – mostly desired – orthogonalvectors. These vectors may be definedapplication driven. The parameterizedproduct of the vectors defines the socalled System Operational Vector Space –SOVS, comprising all so far knownpossible operational conditions underwhich IUT is expected to be functional inthe specified way. In a third step a subsetof the SOVS is selected empirically, drivenfrom application experience, or evenformally. This subset is then expected tobe Sufficiently Exhaustive Minimal Set of(conformance) Test Cases SEMSTC onone hand while comprising not too manytests cases on the other hand resulting ina still acceptable test time.Obviously the approach mentioned aboveis a practically viable procedure to derive asufficiently exhaustive set of test cases.But this procedure of course incorporatesthe risk that the selected subset of testsdoes not cover sufficiently the behavior ofthe component. There are several ways tooptimize the selection of test cases – seebelow: The selection of the of SEMSTCshould be supported by formalmethodsThe application of the SEMSTC inconjunction with the practically gainedexperience that in real applicationcases will be detected that testedcomponents will show deficiencies intheir desired interoperability will lead toan iterative optimization processinfluencing the re-specification of thetest cases and the component. Thenewly detected problem is analyzedleading to:A redefinition of the SOVS, theirparameters and SEMSTC in such away that the new case systematicallyis a member of the sets mentionedabove.A redefinition of the specification of thecomponent, leading to a more precise,more detailed, more formal result,without necessarily being too stringentto the implementers. Based on that'more' formal methods could beapplied to derive tests which wouldthen lead to a better SEMSTC. Theblack box test method would then turnmore into a gray box test technique.OSEK/VDX NETWORK MANAGEMENTTESTS - In the scope of OSEK/VDX theNetwork Management system (NM)provides standardised features whichensure the functionality of inter-networkingby standardised interfaces. The essentialtask of NM is to ensure the safety and thereliability of a communication network forECUs. According to the car systemarchitecture (Fig. 1) NM comprises thefollowing standardised interfaces: Configuration/OS/Application OSEK/VDX NMOSEK/VDX NM OSEK/VDX COMOSEK/VDX NM communicationdriver/mediumc&s has developed a test system fortestingtheconformanceofimplementations of the OSEK/VDX NMconcepttotheOSEK/VDXNMspecification. The development comprisedthe following steps:
derivation of the test casesplaning and realization of the testenvironmentimplementation of testerautomation of test execution andverification3.2.1.2Transition to NMBusSleep: NMNormal / NMActive / Initiator Node of Residual SystemInitial StateSystemConfiguration- Number of Nodes in ResidualSystem- CompositionNet Management Settings- Operating Mode- Local Management SettingsStates of Net Management- Main State- Parallel StateResidual System IUTDerivation of Test Cases - Thespecification of OSEK/VDX NM is given asa state machine. Fig. 2 shows an example.Test StepsResponse--: 1: residual system (c&s node) IUT: 1: networkstatus.bussleep 0: NMNormal: NMActive: logical ring existingnetworkstatus.bussleep(residual system) 0residual system GotoMode(BusSleep)IUT GotoMode(BusSleep)IUT must send next ringmessage with bit sleep.ind 1verification NMBusSleep NMActiveafter TWaitBusSleep after receiving ringmessage with bit sleep.ack 1NM must call ApplBusSleepstatus byte according tospecificationNM must call ApplRingTimeElapsedaccording to specificationFig. 3: test case 3.2.1.2 Transition toNMBusSleep: NMNormal / NMActive / InitiatorNode of Residual SystemFig. 2: part of OSEK/NM specificationA first approach to deriving the test casesis the coverage of each transition from onestate to another with at least one testcase. Further there is a need to test eachcondition which leads to such a transition.The test cases derived by this way are theminimum quantum of test cases whichmust be executed. This quantum of testcases is also commended by a groupwhich deals with methods and tools for thevalidation of OSEK/VDX - baseddistributed architectures (Modistarc).Fig. 3 shows an example for one testcase, which covers the transition from'normal state' to 'BusSleep state' under acertain condition.c&s however has perceived - andexperience has confirmed - that this'formal' methodology for deriving testcases is not sufficient: as implementerstypically try to 'optimize' by for instancesaving local variables for more than onestate environment, additional tests mustbe defined checking more than the directneighbors of the states. Therefore whenderiving the test cases sequences ofsuccessive state transitions must be takeninto account. Another important pointwhich must be considered when derivingthe test cases is the 'real' environment ofan implementation. How does the NMunder test cooperate with other NMimplementations? Which effects has ahigh bus load to the functionality of the NMimplementation under test? Which effectshave bus failures e.g. open wire or shortcircuit of a bus line to the implementation?The test specification developed by c&scomprises test cases which take accountinto all these point of views: each transition from one state toanother is covered by at least one testcaseeach condition which leads to such atransition is at least covered with onetest casetest cases considering at least the prestate n-1 when testing the transitionfrom state n to state n 1
testcasesconsideringthe'environment' of an implementation asbus failures, number of nodes in asystem, wrong data, etc.the tests c&s adapts its test system to theactual hardware of the processor on whichOSEK NM is implemented.Conclusion OSEK/VDX NM Tests - Due tothe enhanced derivation of test cases onthe firm basis of experience the testsystem developed by c&s detects morefaults than others. The test casesderivated by c&s not only check theconformance of an implementation to itsspecification. Therefore potential faultsand loopholes in a specification areTester Architecture - The testingarchitecture is a black box testingarchitecture: the functionality of theimplementation under test (IUT) isobserved and controlled at external pointsof the IUT, the details of the respective NMimplementation are not visible. The testerarchitecture is shown in Fig. 4.ManualAutomaticTest Systemother NodesSystem Under Testc&s Upper TesterSoftwareSupervisorECU 1Software (SW)NWMECU nApplication ServicesConfiguration ServicesOS/COM ServicesSoftware (SW)IOTX.IUTOStest coordinationtest verificationNWMIUTIOTXIUTNetwork rGeneratorGRIOCANCAN BusIO'sFig. 4: tester architecture for OSEK/VDXnetwork management testsThe node with the NM implementationunder test consists of a n under test and a specificUpper Tester software developed by c&s.The realization of the supervisorcomprises one node with a enhancedemulation of NM, a tester software tocoordinate the tests, a program forautomatically verification of the tester andgeneration of the test report, an userinterface for starting one/several/all tests,observing test execution and doing severalsettings. The residual system comprises adedicated number of nodes, eachcontaining a communication chip (e.g.CAN) with µC, an NM implementation anda specific software developed by c&s. Theimplementation of the tester was realizedwith future oriented tools such asMatlab/Simulink/Stateflow in conjunctionwith realtime hardware. In order to executedetected when deriving and executing thetest cases. Although the specification ofthe OSEK/VDX NM is given in a ratherprecise and nearly 'formal' form there maybe a need to complete the specification.This interaction between 'specifying animplementation'and'testinganimplementation' is exactly the way thatleads to a precise and tighter specification,which is the condition precedent tointeroperability.CAN PROTOCOL TESTS - The CANprotocol is a CSMA/CA (Carrier SenseMultiple Access / Collision Avoidance)protocol that is targeted for automotivereal-time applications. Many Europeancars use the CAN protocol for in-vehicledata communication.As shown in Fig. 1 the protocol module, inthis case a CAN protocol implementation,is part of layer 2 of the model; according tothis model the interfaces of the protocolmodule are the following ones:
upper services to: OSEK / VDX NMOSEK / VDX COMApplicationlower services of: physical layer (PMA/MDI - PhysicalMediumAttachment/MediumDependent Interface)The c&s group is worldwide no. 1 i
OSEK / VDX OS Configuration Application OSEK / VDX NM OSEK / VDX COM "Protocol" Layer 7 Layer 3 Layer 2 Layer 1 OSI - COMMUNICATION - STACK. Fig. 1: car system archtitecture Application, such as engine control or gear box control, an operating system like OSEK/VDX OS, or an Configuration Management. OSI
Leveled Reader Teaching Guides include detailed lesson plans and student practice. pages to ensure that students at all levels. master target comprehension skills and. strategies. Teachers can also use the Scott. Foresman Online Leveled Reader Database to search more than 5,000 Scott Foresman. leveled readers by level, content area, and
NFP121 5 Sommaire 1) Tests et tests unitaires - Outil : junit www.junit.org une présentation Tests d'une application - Une pile et son IHM Tests unitaires de la pile Tests de plusieurs implémentations de piles Tests d'une IHM Tests de sources java Invariant et fonction d'abstraction comme tests - Tests en boîte noire - Tests en boîte blanche
AP Biology Practice Tests 2 2020 2020 Practice Tests . AP Calculus AB Practice Tests ; 2 2020 . 2020 . Practice Tests . AP Calculus BC Practice Tests 2 2020 2020 . Practice Tests . AP Chemistry Practice Tests . 2 2020 . 2020 : Practice Tests AP Computer Science 2 2019 2020 Practice Tests . AP English Language and Composition Practice Tests : 2 2020
Initial conformance date: July 21, 2014 Extended conformance date: July 21, 2015 Further conformance date for investments in and relationships with legacy covered funds and foreign funds: July 21, 2016 (2017) U.S. banking or
In Conformance With Plan In Conformance With Plan Comp Plan Conformance: Comp Plan Conformance: Anticipated Date In Service: Anticipated Date In Service: B B Rating: Rating: Continued Continued Status: Status: 3,835.0 2,983.5 Prior Appropriations Prior Appropriations 6,275.6 5,349.7 Six Year Total Six Yea
*Penguin Young Readers are leveled by independent reviewers applying the standards 2 *Penguin Young Readers are leveled by independent reviewers applying the standards developed by Irene Fountas and Gay Su Pinnell in Matching Books to Readers: Using Leveled Books in Guided Reading, Heinemann, 1999.
Dec 12, 2013 · Figure 2, Schedule Example after Resource Leveling Because of the inherent complexity of resource-constrained scheduling algorithms, the project durations of resource leveled schedules can be 10, 20, or 50 percent longer than needed.[7] Even if a resource leveled baseline schedule meets the required project
Read grade-level prose, poetry and informational text in L1 and/or single words of leveled prose and poetry in English. Read grade-level prose, poetry and informational text in L1 and/or phrases of leveled prose and poetry in English. Read short sentences of leveled prose, poetry and infor
