Agile automated software testing into automotive V-Model process: A practical case Xavier Martin Artal Software QA Manager xavier.martin@idneo.es es.linkedin.com/pub/xavier-martin/6/a89/723/
Agenda Introduction Automotive Trends: Car Connectivity Car Telematics project Challenges Use Case Solution: From V-Model to Agile Testing Results and Conclusions
Introduction
What is this presentation about? Expose a practical case of adoption of Agile techniques in automotive testing Converge Spice automotive V-Model to Agile Spice V-Model Agile Present Technical Solution adopted: Automation Test Framework Discuss results and Agile adequacy to Automotive industry
Automotive Trends: Vehicle Connectivity
Car Telematics Car Manufacturers start to add 3G/4G capabilities Connectivity opens new opportunities to develop services for both clients and manufacturers Connectivity Services – – – – – Emergency Call Fleet Management Car Sharing Remote Car Diagnostics Stolen Vehicle Tracking (SVT) – WOTA Update – Dealer Services – User Premium Services
Car telematics: eCall Emergency Call Service for Europe U.E Council proposes eCall obligatory in European Cars for end 2017 Automatic call in case of accident or emergency will force car manufacturers to add IVTU to every new car for European Service Similar regulations for Russia, USA, BRA and PRC
Car Telematics Project Challenges
What is an iVTU? iVTU in Vehicle Telematics Unit - Electronic Unit in charge of granting 2G/3G/LTE connectivity to vehicles - Two Main processors architecture: DSPIC pseudo-real time processor handling CAN interfaces communication Telematics Processor managing Radio Communications and high level services Supervisor Micro, USB, Wi-Fi, Accelerometer, GPS,
iVTU Interfaces Black Box Interfaces: CAN-V USB On-Car Integration: ECM RADIO IVTU CAN-M AT (Debug) BCM DCM TSP CAN-V IVTU EMU CAN-M NAVI GDC USB USER
Project Challenges Embedded System – High dependency of Hardware Development Multiple Software Configurations for Multiple Hardware Variants 2 Car Manufacturers Number of releases – Around 70 Releases per year ISO 26262 Safety Requirements Continuous changes in requirements Specifications blurriness
Hardware-Software Variability JAPAN 2 HW 2 SW CANADA 1 HW 3 SW EUROPE 3 HW 9 SW 2 Brands RUSSIA 3HW 5 SW USA 1 HW 3 SW BRAZIL 3 HW 4 SW 2 Brands AUSTRALI A 1 HW 1 SW CHINA 4 HW 5 SW 2 Brands
Use Case Solution: From V-Model to Agile Testing
Automotive Spice & V-Model (1) Automotive SPICE : Comprises a set of assessment indicators of process performance and process capability Based on ISO/IEC ISO/IEC 15504-2 Defines several levels of adherence to standard Focus on Process V-Model: Extension of Waterfall Methodology Big Design Up Front Phases aligns with Automotive SPICE
Automotive SPICE & V-Model (2) Benefits: The facto industry standards Defines a clear process life-cycle V-Model emphasis on testing, and particularly the importance of early test planning Drawbacks: The V-Model reflects a project management view of software development and fits the needs of project managers, accountants and lawyers rather than software developers or users. Inflexible and slow to respond to change. Process Overload Low Customer Implication
IVTU: Agile Project in a Waterfall World Project Challenges: Multiple HW variants Multiple SW variants Continuous requirement Changes Specifications blurriness ISO 26262 Safety Requirements Industry Adherence
Why Use Agile? Benefits: Flexibility to adapt to customer requirements changing during project life cycle – New features to be implemented – Redefinition of functions Fast time to market Drawbacks: Customer resistance Light process vs Automotive classical approach Not big design up front
Agile testing Incorporate Agile Methodology in System Tests: Test Group Works as an Agile Team – Kanban approach. Adapting pseudo-sprints to customers necessities Client participates actively in Test Team activities – Demos – Specifications redefinition – Feedback Automotive SPICE processes is kept – Focused in Safety requirements – Auto-documented Testing Test Automation to absorb number of release & System variability – Jenkins CI & CD – Long duration Test Campaigns, robustness, stress
Testing Agile Alignment Joint Check Functional Req. VC-LOT 4-8 weeks PT-LOT Client Daily Meeting Retrospective Test Design Analysis Demo Execution SQA Release Release FB SW Analysis Design Design Code UT Code UT Integration Integration
Solution: Deployment Diagram TEST CLIENT TEST SERVER HTTPS - UMTS SMS KVASER USB RELAY BOX CAN iVTU USB RPYC Socket CI Main SERVER LAN
Solution: Logical Diagram DEVELOPMENT and TEST CLIENT LINUX SVN Client – Command Line CONFIGURATION MANAGEMENT SERVER TEST SERVER PLATFORM SVN Repository TCU Automation Code Unit Testing Test Suites Results SVN Client Eclipse - Pydev Subversion Server Robot Framework Build Script Python Jenkins Server Python CONTINUOUS INTEGRATION SERVER KVASER CAN IVTU UNDER TEST GPRS/SMS USB 3G ISP
Robot Framework: Keyword Driven testing Nominal ACNCall Sequence [VCAN Trigger][Front Or Side Crash][Server Crash][Server Hang Up][High Bus Load] [Documentation] Purpose: . The aim of this test is to check whether ACNCall is started when VCAN trigger cause is present for "front or side crash". At same time this test is checking server hang up by sending DTMF tone '6' and service behavior when high CAN bus load present. . . Acceptance criteria: . ACNCall is triggered for "front or side crash" state. . Service behavior with high CAN bus load is correct. . ACNCall is correctly terminated from server side. [Setup] Acncall Test Setup Start Airbag Frame Bus Load ON Crash [Front Or Side Crash] Check MPDT Message [ACNCall Start] Send MPDT Message [ACNCall Started] TSP Server Action [Pick Up Call On Primary Server] Check MPDT Message [ACNCall Voice Calling Check ACNCall MSD Message TSP Server Action [Terminate Request] Check MPDT Message [ACNCall Voice Call Disconnected] [Teardown] Acncall Test Teardown
Results and Conclusions
Results: Project Complexity
Results: Validation Effort
Results: Quality
Conclusions Complexity of Project can be better handled by means of Agile Methodologies Intensive automated testing and client demonstrations helps to produce better products, rising clients satisfaction Agile Methodologies can be introduced with success in some projects of automotive industry Agile methods like SCRUM usually do not support some essentials, which are required in the automotive context. We do not recommend a pure Agile approach, but rather a custom version integrated in current project life-cycles. Agile and SPICE Automotive can be combined to obtain best of both worlds.
Future Challenges Reduce number of bugs reported by client Involucrate client into sprint Converge process to accomplish SPICE reporting – Automatic Doors Reporting – Automatic Data Test Package Generation Extend Agile to SW Development
THANK YOU!!!! QUESTIONS¿? Xavier Martin Artal Software QA Leader xavier.martin@idneo.es es.linkedin.com/pub/xavier-martin/6/a89/723/
Automotive Spice & V-Model (1) Automotive SPICE : Comprises a set of assessment indicators of process performance and process capability Based on ISO/IEC ISO/IEC 15504-2 Defines several levels of adherence to standard Focus on Process V-Model: Extension of Waterfall Methodology Big Design Up Front Phases aligns with .
1. The need for an agile way of working 6 2. The need for an agile way of working 9 3. Agile Core Values - Agile Project Management Vs. 10 Agile Event Management 4. Agile principles 12 _Agile Principles of Agile Project Management 13 _Agile Principles of VOK DAMS Agile Event Management 14 5. Agile Methods 16 _Scrum in Short 16 _Kanban in Short 18
Agile testing Agile testing can mean many kinds of testing: -Any testing that is not based on test case level plans. -Exploratory (sometimes called explorative) testing, where the tester proceeds based on his/her observations of the software. -Sometimes it means testing is agile software development.
AGILE TESTING For agile testers, test engineers, test managers, developers, technical leads. Ensure the production of effective and valuable software. Agile Fundamentals Agile Programming Agile Software Design Agile Fundamentals Agile Testing Agile Test Automation ICP CERTIFIED PROFESSIONAL ICP CERTIFIED PROFESSIONAL ICP-PRG CERTIFIED .
Agile Estimating and Planning by Mike Cohn Agile Game Development with Scrum by Clinton Keith Agile Product Ownership by Roman Pichler Agile Project Management with Scrum by Ken Schwaber Agile Retrospectives by Esther Derby and Diana Larsen Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and .
After searching with the keywords: Agile testing software, Scrum agile testing software, Kanban agile testing software, Test Driven Development agile test software, Behavior Driven Development test software, automation test software, the tables 1, 2 and 3 show the number of scientific articles retrieved. This search was made with a restriction
1.1 Purpose of the Agile Extension to the BABOK Guide1 1.2 What is Agile Business Analysis?2 1.3 Structure6 Chapter 2:The Agile Mindset 2.1 What is an Agile Mindset?7 2.2 The Agile Mindset, Methodologies, and Frameworks8 2.3 Applying the Agile Mindset9 2.4 Agile Extension and the Agile Ma
Agile World View "Agility" has manydimensions other than IT It ranges from leadership to technological agility Today's focus is on organizational & enterprise agility Agile Leaders Agile Organization Change Agile Acquisition & Contracting Agile Strategic Planning Agile Capability Analysis Agile Program Management Agile Tech.
The most popular agile methodologies include: extreme programming (XP), Scrum, Crystal, Dynamic Sys-tems Development (DSDM), Lean Development, and Feature Driven Development (FDD). All Agile methods share a common vision and core values of the Agile Manifesto. Agile Methods: Some well-known agile software development methods include: Agile .