Agile Development In A Medical Device Company

using a waterfall/iterative process throughoutthe rest of the organisationthe identification of the recipient safety aspectsof a product and the mitigation of such risks ina structured and traceable waymanaging the evolution of requirements in atraceable waythe long validation cycles required to validatethe marketing claims and ultimately the userrequirementsthe extensive manual verification testingrequired to verify that the electrical stimulationof the implant under software applicationcontrol is within safe limitsdevice drivers with a large amount of legacycode that was neither unit tested nor conduciveto any other type of automated testing.Product Definition/Product ValidationIn this phase the first conflict between the classicalScrum approach and our highly regulated environmentbecame evident.Traditionally we provided the validation team witha Use Case document. The validation team used thisdocument as the basis of their validation protocol. TheUse Case document contained all the functionalrequirements. This document was supplemented bythe Software Requirement Specification (SRS) whichdetailed all the non-functional requirements. Aftervalidation the reports generated by the validation teamalong with the documents described above wereprovided to the regulatory bodies as evidence of theproduct meeting its safety and efficacy requirements.We could not replace this process with disposablerequirements, as recommended by Scrum.Thealternative had to show the same traceability as theoriginal approach while allowing the evolution ofindividual requirements.We decided to adapt the User Stories concept usedby Extreme Programming (XP) as described by MikeCohn [1]. These User Stories needed to be storedelectronically in a fashion that allowed easy access toindividual stories but also provided the ability togenerate reports showing each User Story or Constraintand how they related to a validation activity.As a start we only identified enough User Storiesfor the first two development iterations.We realised that we would not only have to evolveour product requirements but also our process as wewent along.5. Using Scrum for the CS19 projectIn this section I will present how Scrum wasintroduced for the CS19 project. I assume that thereader is familiar with the Scrum process. For a goodintroduction to Scrum we recommend “AgileDevelopment with Scrum” by Ken Schwaber [4]I will show how we mapped Scrum to the DCP andI will point out the challenges still remaining. In thenext section I will introduce the tools that we used andexplain how we adapted these to fit our purposes.Project definition/closeoutIn the first phase of the project we followed theprocedure mandated by the DCP. We developed thebusiness case and a project management plan where wedescribed the development process we would befollowing. The quality system allows for deviationsfrom the development process described in the DCPprovided the deviations is defined and justified in theproject management plan. Additionally it had to beapproved by the Quality Division. To obtain approvalwe had to show that we would still adhere to the levelsof traceability, verification, risk mitigation andvalidation required by our quality system and theregulatory standards.Scrum was justified as an appropriate process forthe project based on the high level of uncertaintyassociated with the capabilities of the measurementtechnology that we were using in this product.The main issue in this phase was setting thetimeline as all our estimations and experience werebased on a different development process. Thetimeline ended up being very far off the actual projectduration; one thing that we are still struggling with.Design, Implementation and VerificationDuring the first few iterations we aimed to deliver aslice through the entire system that could be used tostart investigating the feasibility of the concept onwhich the product was to be based. Having done justenough architectural design to identify the majorsystem components, development started.Almost immediately another big challenge becameapparent; since the system needed to interface withexternal hardware, most of the safety testing had to bedone manually. Because we only consider a UserStory completed once it was implemented and tested, itproved to be impossible to complete a User Storywithin a Sprint (four weeks at this point). Initially wetried to circumvent this issue by allowing the testing ofa User Story to happen during the subsequent Sprintwhile the developers worked on something else. Asmost of those experienced in Agile will know, this220

document and mitigate the risks by adding a section tothe tests to exposes the risk, in the form of a failing testand then ensures that the risk is contained by gettingthe test to pass. Where this could not be done the riskwas again added as a constraint to the product backlog.proved to be a bad idea. Stories that were supposed tobe done kept creeping back by way of bugs into thebacklog and this quickly started to seriously degradeour velocity.A difficult decision had to be made at this point; dowe spend a significant amount of time automating thetesting or do we keep on going the way we are? Wedecided to automate a part of the manual testing, thedata analysis, but not the rest. A project which starteda few months after the CS19 project subsequentlymanaged to successfully automate the entire manualtesting process, a significant but ultimately worthwhileeffort.We also decided to introduce the developers to TestDriven Development (TDD) using the Framework forIntegrated Tests (FIT) developed by WardCunningham [2] for the acceptance tests and NUnit forthe component/object verification. At some pointduring the process the hardware testing load starteddropping and we shortened the iterations to two weeks.Finally Scrum really started showing its strengths.For tracking issues we used the standard procedureof adding these to the product backlog along with thenew features. We used a fairly formal change controlprocess for prioritising issues. The advantage to theformal process was that it ensured that we spend thetime and effort required to understand every issue andits implications, as well as allowing us to spot trendsthat might point to underlying problems early.The clinical validation still had to be done manuallybut through iteration and extensive bench testing wemanaged to ensure that the validation only providedconfirmation that all the requirements had been met. Itwas possible to decouple the clinical validation to alarge extent from the development by increasing theamount of bench validation.6. The tools we usedRequirements ManagementWe started off using CaliberRM from /rm.html).This proved to be too hard for the developers tonavigate as it was very different from the tools theywere used to, viz. Attlasian Jira and Confluence(http://www.atlassian.com/). Thus we transferred allthe requirements over to Confluence, making use ofsome advanced macros to allow the generation ofreports. Each User Story had its own Confluence page,and consisted of three sections: the summary, containing the estimate, priority,status and description of the User Story the discussion, containing the minutes of anydiscussions around that User Story as well asany mock-ups, models and designs associatedwith the User Story the tests, containing both the manual tests andthe automated tests associated with the UserStory.Using advanced macros the relevant sections can beextracted for reporting, regulatory and disseminationpurposes.Developers use the Jira issue tracking system tomanage and track the tasks associated with every UserStory, these tasks are linked on the one side to the UserStory and on the other side to the source code www.perforce.com/).Risk ManagementRisk management is important in any industry andespecially so in medical devices. Initially we tried tomanage risks according to the DCP. The DCP requiresa hazards analysis to be created during the productdefinition stage and then updated at major milestones.This approach was not very successful as we weretrying to determine risks on features that were not yetwell defined. As an alternative we split the processinto two stages. We identified high level system risksduring the initial phase of the project and added theseas constraints to our product backlog. Any other risksthat became apparent during development we alsoadded to the backlog. Then we identified, for eachUser Story, the relevant risks at the point where wemoved the User Story from the product backlog to theSprint backlog. For these User Stories we tended toAcceptance TestingWe started off using the FIT framework foracceptance testing but it proved to be difficult tointegrate with our other systems and we were hesitantto start using another completely separate tool. Aftersome searching we found the Greenpepper rsoftware.com/en/),whichwhile using fixtures very similar to FIT, allowsseamless integration with Confluence.Our process evolved to the point that developerstogether with the verification engineer would write theacceptance tests in Confluence. Developers wouldthen write the unit tests to express their design intent221

first time, as we only allow enough story points to do itonce.I like the suggestion by Alistair Cockburn that weallow the user to see the software twice and requestchanges before considering a User Story as done. Formore information see [3].after which the code was written to first pass the unittests and then to pass the acceptance tests. In this waywe managed to get rid of manual testing in almost allareas of code.Project monitoring and controlBenefits of AgileInitially we used just a burndown chart, updated byhand for project monitoring. This allowed somevisibility of the project status but often it would not getupdated, either because the developers have notupdated their effort from the previous day or becausesomeone forgot to update the chart itself.The entire monitoring and control area is one westill struggle with (for more see the section onchallenges). We have however found enpeppersoftware.com/en/),veryuseful for automating our burndown charts.So what do we consider the biggest benefits ofusing Agile in our environment? By using test drivendevelopment along with the iterative development offeatures we have been able to fairly consistentlyproduce high quality code. While we did not find thedevelopment process to be any more efficient than theprocess we were used to we think this is due to thelarge amount of supporting frameworks we had toconstruct to allow it to operate in our environment. Aswe move forward we fully expect to be reusing a lot ofwhat we have developed and therefore increasedefficiencies.As we are almost always faced with a high degreeof uncertainty in our projects we appreciate theincreased flexibility afforded by using Scrum.7. Lessons learnedSystem Architecture DesignAlthough we did some up front design of the systemarchitecture we did not do nearly enough in terms ofconsciously evolving and revisiting the architecture atregular intervals. This led to code being added withoutsufficient thought being given to how it wouldinfluence the big picture. The result was seeminglysmall code changes being nearly impos

Agile Development in a Medical Device Company Pieter Adriaan Rottier, Victor Rodrigues Cochlear Limited rrottier@cochlear.com.au Abstract This article discuss the experience of the software development group working in Cochlear with introducing Scrum as an Agile methodology. We introduce the unique challenges we faced due to the nature of our product and the medical device industry. These .