Agile Development In A Medical Device Company

2y ago
340.45 KB
6 Pages
Last View : 1m ago
Last Download : 6m ago
Upload by : Eli Jorgenson

Agile 2008 ConferenceAgile Development in a Medical Device CompanyPieter Adriaan Rottier, Victor RodriguesCochlear support this “mapping” process Cochlear provides the audiologist with a software applicationthat interacts with the sound processor, audiologist andthe patient to obtain the best mapping of sound untothe patient’s nervous system.Cochlear alsoprovides the audiologist and patient with several otherapplications including an application to diagnose thecondition of the implanted part of the Nucleus system.The Audiological Software group of the Researchand Development (R&D) division is responsible forthe development and maintenance of these softwareapplications.AbstractThis article discuss the experience of the softwaredevelopment group working in Cochlear withintroducing Scrum as an Agile methodology. Weintroduce the unique challenges we faced due to thenature of our product and the medical device industry.These are long validation cycles, strict regulatorystandards and a high level of dependence on physicaldevices among others. We then show how we managedto overcome these by adapting Scrum and mapping itunto our current process in such a way as to continueto satisfy the strict standard and regulations. We showhow we automated the testing of physical devices toovercome our dependence on manual testing. Wepresent the tools we used for managing requirement,validation, reporting and performance metrics. Finallywe briefly look at the implications of the proposedIEC62304 standard on software development withScrum.2. History/EnvironmentAll medical devices have to adhere to certainstandards to ensure patient safety and wellbeing.Cochlear implants are no exception. As our implantsystems deliver electric current directly to the nervoussystem of a human being they have to adhere to thehighest level of the medical device standards. Thesoftware directly controls levels of stimulation andreceives the same level of regulatory scrutiny as therest of the system. Some of these standards are shownin Table 1.1. IntroductionCochlear is a medical device company thatdelivers innovative hearing solutions for severely toprofoundly deaf people. Cochlear’s current productrange is the Nucleus Freedom cochlear implantsystem. This system consists of three components: a thin, flexible electrode array that fits insidethe cochlea an implanted receiver/stimulator that transmitselectric stimuli to the nerves of the cochlea viathe electrode array an externally worn sound processor thatreceives sound from the environment, encodesit and transmits it via an RF link to thereceiver/stimulatorThe cochlear implant system interfaces directly withthe human nervous system. Therefore the electricalimpulses have to be customised to the nervous systemof the individual. This “customisation” is referred toas “mapping” and is done by audiologists thatspecialises in cochlear implants. This is where thesoftware enters the picture.978-0-7695-3321-6/08 25.00 2008 IEEEDOI 10.1109/Agile.2008.52Table 1: Regulatory standards applicable toCochlear softwareEN ISO 14971:2007Medical devices – Application of risk2007management to medical devices (ISO14971:2007) part 1 and 2EN60601-1-4:1996Medical electrical equipment – Part 1-4:1996(IEC60601-1-4:1996), General requirements for safety – A1: 1999AmendmentIEC Collateralstandard:Programmable60601-1electrical medical systems4:1996/A1:1999IEC 60950-1Information technology equipment - 2005Safety - Part 1: General requirementsANSI/AAMI/IEC 62304: Medical device software - Software life2006cycle :processesANSI/AAMIMedical Device Software – Software LifeSW68:2001Cycle uidanceTo ensure compliance to these standards andtherefore that our implants are safe and effectiveCochlear’s development procedures and quality systemare audited on a regular basis. Cochlear has developedits own product development process, shown in Figure218

Scrum (the process developed by Ken Schwaber andJeff Sutherland) was going to be the best fit for ourenviroment. It just made sense!At that point the software development group wasworking on the third version of the “mapping”software and just starting a new project to improveCochlear’s diagnostic software capability.Scrum was introduced to both projects at the sametime but its success with the first project was slowerdue to the amount of legacy code in the project. Thesecond project was able to implement the processmuch more easily and our experience in this projectwill form the basis of this paper.1. This process is based on the V-model. The Vmodel is often used in regulated environments and is atype of waterfall process.The process shown in Figure 1 is the one that wasused for software development, which is why the phasedealing with the manufacturing process development iscrossed out.Project Definition/ LaunchProcedureI40162AEProduct Definition/MATProcedureE11429AEProject DefinitionStartProduct DefinitionGenerate PMS,PMP, stem Development /Integration ProcedureN94194AESystem DevelopmentNeeds Analysisand ConceptDevelopmentDesignProcess DevelopmentPreliminary andDetailed DesignsSystem DevelopmentT1aT1Process chT3T4MarketAcceptanceTestMATSystem Integration,V&VSystems Integration andVerificationUnit Test &DesignVerificationsDesign VerificationProcessVerification, BuildSub-systemsProcess Verification4. Challenges to the introduction of agileFigure 1:Cochlear’s Design Control Process forsoftwareOrganisational challengesA detailed description of this process is beyond thescope of this paper. Suffice to say that it consists offormal definition, design, implementation, verificationand validation phases with a sign-off between each ofthe phases called a Tollgate.Perhaps an important aspect of why Scrum wassuccessfully adopted at Cochlear, at least within thesoftware development group, was that it was drivenfrom senior management. Admittedly a lot of lobbyingwent on and it was not immediately accepted as it waslargely seen as a process with short term goals and didnot cover the longer term business objectives andtiming needs. It was really an educational processsupplemented by high visibility on the execution.Senior management and the software teamsencouraged people from around the organisationencouraged to attend Daily Scrum and Reviewmeetings. The business started seeing benefits in a veryshort time and soon Scrum became a word that wasubiquitous within the R&D division.3. The Introduction of ScrumIn 2001 the software development group beganadopting some Agile development practices,implementing a small number of changes at a time inan attempt to increase the chances of success within awaterfall organisation. Practices such as automatednightly builds, peer reviews, unit testing and otherswere instrumental in the improvement of productquality. The group also introduced the refinementsshown in Figure 2 to the DCP process in an effort tomake the design and verification phases lessmonolithic and more iterative.T1AFeature 1 SpecifyProcess challengesThe medical devices industry is regulated by anumber of regulatory bodies of which the FDA isperhaps the best known example. For these bodies it isextremely important to be able to see that the systemyou are developing is safe as well as effective for itsintended purpose. As a company Cochlear has avery good reputation based on its quality system andits track record. Cochlear’s quality system is based onGood Automated Manufacturing Practices (GAMP)guidelines, the same as commonly used bypharmaceutical companies. These guidelines require alarge amount of documentation and extensivecomponent, system, product and process verificationand validation.Other issues that faced introduction of Agilepractices at Cochlear were: the disparity between using an Agilemethodology in the software department versusT3Design DevelopTestFeature 2Feature 3Feature 4Feature 5Feature 6Feature NFigure 2: Software design, implementation andverificationThe benefits of these changes were obvious andwere welcomed by the organisation.Softwareproducts were far more robust than before but therewas still something missing. “Feature creep” was stillquite prominent and schedule slippages werecommonplace. After some further research into Agiledevelopment practices, the group decided in 2005 that219

using a waterfall/iterative process throughoutthe rest of the organisationthe identification of the recipient safety aspectsof a product and the mitigation of such risks ina structured and traceable waymanaging the evolution of requirements in atraceable waythe long validation cycles required to validatethe marketing claims and ultimately the userrequirementsthe extensive manual verification testingrequired to verify that the electrical stimulationof the implant under software applicationcontrol is within safe limitsdevice drivers with a large amount of legacycode that was neither unit tested nor conduciveto any other type of automated testing.Product Definition/Product ValidationIn this phase the first conflict between the classicalScrum approach and our highly regulated environmentbecame evident.Traditionally we provided the validation team witha Use Case document. The validation team used thisdocument as the basis of their validation protocol. TheUse Case document contained all the functionalrequirements. This document was supplemented bythe Software Requirement Specification (SRS) whichdetailed all the non-functional requirements. Aftervalidation the reports generated by the validation teamalong with the documents described above wereprovided to the regulatory bodies as evidence of theproduct meeting its safety and efficacy requirements.We could not replace this process with disposablerequirements, as recommended by Scrum.Thealternative had to show the same traceability as theoriginal approach while allowing the evolution ofindividual requirements.We decided to adapt the User Stories concept usedby Extreme Programming (XP) as described by MikeCohn [1]. These User Stories needed to be storedelectronically in a fashion that allowed easy access toindividual stories but also provided the ability togenerate reports showing each User Story or Constraintand how they related to a validation activity.As a start we only identified enough User Storiesfor the first two development iterations.We realised that we would not only have to evolveour product requirements but also our process as wewent along.5. Using Scrum for the CS19 projectIn this section I will present how Scrum wasintroduced for the CS19 project. I assume that thereader is familiar with the Scrum process. For a goodintroduction to Scrum we recommend “AgileDevelopment with Scrum” by Ken Schwaber [4]I will show how we mapped Scrum to the DCP andI will point out the challenges still remaining. In thenext section I will introduce the tools that we used andexplain how we adapted these to fit our purposes.Project definition/closeoutIn the first phase of the project we followed theprocedure mandated by the DCP. We developed thebusiness case and a project management plan where wedescribed the development process we would befollowing. The quality system allows for deviationsfrom the development process described in the DCPprovided the deviations is defined and justified in theproject management plan. Additionally it had to beapproved by the Quality Division. To obtain approvalwe had to show that we would still adhere to the levelsof traceability, verification, risk mitigation andvalidation required by our quality system and theregulatory standards.Scrum was justified as an appropriate process forthe project based on the high level of uncertaintyassociated with the capabilities of the measurementtechnology that we were using in this product.The main issue in this phase was setting thetimeline as all our estimations and experience werebased on a different development process. Thetimeline ended up being very far off the actual projectduration; one thing that we are still struggling with.Design, Implementation and VerificationDuring the first few iterations we aimed to deliver aslice through the entire system that could be used tostart investigating the feasibility of the concept onwhich the product was to be based. Having done justenough architectural design to identify the majorsystem components, development started.Almost immediately another big challenge becameapparent; since the system needed to interface withexternal hardware, most of the safety testing had to bedone manually. Because we only consider a UserStory completed once it was implemented and tested, itproved to be impossible to complete a User Storywithin a Sprint (four weeks at this point). Initially wetried to circumvent this issue by allowing the testing ofa User Story to happen during the subsequent Sprintwhile the developers worked on something else. Asmost of those experienced in Agile will know, this220

document and mitigate the risks by adding a section tothe tests to exposes the risk, in the form of a failing testand then ensures that the risk is contained by gettingthe test to pass. Where this could not be done the riskwas again added as a constraint to the product backlog.proved to be a bad idea. Stories that were supposed tobe done kept creeping back by way of bugs into thebacklog and this quickly started to seriously degradeour velocity.A difficult decision had to be made at this point; dowe spend a significant amount of time automating thetesting or do we keep on going the way we are? Wedecided to automate a part of the manual testing, thedata analysis, but not the rest. A project which starteda few months after the CS19 project subsequentlymanaged to successfully automate the entire manualtesting process, a significant but ultimately worthwhileeffort.We also decided to introduce the developers to TestDriven Development (TDD) using the Framework forIntegrated Tests (FIT) developed by WardCunningham [2] for the acceptance tests and NUnit forthe component/object verification. At some pointduring the process the hardware testing load starteddropping and we shortened the iterations to two weeks.Finally Scrum really started showing its strengths.For tracking issues we used the standard procedureof adding these to the product backlog along with thenew features. We used a fairly formal change controlprocess for prioritising issues. The advantage to theformal process was that it ensured that we spend thetime and effort required to understand every issue andits implications, as well as allowing us to spot trendsthat might point to underlying problems early.The clinical validation still had to be done manuallybut through iteration and extensive bench testing wemanaged to ensure that the validation only providedconfirmation that all the requirements had been met. Itwas possible to decouple the clinical validation to alarge extent from the development by increasing theamount of bench validation.6. The tools we usedRequirements ManagementWe started off using CaliberRM from /rm.html).This proved to be too hard for the developers tonavigate as it was very different from the tools theywere used to, viz. Attlasian Jira and Confluence( Thus we transferred allthe requirements over to Confluence, making use ofsome advanced macros to allow the generation ofreports. Each User Story had its own Confluence page,and consisted of three sections: the summary, containing the estimate, priority,status and description of the User Story the discussion, containing the minutes of anydiscussions around that User Story as well asany mock-ups, models and designs associatedwith the User Story the tests, containing both the manual tests andthe automated tests associated with the UserStory.Using advanced macros the relevant sections can beextracted for reporting, regulatory and disseminationpurposes.Developers use the Jira issue tracking system tomanage and track the tasks associated with every UserStory, these tasks are linked on the one side to the UserStory and on the other side to the source code ManagementRisk management is important in any industry andespecially so in medical devices. Initially we tried tomanage risks according to the DCP. The DCP requiresa hazards analysis to be created during the productdefinition stage and then updated at major milestones.This approach was not very successful as we weretrying to determine risks on features that were not yetwell defined. As an alternative we split the processinto two stages. We identified high level system risksduring the initial phase of the project and added theseas constraints to our product backlog. Any other risksthat became apparent during development we alsoadded to the backlog. Then we identified, for eachUser Story, the relevant risks at the point where wemoved the User Story from the product backlog to theSprint backlog. For these User Stories we tended toAcceptance TestingWe started off using the FIT framework foracceptance testing but it proved to be difficult tointegrate with our other systems and we were hesitantto start using another completely separate tool. Aftersome searching we found the Greenpepper,whichwhile using fixtures very similar to FIT, allowsseamless integration with Confluence.Our process evolved to the point that developerstogether with the verification engineer would write theacceptance tests in Confluence. Developers wouldthen write the unit tests to express their design intent221

first time, as we only allow enough story points to do itonce.I like the suggestion by Alistair Cockburn that weallow the user to see the software twice and requestchanges before considering a User Story as done. Formore information see [3].after which the code was written to first pass the unittests and then to pass the acceptance tests. In this waywe managed to get rid of manual testing in almost allareas of code.Project monitoring and controlBenefits of AgileInitially we used just a burndown chart, updated byhand for project monitoring. This allowed somevisibility of the project status but often it would not getupdated, either because the developers have notupdated their effort from the previous day or becausesomeone forgot to update the chart itself.The entire monitoring and control area is one westill struggle with (for more see the section onchallenges). We have however found,veryuseful for automating our burndown charts.So what do we consider the biggest benefits ofusing Agile in our environment? By using test drivendevelopment along with the iterative development offeatures we have been able to fairly consistentlyproduce high quality code. While we did not find thedevelopment process to be any more efficient than theprocess we were used to we think this is due to thelarge amount of supporting frameworks we had toconstruct to allow it to operate in our environment. Aswe move forward we fully expect to be reusing a lot ofwhat we have developed and therefore increasedefficiencies.As we are almost always faced with a high degreeof uncertainty in our projects we appreciate theincreased flexibility afforded by using Scrum.7. Lessons learnedSystem Architecture DesignAlthough we did some up front design of the systemarchitecture we did not do nearly enough in terms ofconsciously evolving and revisiting the architecture atregular intervals. This led to code being added withoutsufficient thought being given to how it wouldinfluence the big picture. The result was seeminglysmall code changes being nearly impos

Agile Development in a Medical Device Company Pieter Adriaan Rottier, Victor Rodrigues Cochlear Limited Abstract This article discuss the experience of the software development group working in Cochlear with introducing Scrum as an Agile methodology. We introduce the unique challenges we faced due to the nature of our product and the medical device industry. These .

Related Documents:

1. The need for an agile way of working 6 2. The need for an agile way of working 9 3. Agile Core Values - Agile Project Management Vs. 10 Agile Event Management 4. Agile principles 12 _Agile Principles of Agile Project Management 13 _Agile Principles of VOK DAMS Agile Event Management 14 5. Agile Methods 16 _Scrum in Short 16 _Kanban in Short 18

Agile Estimating and Planning by Mike Cohn Agile Game Development with Scrum by Clinton Keith Agile Product Ownership by Roman Pichler Agile Project Management with Scrum by Ken Schwaber Agile Retrospectives by Esther Derby and Diana Larsen Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and .

The most popular agile methodologies include: extreme programming (XP), Scrum, Crystal, Dynamic Sys-tems Development (DSDM), Lean Development, and Feature Driven Development (FDD). All Agile methods share a common vision and core values of the Agile Manifesto. Agile Methods: Some well-known agile software development methods include: Agile .

1.1 Purpose of the Agile Extension to the BABOK Guide1 1.2 What is Agile Business Analysis?2 1.3 Structure6 Chapter 2:The Agile Mindset 2.1 What is an Agile Mindset?7 2.2 The Agile Mindset, Methodologies, and Frameworks8 2.3 Applying the Agile Mindset9 2.4 Agile Extension and the Agile Ma

Agile World View "Agility" has manydimensions other than IT It ranges from leadership to technological agility Today's focus is on organizational & enterprise agility Agile Leaders Agile Organization Change Agile Acquisition & Contracting Agile Strategic Planning Agile Capability Analysis Agile Program Management Agile Tech.

1. Agile methods are undisciplined and not measurable. 2. Agile methods have no project management. 3. Agile methods apply only to software development. 4. Agile methods have no documentation. 5. Agile methods have no requirements. 6. Agile methods only work with small colocated teams.-7. Agile methods do not include planning. 8.

The Agile Customer . 9/6/2012 6 Agile Development Team Agile Analyst . 9/6/2012 7 Agile Programmer Agile Tester . 9/6/2012 8 Agile Manager Agile Usability Designer . 9/6/2012 9 Kicking off a project The Inception Deck –Ten questions you’d be crazy not to ask before starting any

The Agile Customer. 9/4/2013 6 Agile Development Team Agile Analyst. 9/4/2013 7 Agile Programmer Agile Tester. 9/4/2013 8 Agile Manager Agile Usability Designer. 9/4/2013 9 Kicking off a project The Inception Deck –Ten questions you’d be crazy not to ask before starting any software