Scrum Engineering Practices: Experiences Of Three .
Scrum Engineering Practices: Experiences of Three Microsoft TeamsLaurie WilliamsGabe Brown, Adam Meltzer, Nachiappan NagappanNorth Carolina State UniversityRaleigh, NC 27695, USAwilliams@csc.ncsu.eduMicrosoft CorporationRedmond, WA 98052, USA{gabeb, ameltzer, nachin}@microsoft.comAbstract - The Scrum methodology is an agile softwaredevelopment process that works as a project managementwrapper around existing engineering practices to iterativelyand incrementally develop software. With Scrum, for adeveloper to receive credit for his or her work, he or she mustdemonstrate the new functionality provided by a feature at theend of each short iteration during an iteration review session.Such a short-term focus without the checks and balances ofsound engineering practices may lead a team to neglect quality.In this paper we present the experiences of three teams atMicrosoft using Scrum with an additional nine soundengineering practices. Our results indicate that these teamswere able to improve quality, productivity, and estimationaccuracy through the combination of Scrum and nineengineering practices.Keywords - Agile software development, ScrumI. INTRODUCTIONScrum [16, 29] is the most often used [6, 30, 31] agile[10] software development methodology among teams thatutilize an agile methodology. A large-scale survey [31]deployed in the software engineering industry fromJune/July 2008 received 3061 respondents from 80 differentcountries. For the question “Which Agile methodology doto you closely follow” 49% of the respondents mentionedScrum and an additional 29% mentioned Scrum withExtreme Programming. Additionally, a survey of 10% ofall engineers at Microsoft, indicated that more than 60% ofthe engineers use Scrum (Figure 2) [6]. Scrum provides aproject management structure to a team. However, theScrum methodology does not prescribe the engineeringpractices a team should use, purportedly to giveorganizations as much flexibility as possible in choosingtheir engineering practices.With Scrum, gone are the days of a software developerreporting to the project manager that a new feature is 80%complete. Instead, in a Scrum environment, credit is “all ornothing” whereby a feature that is 99% done is considered“not done.” For the developer to receive credit for his or herwork, he or she must demonstrate the new functionalityprovided by a feature at the end of each short iterationduring an iteration review session. Developers, testers, theproject managers, the product owner/manager, and othersattend this iteration review session. The expectations fordemonstrating all planned features at the end of eachiteration are high as the team works to meet its iterationgoal.This short-term focus of iterations coupled with a lackof prescribed engineering practices may lead to trouble.“Flaccid Scrum1” is a term coined by Martin Fowler to referto teams that utilize only Scrum’s project managementpractices. Progress eventually slows for Flaccid Scrumteams, according to Fowler, because the team has not paidenough attention to the quality of the code produced duringeach iteration. In some cases, only the easiest scenario of afeature (often referred to as the “happy path”) isdemonstrated at the end of the iteration. This “happy path”may be formally specified as the acceptance criteria for thefeature. The feature can then be considered to be “done”,with the development team getting credit for the feature.Focus then turns to a new set of commitments to deliverfeatures for the next iteration, even if only the “happy path”of prior features has been done.The Scrum methodology, however, may provide a solidproject management framework for a team that also utilizessound engineering practices. In this paper, we share theexperiences and quantitative productivity and quality resultsof three Microsoft teams who utilized a Scrum-basedsoftware development methodology augmented with nineengineering practices recommended by the MicrosoftEngineering Excellence group that takes care ofcompanywide process initiatives.Software engineering research can aid practitioners intheir technology and/or process choices. Practitioners whoread this paper will gain an understanding of the need to addengineering practices to a Scrum process to prevent FlaccidScrum.The rest of this paper is structured as follows. Weexplain Scrum and provide background in Sections 2 and 3.We provide the motivation for our paper in Section 4. InSection 5, we describe the practices adopted by the team.We then provide the results of the teams in Section 6 andlimitations of our study in Section 7.We summarize thestudy in Section 8.II. SCRUMThe Scrum methodology is an agile softwaredevelopment process that works as a wrapper with existingengineering practices to iteratively and /www.nunit.org/index.php4http://junit.org/3
develop software. Scrum is composed of the followingproject management practices: The Product Owner creates the requirements, prioritizesthem, and documents them in the Product Backlogduring Release Planning. In Scrum, requirements arecalled features. Scrum teams work in short iterations. When Scrumwas first defined [16, 29], iterations were 30-days long.More recently Scrum teams often use even shorteriterations, such as two-week iterations. In Scrum, thecurrent iteration is called the Sprint. A Sprint Planning Meeting is held with thedevelopment team, testers, management, the projectmanager, and the Product Owner. In the SprintPlanning Meeting, this group chooses which features(which are most often user-visible, user valued, andable to be implemented within one iteration) from theproduct backlog are to be included in the next iteration,driven by highest business value and risk and thecapacity of the team. Once the Sprint begins, features cannot be added to theSprint. Short, 10-15 minute Daily Scrum meetings are held.While others (such as managers) may attend thesemeetings, only the developers and testers and the ScrumMaster (the name given to the project manager inScrum) can speak. Each team member answers thefollowing questions:oooWhat have you done since the last Daily Scrum?What will you do between now and the next DailyScrum?What is getting in your way of doing work? At the end of a Sprint, a Sprint Review takes place toreview progress and to demonstrate completed featuresto the Product Owner, management, users, and the teammembers. After the Sprint Review, the team conducts aRetrospective Meeting. In the Retrospective Meeting,the team discusses what went well in the last Sprint andhow they might improve their processes for the nextSprint.III. BACKGROUND AND RELATED WORKIn this section, we provide related work on the Scrumagile software development methodology. We also discusscase study research.A. Scrum ResearchA myriad of qualitative experience reports about theScrum software development methodology have beenpublished. However, few studies have been conducted onthe Scrum that report quantitative results, as ours does. Inthis section, we summarize information available in theliterature about the use of Scrum by industrial softwaredevelopment teams whereby the papers provided detailsbeyond qualitative experience reports. A pattern among thepublished studies is that the successful Scrum teams alsoutilized proven engineering practices.Tain, a Swedish gaming company, adopted Scrum andExtreme Programming (XP) engineering practices todevelop an online poker game [21]. The team delivered astable, scalable product on schedule. During this time, theteam also was reduced in size by half and those thatremained worked less overtime to produce more businessvalue than previously. The engineering practices the teamenumerated as critical to their success are the following:continuous integration, refactoring, and test-drivendevelopment.A development team for an Internet portal utilized theScrum methodology [12]. Initially, the short term focus ofScrum caused this team to ignore the use of some bestengineering practices, leading to “cumulative and oftenirreversible” problems. Early in the development cycle, theteam established source control, coding standards, processesfor code reviews and check-ins, and informal rules fordesign discussions and team meetings. However, the teamdid not initially establish an automated build system, a unittest framework, or a practice of creating automated qualityassurance tests. The eventual implementation of thesepractices aided the team in successfully implementing ahigher quality product by a team with improved morale.Two teams at Systematic utilized a Scrum-basedprocess [18]. Systematic is an independent software andsystems company focused on complex and criticalinformation technology solutions.Systematic oftenproduces products that are mission critical with highdemands for reliability, safety, accuracy, and usability. In2005, Systematic was rated a Capability Maturity Model –Integrated (CMM-I)2 Level 5 company, an indication of itsuse of engineering best practices. Through the use of Scrum,Systematic estimates that it doubled its productivity and cutdefects by 40%.B. Case Study ResearchThe experiences shared in this paper can be classifiedas case study research. Case studies can be viewed as“research in the typical” [13, 19]. As opposed to formalexperiments, which often have a narrow focus and anemphasis on controlling context variables, case studies insoftware engineering test theories and collect data throughobservation of a project in an unmodified setting [34].However, because the corporate, team, and projectcharacteristics are unique to each case study, comparisonsand generalizations of case study results are difficult and aresubject to questions of internal validity [20]. Nonetheless,case studies are valuable because they involve factors that2http://www.sei.cmu.edu/cmmi/
staged experiments generally do not exhibit, such as scale,complexity, unpredictability, and dynamism [27]. Casestudies are particularly important for industrial evaluation ofsoftware engineering methods and tools [19]. Researchersbecome more confident in a theory when similar findingsemerge in different contexts [19]. By recording the contextvariables of multiple case studies and/or experiments,researchers can build evidence through a family ofexperiments. Replication of case studies addresses threatsto experimental validity [2].IV. MICROSOFT TEAM AND PROCESSIn this section we provide information on the threeMicrosoft teams included in our study that utilized a Scrumbased software development methodology plus engineeringpractices. We then discuss the software developmentprocess used by the teams.A. Research MethodologyThe second and third authors can be considered actionresearchers. They have participated as software engineerson the three teams. The first author obtained informationabout the teams’ experiences by interviewing the secondauthor using pre-prepared questions, which were intertwinedwith opportunistic follow-on questions based upon hisanswers. One interview was conducted on the phone andthe second in person. Both interviews were approximatelyone hour in duration. The fourth author participated in theinterviews. He also had numerous informal conversationswith the two software engineers on the teams. The secondand third authors provided quantitative data, which wasinterpreted and analyzed by the first and fourth authors.B. Team Demographics and ContextTable I provides an overview of the context factors ofthe three teams. The context factors were motivated basedupon those specified in the Extreme ProgrammingEvaluation Framework (XP-EF) [33]. We do not provideinformation about the exact Microsoft products the teamsimplemented to enable us to share more information aboutthe team’s results. The domain of the each of the productsis provided in the table and ranges from infrastructure to testinfrastructure to mobile web applications. In all cases, theteams were working on the first release of their products ineither C# or C . The teams produced between 9 and 31thousand lines of implementation code during a period ofbetween 11 and 18 months long.Teams A and B were small teams of between three andfive members. Team C was larger with 19 members.Teams B was co-located teams while Teams A and C weredistributed between the US and China, challenging the usualface-to-face communication advocated in the AgileManifesto [5]. Other context factors presented in Table Iwill be discussed in Section 5.C. Scrum-based Process Used by TeamsThe three teams utilized a Scrum-based softwaredevelopment process and added nine additional engineeringpractices. These nine practices are Microsoft EngineeringExcellence Best Practices.Microsoft EngineeringExcellence is an organization responsible for supportingMicrosoft's engineering community by providing theengineers with learning and development opportunities, andwith discovering and propagating engineering best practicesacross the company and into the information technology(IT) ecosystem.This sub-section provides information on the softwaredevelopment methodology used by the teams.1) Basic ScrumThe teams utilized the basic practices of Scrum laid outin Section II. All teams began with a four-week iteration.Team A then transitioned to a two-week iterations. Team Ateam found estimating and planning for a two-week periodeasier than planning for a four week period. They alsofound the move to a two-week iteration allowed them torespond faster to changing business requirements reducedrisk because the team was able to address issues morerapidly through more frequent iterations.The teams performed “just-in-time” design of featuresbefore or during the iteration in which the feature was to bedeveloped. The form of the design was often class diagramsand annotations on interactions with other majorcomponents.Team A team members in Shanghaisometimes created a prototype (called a “spike” among agilesoftware developers) of larger features or those withsignificant unknowns before the feature was accepted into aSprint. The purpose of the prototype was to gain knowledgeabout the feature prior to accepting the feature into a Sprint.Only when enough knowledge was available for the featurewould it be considered ready for a Sprint. Such delaying ofstories until adequate information is available was also doneby the Systematic team discussed in Section III [18].The teams only held the Daily Scrum three times per weekin Redmond, Washington, USA. Subsequently, a Redmondteam member would follow up with a call to their Chinesecolleagues. The teams conducted retrospective meetings atthe end of every Sprint.2) Planning PokerThe teams used Planning Poker to estimate the personhours required to complete functionality within an iteration.In recent years, some agile software development teamshave estimated the effort needed to complete therequirements chosen to be implemented in an Sprint and/orin a release via a Wideband Delphi [8] practice commonlycalled Planning Poker [11, 14]. Several reports on the use ofPlanning Poker are found in the literature. One Norwegianindustrial team “immediately took a liking to the newestimation process” [15] of Planning Poker, and thetechnique was “very well received.” Another Norwegian
TABLE I: MICROSOFT TEAM CONTEXT FACTORSTeam ATeam BTeam CScrumScrumScrum4Redmond Shanghai319RedmondRedmond BeijingExperience 10 years111Experience 6-10 years2010Experience 5 wMedium/HighProject ManagementTypeTeam SizeTeam LocationDomain ExpertiseLanguage ExpertiseProgram ManagerExpertiseProgramming Lang.C#C#C DistributedLocalDistributedInfrastructureTest InfrastructureMobile Web1st Release1st Release1st ReleaseSource LoC24,9528,82631,399Test LoC20,9124,03126,2830.840.460.8482%53%N/A14 months11 months18 monthsnoEvery checkin/dailynoEvery checkin/dailynoEvery check-in/daily14 P1, 56 P276 P1, 111 P28 P1, 141 P2OfficesOnsite, email, chat,phoneOfficesOnsite, email,chat, phoneOffice/shared spacepartners, in person,emailOn-Site, RemoteOn-SiteN/ATeam LocationDomainVersion/LegacyTest LoC / Source LoC% of Code Coverage(unit-tests)Development TimeLegacy CodeTest Run FrequencyActual Defects(Sev 1,2,3,4)Physical LayoutCustomerCommunicationCustomer Cardinalityand Locationteam [25] “decided to implement it for all tasks in theproject’s future” because they felt it was an effective meansof collaboratively producing unbiased estimates.Planning Poker is “played” by the team as a part of theSprint Planning meeting. A Planning Poker session beginsby the customer or marketing representative explaining eachrequirement to the extended development team. We use theterm extended development team (often called the “wholeteam” [4] by agile software developers) to refer to all thoseinvolved in the development of a product, including productmanagers, project managers, software developers, testers,usability engineers, security engineers and others. In turn,the team discusses the work involved in fully implementingand testing a requirement until they believe that they haveenough information to estimate the effort. Each teammember then privately and independently estimates theeffort.The team members reveal their estimatessimultaneously. Next, the team members with the lowestand highest estimate explain their estimates to the group.Discussion ensues until the group is ready to re-vote on theirestimates. More estimation rounds take place until the teamcan come to a consensus on an effort estimate for therequirement. Most often, only one or two Planning Pokerrounds are necessary on a particular requirement beforeconsensus is reached.Planning Poker provides a structured means for:
obtaining a shared understanding;exposing hidden assumptions of the technicalaspects of implementation and verification; discussing the implications throughout thesystem for implementing a requirement; surfacing and resolving ambiguities realizedvia divergent perspectives on the requirement;and exposing easy and hard alternatives forachieving desired goals.The Microsoft teams felt the use of Planning Pokerenabled their team to have relatively low estimation errorfrom the beginning of the project. Figure 1 depicts theestimation error for Team A (the middle line) relative to thecone of uncertainty (the outer lines).The cone ofuncertainty is a concept introduced by Boehm [8] and madeprominent more recently by McConnell [24] based upon theidea that uncertainty decreases significantly as one obtainsnew knowledge as the project progresses [22]. Team A’sestimation accuracy was relatively low starting from thefirst iteration. The team attributes their accuracy to the useof the Planning Poker practice. 3) Continuous Integration300%The teams utilized the continuous integration practice.Continuous integration is a software development practicewhere members of a team integrate their work into the mainbuild system frequently. Usually each developer integratesat least daily. Each integration is verified by an automatedprocess that runs all automated tests that should detectintegration errors as quickly as possible.As shown in Table I, the Microsoft teams checked intheir new code at least once per day. Each check-in initiateda build. Each build entailed the running of automated unittests and associated test coverage computation. The teamautomatically received an email confirmation of thecompletion of the build providing test results. All tests mustpass for the build to be considered successful. The usedMicrosoft Visual Studio Team Build Server to manage theircheck-ins, build process, and automated test runs.The Microsoft teams managed their build/continuousintegration process themselves rather than getting help froma build support organization. They indicated that thebenefits from continuous integration’s ability to keep aconstant focus on quality come with a cost. Builds and
Keywords with the- Agile software development, Scrum I. INTRODUCTION Scrum [16, 29] is the most often used [6, 30, 31] agile [10] software development methodology among teams that utilize an agile methodology. A large-scale survey [31] deployed in the software engineering industry from June/July 2008 received 3061 respondents from 80 different countries. For the question “Which Agile .
This Scrum and Scrum Master Guide is a free, quick reference material designed to help aspiring scrum masters discover the ins and outs of Scrum. It throws light on the fundamental principles of the scrum, scrum terminologies, Agile Manifesto, scrum theories, scrum tools, different roles, responsibilities, and more. SCRUM & SCRUM MASTER
enacted. Scrum Masters do this by ensuring that the Scrum Team adheres to Scrum theory, practices, and rules. The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the
Scrum framework, the Scrum Master and the Scrum Product Owner share the role and responsibilities of a typical project manager. Nonetheless, a Scrum Master or a Scrum Product is never allowed to overrule the democratic decision-making capability of a Scrum Team. For instance, only the Scrum team members can
Agile development method - Scrum is one of the growing development methods in software projects [13]. Scrum is a process skeleton that includes a set of practices and predefined roles [13, 14]. The Scrum team composed of Scrum master, Product owner and development team. A set of practices include Scrum sprint and Scrum meetings.
scrums”. Scrum rules are product owner, scrum master and team. Scrum is easy with changes; it accommodates with changes. Scrum [1][5][6] is a simple framework used to higher quality. Scrum is easy with changes; it accommodates with changes. Some key scrum practices are discussed below [1][3][4][5].
The Scrum Master Finally has some Authority .11 Conclusion .12 Purpose of Analysis In practice, Scrum is a vague concept. There are many different, incompatible, kinds of Scrum; and for each of these kinds of Scrum, there can be different descriptions. We like the Scrum that is described in the 2017 Scrum Guide, but we .
Method Scrum Scrum Scrum Scrumban Scrum Scrum Scrum Scrum Size 24 PM 30 PM 30 PM 100 PM 30 PM 12 PM 72 PM 120 PM Duration 3 M . Continuous delivery Delivery on time testing on unit le
challenges Training (Scrum Master, Product Owner, Agile Leadership, online courses, etc.) Consulting (linking Scrum and business strategy, customizing Scrum) Coaching (hands-on support to Scrum teams) Find out more at www.scruminc.com. We run our company using Scrum as the primary management framework, making us a living
