Rolling Out Agile In A Large Enterprise

Rolling out Agile in a Large EnterpriseGabrielle tYahoo! is a large enterprise with a 32 billion market capand has one of the largest Agile implementations in theworld. The adoption of Scrum and Agile practices hasbeen steadily growing over the past two years, and nowencompasses more than 150 Yahoo! teams in the UnitedStates, Europe, and Asia-Pacific. The projects range fromnew product development for properties such as Yahoo!Autos to heavy-duty infrastructure work on Yahoo! Mail,which serves 250 million users each month around theglobe.1. IntroductionIn the highly competitive Internet space, gettingproducts to market quickly while being both flexible andadaptive to change is critical. Yahoo! needed a softwaredevelopment process that could support an Internetstartup culture within the structure of a company thatprovides products and services to more than 500 millionusers worldwide. I joined Yahoo! in 2005 to help Yahoo!adopt and utilize the highly effective frameworks ofScrum and Agile throughout the organization. We startedwith Scrum, using its lightweight framework to createhighly collaborative self-organizing teams that couldeffectively deliver products to market. Next we started toadd in Agile engineering practices and Lean fundamentalsto deliver greater business value and reduceorganizational waste. In the two plus years since thiseffort began, there have been tremendous successes andvaluable lessons learned. This is a retrospective look atmy experiences in implementing Agile at a largecompany.2. BackgroundYahoo! went from being a small startup to a largeenterprise company quickly. As such, the culture atYahoo! is very much like a large startup. There is aconstant stream of innovative ideas and product launchesas the company strives to be the first to market with newservices, while continuing to meet the everyday needs ofits users. The company wanted to preserve the feeling ofbeing a startup, but also recognized the need to adoptstandard process and practices to help teams deliver betterproducts faster.Yahoo!’s first effort to meld its culture with amanaged software development process began in 2002with the release of a globally mandated waterfall processcalled the “Product Development Process” (PDP).Unfortunately, many teams simply ignored the process or,where they couldn’t ignore it, paid lip service and made itlook like they had adhered to the steps retroactively. Theteams that did follow the PDP found that it was heavy,slowed them down, and added little real value.Some grass-roots efforts to experiment with Agilepractices began to emerge within the company and inNovember of 2004, Tobias Mayer, an engineer on anAgile team invited Jeff Sutherland, one of the inventors ofScrum, to speak at Yahoo!. Pete Deemer, the VP ofProduct Development attended Jeff’s talk and was veryenthusiastic about what he heard. He asked Jeff to returnand speak to the senior executive team at an offsite. Thistalk was very well received, and sparked a lively internaldiscussion about how Yahoo! should embrace Agiledevelopment methods. The debate was whether to initiatea large-scale, top-down “forced adoption”, or to try tocultivate adoption from the ground-up in a grassrootsfashion. The decision was made to proceed with the latterapproach, and Pete Deemer was given a budget andmandate to lead this effort.3. KickoffAt the time, the most visible symptoms ofdysfunction in Yahoo! product development were at theproject and team layer (centered around issues ofplanning, project management, release management, andteam interactions), rather than at the technical practices ortools layer. As a result, Yahoo!’s initial focus was on theadoption of Scrum. There was active debate aboutwhether Agile engineering practices should also beadopting in parallel; in hindsight, it would have

accelerated the benefits had they been. But the constraintsof time and budget, as well as a nervousness about “tryingto make the elephant dance too fast, too soon” led to thepostponement of those efforts initially.Yahoo! launched its Scrum pilot program inFebruary, 2005. Four teams volunteered to try Scrum andshare their experiences with the rest of the company. Theteams covered a broad set of products and servicesincluding the Yahoo! Photos 3.0, a new backend forYahoo! Mail, internal tools for managing small businesssites, and a media site redesign.The initial commitment made by the teams was (i) tocomplete comprehensive Scrum training (which translatedinto Certified ScrumMaster training for most members ofthe team); (ii) to work with outside Scrum coaches duringthe first several Sprints; (iii) to use all the standard Scrumpractices described in Ken Schwaber’s “Agile ProjectManagement with Scrum”; and (iv) to complete at leastone Sprint (the term of art for a 30-day iteration inScrum). It was made clear to the teams that after the firstSprint they could at any time choose to abandon Scrum ifthey found it unsatisfactory.At the end of the two months, the feedback was forthe most part very positive; the teams liked the processand experience, and management saw positive results.What’s more, the positive word-of-mouth was spreadingwithin the company, and other teams were beginning toexpress interest.3.1. Roles and ResponsibilitiesThe pilot teams received some early coaching andtraining from leading Agile thinkers, Ken Schwaber, PaulHodgetts, Mike Cohn, and Esther Derby. I was initiallybrought in as a consultant as well. As the programexpanded, I accepted a permanent position to build aninternal coaching team, working with Pete Deemer toevangelize the benefits of Scrum throughout the company,and to train, coach and support the teams that wished touse it. With just two dedicated coaches in-house andburgeoning demand, we continued to work with externalconsultants to ensure that teams had as much support aspossible in the often difficult early stages of their use ofScrum.The coaching at this early stage consisted offacilitating key events in the Scrum methodology, such asiteration planning and retrospectives, attending dailystand-up meetings and working with key team members tohelp answer questions and provide guidance on anas-needed basis. Also, because the early Scrum teamsuncovered broader systemic impediments, we tried toknock them down as we went so that the issues we solvedfor one team would be solved for many. These solutionsincluded working with facilities to secure meeting roomsand take down cube walls, removing governance gateswhere processes were overly bureaucratic, and changingthe way we conducted resource planning and portfoliomanagement. We did not want individual Scrum teamsburdened with breaking down resistance from uppermanagement on extraneous issues and were keen toreduce the paperwork and process gates that these earlyteams would face.3.2. Tracking ProgressThroughout the transition, we were very active inreaching out across the organization to solicit feedback.We tried to be as honest and transparent as possible,capturing and presenting the challenges along withsuccesses.Many teams didn’t want to try something new unlessit was tested and proven a success with Yahoo! teamsalready. To help combat this problem, at the end of theirfirst month of using Scrum we had all team members andtheir managers participate in an online survey toanonymously provide their feedback. As an incentive, theresponders received a custom-printed Scrum t-shirt forparticipating.The overall response rate to the survey was 71percent (85 percent for Scrum pilot team members). Therespondents rated their experiences against the previousprocesses they had used. The data revealed importantinsight into both how Scrum had benefited the Yahoo!teams that had participated so far and also their greatestpain points. The survey demonstrated the power of Scrumand provided a strong incentive for other teams toovercome their doubts and get on board.Figure 1. 74 percent of respondents said Scrum improvedthirty-day productivity

Figure 2. 80 percent of respondents said Scrum helpedclarify team goals.Figure 5. 89 percent of respondents said Scrum helpedcollaboration and cooperation with the team.Figure 3. 64 percent of respondents felt Scrum improvedthe business value of their product at the thirty-day mark.Figure 6. 68 percent of those surveyed said Scrum helpedreduce the amount of time wasted.Figure 4. 54 percent of respondents said Scrum improvedoverall quality.Figure 7. 77 percent of those surveyed had positivefeelings concerning Scrum.

we found that the worst time to be trying to explain Scrumto a senior manager was in the midst of a difficultsituation where Scrum was the apparent source of theproblem; since Scrum tends to make all dysfunctionvisible, it is often unfairly blamed for the bad news itbrings up. Similarly, we tried hard not to “oversell”Scrum to management, because we were concerned thatScrum might be misinterpreted as a silver bullet. Wewanted to ensure that people understood the hard workrequired by such a major change.3.4. Coaching ModelFigure 8. 81 percent of respondents wanted to continueusing Scrum.We continued to track the overall happiness of teamswho were using Scrum as a baseline metric to show howwe were trending as we scaled. The number has remainedfairly consistent over the last few years.Figure 9. Over the past three years, the number ofrespondents who want to continue using Scrum hasremained consistent.3.3. Management SupportWe noticed across the board that management felt morecomfortable with our progress when they heard realfeedback from their peers. We tended to focus on thebenefits rather than on the mechanics of Scrum, as generalmanagers are typically not interested in the specifics ofthe process as long as it generates positive results. This inhindsight may have led to some problems; those seniormanagers that had a clear understanding of the principlesand practices of Scrum tended to be much better atsupporting their teams use of Scrum, not surprisingly; andOnce we laid the groundwork for the pilot program,we experimented with an engagement model that allowedus to coach multiple teams effectively. We wanted towork closely with teams until kickoff and then slowlywean them off our services.While designing the model we tried to be sensitive tothe fact that no two teams would face the same challengesin implementing Scrum, nor would they have the samesolutions. Some teams might need revolution and rapidchange to get rid of major dysfunction, while other teamsmight need gentle persuasion and understanding. Weneeded to allow time for coaches to learn the individualneeds of a team and be able to help them tailor the Scrumframework specifically for them. We also made theconscious decision not to try to prescribe Scrumthroughout the organization all at once, or to allow teamsto be forced into Scrum by others; the only teams thatwould use Scrum were the teams that wished to useScrum.Our initial rollout strategy consisted of the followingengagement model:Initial discussion Meet with people interested in Scrum anddiscuss their context and challenges. Schedule an overview for key members of theteam. Organize training and coaching, includingScrum Master training and team training.Preparation Work with the Product Owner to prepare theProduct Backlog (Scrum’s prioritization of workitems).Training Conduct two-day Scrum training for the wholeteam.CoachingFor the first sprint, the Agile coach would facilitate thefollowing events: First sprint planning meeting First sprint review (including setup)