Mastering Cloud Computing - Buyya

Contents7.1.3 Frameworks for Task Computing7.2 Task-based Application Models7.47.57.2.1 Embarrassingly Parallel Applications7.57.2.2 Parameter Sweep Applications7.67.2.3 MPI Applications7.2.4 Workflow Applications with Task Dependencies7.3 Aneka Task-Based Programming7.3.1 Task Programming Model7.87.107.137.137.3.2 Developing Applications with the Task Model7.147.3.3 Developing Parameter Sweep Application7.307.3.4 Managing Workflows7.34Summary7.36Review Questions7.37Chapter 8—Data Intensive Computing: Map-Reduce Programming8.1 What is Data-Intensive Computing?8.18.18.1.1 Characterizing Data-Intensive Computations8.28.1.2 Challenges Ahead8.28.1.3 Historical Perspective8.38.2 Technologies for Data-Intensive Computing8.2.1 Storage Systems8.2.2 Programming Platforms8.3 Aneka MapReduce Programming8.3.1 Introducing the MapReduce Programming Model8.3.2 Example Application8.78.78.148.208.218.44Summary8.56Review Questions8.56Chapter 9—Cloud Platforms in Industry9.19.1 Amazon Web Services9.19.1.1 Compute Services9.29.1.2 Storage Services9.1.3 Communication Services9.79.149.1.4 Additional Services9.159.1.5 Summary9.169.2 Google AppEngine9.169.2.1 Architecture and Core Concepts9.169.2.2 Application Life-Cycle9.219.2.3 Cost Model9.239.2.4 Observations9.239.3 Microsoft AzureFM.indd ixix9.249.3.1 Azure Core Concepts9.249.3.2 SQL Azure9.281/28/2013 12:11:01 PM

xContents9.3.3 Windows Azure Platform Appliance9.309.3.4 Summary9.309.4 Observations9.31Review Questions9.31Chapter 10—Cloud Applications10.1 Scientific Applications10.110.1.1 Healthcare: ECG Analysis in the Cloud10.110.1.2 Biology: Protein Structure Prediction10.310.1.3 Biology: Gene Expression Data Analysis for Cancer Diagnosis10.410.1.4 Geoscience: Satellite Image Processing10.510.2 Business and Consumer Applications10.2.1 CRM and ERP10.2.2 Productivity10.610.610.810.2.3 Social Networking10.1110.2.4 Media Applications10.1210.2.5 Multiplayer Online Gaming10.15Summary10.16Review Questions10.17Chapter 11 —Advanced Topics in Cloud Computing11.1 Energy Efficiency in Clouds11.1.1 Energy-Efficient and Green Cloud Computing Architecture11.111.111.311.2 Market Based Management of Clouds11.511.2.1 Market-Oriented Cloud Computing11.511.2.2 A Reference Model for MOCC11.611.2.3 Technologies and Initiatives Supporting MOCC11.1111.2.4 Observations11.1611.3 Federated Clouds / InterCloud11.1611.3.1 Characterization and Definition11.1611.3.2 Cloud Federation Stack11.1711.3.3 Aspects of Interest11.2211.3.4 Technologies for Cloud Federations11.3811.3.5 Observations11.4111.4 Third Party Cloud Services11.4.1 MetaCDN11.4.2 SpotCloud11.4211.4211.43Summary11.44Review Questions11.45ReferencesIndexFM.indd x10.1R.1I.11/28/2013 12:11:01 PM

PrefaceThe growing popularity of the Internet and the Web along with the availability of powerful hand-heldcomputing, mobile and sensing devices are changing the way we interact, manage our lives, conductbusiness, and access or deliver services. The lowering costs of computation and communication aredriving the focus from personal to Data Center-centric computing. Although parallel and distributedcomputing has been around for several years, its new forms, Multicore and Cloud computing, havebrought about a sweeping change in the industry. These trends are pushing the industry focus from developing applications for PCs to Cloud Data Centers enabling millions of users to make use of softwaresimultaneously.Computing is being transformed to a model consisting of commoditised services delivered in a manner similar to utilities such as water, electricity, gas, and telephony. As a result, IT (Information Technology) services are billed and delivered as “computing utilities” over shared delivery networks akin to thewater, electricity, gas and telephony services. In such a model, users access services based on theirrequirements regardless of where they are hosted. Several computing paradigms have promised todeliver this utility-computing vision. Cloud computing is the most recent emerging paradigm promisingto turn the vision of “computing utilities” into a reality.Cloud computing has become one of the buzzwords in the IT industry. Several IT vendors arepromising to offer storage, computation and application-hosting services, and provide coveragein several continents, offering Service-Level Agreements (SLA) backed performance and uptimepromises for their services. They offer subscription-based access to infrastructure, platforms, andapplications popularly termed IaaS (Infrastructure as a Service), PaaS (Platform as a Service), andSaaS (Software as a Service). Whilst these emerging services have reduced the cost of computationand application hosting by several orders of magnitude, there is a significant complexity involved inthe development and delivery of applications and their services in a seamless, scalable, and reliablemanner.There exist several Cloud technologies and platforms in the market. To mention a few: Google AppEngine, Microsoft Azure, and Manjrasoft Aneka. Google AppEngine provides an extensible runtimeenvironment for Web-based applications, which leverage huge Google IT infrastructure. MicrsoftAzure provides a wide array of Windows-based services for developing and deploying Windowsbased applications on the Cloud. Manjrasoft Aneka provides a flexible model for creating Cloud applications and deploying them on a wide variety of infrastructures including public Clouds such asAmazon EC2.With this sweeping shift from developing applications from PCs to Data Centres, there is a huge demand for manpower with new skill sets in Cloud computing. Universities play an important role inthis regard by training the next generation of IT professionals and equipping them with the necessarytools and knowledge to tackle these challenges. They need to be able to set up a Cloud computingenvironment for teaching and learning with a minimal investment.Currently, expert developers are required to create Cloud applications and services. Cloud researchers, practitioners, and vendors alike are working to ensure potential users are educated about thebenefits of Cloud computing and the best way to harness its full potential. However, being a new andpopular paradigm, the very definition of Cloud computing depends on which computing expert is asked.So, while the realization of true utility computing appears closer than ever, its acceptance is currentlyrestricted to Cloud experts due to the perceived complexities of interacting with Cloud computing providers. This book aims to change the game by simplifying and imparting Cloud computing foundations,technologies, and programming skills to readers such that even the average programmers and softwareengineers are able to develop Cloud applications easily.FM.indd xi1/28/2013 12:11:01 PM

xiiPrefaceSalient Features Introduction to Cloud Computing, Cloud Architecture, Cloud Applications, Programming of Clouds,and Cloud PlatformsFocuses on the platforms and technologies essential to Cloud Computing like Google AppEngine,Microsoft Azure, and Manjrasoft AnekaDedicated chapter on Aneka: A Software Platform for .NET-based Cloud ComputingDetailed coverage to Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructureas a Service (IaaS), Server and Desktop VirtualizationPedagogy: 120 Illustrations 191 Review Questions Footnotes interspersed within chapter contentsThe Book at a GlanceThis book introduces the fundamental principles of Cloud computing and its related paradigms. It discusses the concepts of virtualization technologies along with the architectural models of Cloud computing. It presents prominent Cloud computing technologies available in the marketplace. It contains dedicated chapters on concurrent, high-throughput and data-intensive computing paradigms and their usein programming Cloud applications. Various application case studies from domains such as science,engineering, gaming, and social networking are introduced along with their architecture and how theyleverage various Cloud technologies. This allows the reader to understand the mechanisms needed toharness Cloud computing in their own respective endeavors. Finally, many open research problems andopportunities that have arisen from the rapid uptake of Cloud computing are detailed. We hope that thismotivates the reader to address these in their own future research and development.The book contains 11 chapters, which can be organized into three major parts:Part I: FoundationsChapter 1—IntroductionChapter 2—Principles of Parallel and Distributed ComputingChapter 3—VirtualizationChapter 4—Cloud Computing ArchitecturePart II: Cloud Application Programming and the Aneka PlatformChapter 5—Aneka: Cloud Application PlatformChapter 6—Concurrent Computing: Thread ProgrammingChapter 7—High-Throughput Computing: Task ProgrammingChapter 8—Data Intensive Computing: Map-Reduce ProgrammingPart III: Industrial Platforms and New DevelopmentsChapter 9—Cloud Platforms in IndustryChapter 10—Cloud ApplicationsChapter 11—Advanced Topics in Cloud ComputingThe book serves as a perfect guide into the world of Cloud computing. By starting from the fundamentals, the book drives students and professionals through the practical use of these concepts by handson sessions on how to develop Cloud applications by using Aneka, Amazon Web Services, GoogleAppEngine and Microsoft Azure. The last part introduces real applications, identifies emerging trendsand offers future directions of cloud computing.FM.indd xii1/28/2013 12:11:01 PM

PrefacexiiiOnline Learning CenterThe book also comes with an associated Web site (hosted at http://www.mhhe.com/buyya/mcc1 andhttp://www.buyya.com/MasteringClouds) containing pointers to additional online resources, PowerPointslides and research papers.Benefits and ReadershipGiven the rapid emergence of Cloud computing as a mainstream computing paradigm, it is essential toboth have a solid understanding of the core concepts characterizing the phenomenon and a practicalgrasp of how to design and implement Cloud computing applications and systems. This set of skills isalready fundamental today for software architects, engineers, and developers as many applications arebeing moved to the Cloud. It will become even more important in the future when this technology matures further. This book provides a perfect blend of background information, theory, and practical Cloudcomputing development, expressed in a language that is accessible to a wide range of readers: fromgraduate-level students to practitioners, developers, and engineers who want to, or need to, design andimplement Cloud computing solutions. Moreover, more advanced topics presented at the end of themanuscript, make the book an interesting reading for researchers in the field of Cloud computing thatare can get an overview of the next challenges in Cloud computing for the coming years.This book is a timely contribution to the Cloud computing field that is gaining considerable commercial interest and momentum. The book is targeted at graduate students and IT professionals such assystem architects, practitioners, software engineers, and application programmers. As Cloud computingis recognised as one of the top five emerging technologies that will have a major impact on the quality ofscience and society over the next 20 years, its knowledge will help position our readers at the forefrontof the field.Directions for Adoption: Theory, Laboratories, and ProjectsGiven the importance of the Cloud computing paradigm and its rapid uptake in industry, universities/educational institutions need to upgrade their curriculum by introducing one or more subjects in the areaof Cloud computing and related topics such as parallel computing and distributed systems. We recommend that they offer at least one subject on Cloud computing as part of their undergraduate and postgraduate degree programs such as BE/B.Tech./BSc (Hons) in Computer Science and related areas;and Masters including the MCA (Master of Computer Applications). We believe that this book will serveas an excellent textbook for such subjects. If the students have already had exposure to the concepts ofparallel and distributed computing, Chapter 2 can be skipped.For those aiming to make their curriculum rich with Cloud computing, we recommend to offer twosubjects: “Introduction to Cloud Computing” and “Advanced Cloud Computing” in two different semesters. This book has sufficient content to cater to both of them. The first subject can be based on Chapters 1 to 6 and the second one based on Chapters 7 to 11.In addition to theory, we strongly recommend for the introduction of a laboratory subject thatoffers hands-on experience. The lab exercises and assignments can focus on creating high-performance Cloud applications and assignments on a range of topics including parallel execution ofmathematical functions, sorting of large data in parallel, image processing, and data mining. By usingCloud software systems, institutions can easily set up a private/enterprise Cloud computing facility byutilising existing LAN-connected PCs running Windows. Students can make use of this facility to learnabout various Cloud application programming models and interfaces discussed in Chapter 6 (ThreadProgramming), Chapter 7 (Task Programming), and Chapter 8 (MapReduce Programming). Studentsneed to learn various programming examples discussed in these chapters and execute them onCloud facility. We encourage students to take up some of programming exercises noted in the Review Questions section of these chapters as lab assignments and develop their own solutions.FM.indd xiii1/28/2013 12:11:01 PM

xivPrefaceStudents can also carry out their final-year projects focused on developing Cloud applications solvingreal world problems. For example, students can work with academics/researchers/experts from otherscience and engineering disciplines such as Life and Medical Sciences or Civil and Mechanical Engineering and develop suitable applications that can harness the power of Cloud computing. For inspiration, please read various application case studies presented in Chapter 11.Rajkumar BuyyaChristian VecchiolaS Thamarai SelviPublisher’s NoteDo you have any further request or a suggestion? We are always open to new ideas (the best onescome from you!). You may send your comments to tmh.csefeedback@gmail.comPiracy-related issues may also be reported!FM.indd xiv1/28/2013 12:11:01 PM

AcknowledgmentsFirst and foremost, we are grateful to all researchers and industrial developers worldwide for their contributions to various concepts and technologies discussed in the book. Our special thanks to all the members/consultants of Manjrasoft, CLOUDS (Cloud Computing and Distributed Systems) Lab from theUniversity of Melbourne, and Melbourne Ventures who contributed towards the development of AnekaCloud Application Platform, the preparation of associated application demonstrators and documents,and/or the commercialization of the Aneka technology. They include Chu Xingchen, Srikumar Venugopal, Krishna Nadiminti, Dileban Karunamoorthy, Chao Jin, Rodrigo Calheiros, Michael Mattess, JessieWei, Enayat Masoumi, Ivan Mellado, Richard Day, Wolfgang Gentzsch, Laurence Liew, David Sinclair,Suraj Pandey, Abhi Shekar, Dexter Duncan, Murali Sathya, Karthik Sukumar, and Sita Venkatraman.We are grateful to the Australian Research Council (ARC) and the Department of Innovation, Industry, Science and Research (DIISR) for supporting our research and commercialization endeavors.We thank all of our colleagues at the University of Melbourne especially Professors Rao Kotagiri,Iven Mareels, and Glyn Davis for their mentorship and positive support for our research and knowledgetransfer efforts.A note of appreciation is due to all colleagues and users of the Aneka technology for their direct/indirect contributions towards application case studies reported in the book. Our special thanks to Raghavendra from ADRIN/ISRO for his enthusiastic efforts in creating satellite image-processing applicationusing Aneka and publishing articles in this area. Srinivasa Iyengar from MSRIT deserves a specialmention for creating data-mining applications using Aneka and demonstrating the power of Aneka toacademics from the early days of Cloud computing.We thank the members of the CLOUDS Lab for proofreading one or more chapters. They includeRodrigo Calheiros, Nikolay Grozev, Amir Vahid, Anton Beloglazov, Adel Toosi, Deepak Poola, Mohammed AlRokayan, Atefeh Khosravi, Sareh Piraghaj, and Yaser Mansouri.Dr Buyya is grateful to his family members, including Smrithi Buyya, Soumya Buyya, and RadhaBuyya, for their love and understanding during the preparation of the book.We sincerely acknowledge external reviewers K Chandra Sekaran of NIT Shrathkal, Sateesh KPeddoju of IIT Roorkee, M P Sebastian of IIM Kozhikode and a few others for their critical commentsand suggestions on enhancing the presentation and organization of many chapters at a finer level. Thishas greatly helped us in improving the quality of the book.Finally, we would like to thank the staff at Tata McGraw Hill Education for their enthusiastic supportand guidance during the preparation of the book. In particular, Vibha Mahajan inspired us to take up thisproject and set the publication process in motion; Smruti Snigdha and Amiya Mahapatra for coordinatingthe publication process. They were wonderful to work with!FM.indd xv1/28/2013 12:11:02 PM

FM.indd xvi1/28/2013 12:11:02 PM

Guided TourCovers all important subject areasGuided Tour.indd xvii1/29/2013 10:27:36 AM

xviiiGuided Tour More than 100 illustrations and diagrams are present toenhance the concepts. Full-page figures add greater clarity to the subject.Guided Tour.indd xviii1/29/2013 10:27:37 AM

Guided TourxixEach chapter has an extensiveSummary for quick recapitulationof the concepts discussed.Review questions are given in eachchapter to test the student’s subjectivegrasp on the topics, terms and definitions, and revision of conceptsA Reference list is provided at the endof the book to help students find booksand journals for further reading.Guided Tour.indd xix1/29/2013 10:27:38 AM

Guided Tour.indd xx1/29/2013 10:27:39 AM

Chapter 1—Introduction 1.1 1.1 Cloud Computing at a Glance 1.1 1.1.1 The Vision of Cloud Computing 1.2 1.1.2 Defi ning a Cloud 1.4 1.1.3 A Closer Look 1.6 1.1.4 Cloud Computing Reference Model 1.8 1.1.5 Characteristics and Benefi ts 1.9 1.1.6 Challenges Ahead 1.10 1.2 Historical Developments 1.11 1.2.1 Distributed Systems 1.12