EXPERT’S VOICE IN OPEN SOURCE Practical MongoDB

E X P ER T ’S VOIC E IN OP EN S OURC EPracticalMongoDBArchitecting, Developing,and Administering MongoDB—Shakuntala Gupta EdwardNavin Sabharwalwww.allitebooks.com

Practical MongoDBArchitecting, Developing, andAdministering MongoDBShakuntala Gupta EdwardNavin Sabharwalwww.allitebooks.com

Practical MongoDB: Architecting, Developing, and Administering MongoDBShakuntala Gupta EdwardGhaziabad, Uttar Pradesh, IndiaNavin SabharwalNew Delhi, Delhi, IndiaISBN-13 (pbk): 978-1-4842-0648-5DOI 10.1007/978-1-4842-0647-8ISBN-13 (electronic): 978-1-4842-0647-8Library of Congress Control Number: 2015959699Copyright 2015 by Shakuntala Gupta Edward and Navin SabharwalThis work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the materialis concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting,reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval,electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafterdeveloped. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarlyanalysis or material supplied specifically for the purpose of being entered and executed on a computer system,for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted onlyunder the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for usemust always be obtained from Springer. Permissions for use may be obtained through RightsLink at the CopyrightClearance Center. Violations are liable to prosecution under the respective Copyright Law.Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with everyoccurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashionand to the benefit of the trademark owner, with no intention of infringement of the trademark.The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identifiedas such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.While the advice and information in this book are believed to be true and accurate at the date of publication, neitherthe authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that maybe made. The publisher makes no warranty, express or implied, with respect to the material contained herein.Managing Director: Welmoed SpahrAcquisitions Editor: Celestin Suresh JohnDevelopmental Editor: Douglas PundickTechnical Reviewer: Gopala ManchukundaEditorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Louise Corrigan,James DeWolf, Jonathan Gennick, Robert Hutchinson, Celestin Suresh John, Michelle Lowman,James Markham, Susan McDermott, Matthew Moodie, Jeffrey Pepper, Douglas Pundick,Ben Renow-Clarke, Gwenan Spearing, Matt Wade, Steve WeissCoordinating Editor: Rita FernandoCopy Editor: Mary BehrCompositor: SPi GlobalIndexer: SPi GlobalDistributed to the book trade worldwide by Springer Science Business Media New York, 233 Spring Street,6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com,or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is SpringerScience Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.For information on translations, please e-mail rights@apress.com, or visit www.apress.com.Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use.eBook versions and licenses are also available for most titles. For more information, reference our Special BulkSales–eBook Licensing web page at www.apress.com/bulk-sales.Any source code or other supplementary materials referenced by the author in this text is available to readers atwww.apress.com. For detailed information about how to locate your book’s source code, go to www.apress.com/source-code/.Printed on acid-free paperwww.allitebooks.com

Dedicated to people who made my life worth living and carved me into an individualI am today and to God who shades every step of my life.—Shakuntala Gupta EdwardDedicated to the people I love and the God I trust.—Navin Sabharwalwww.allitebooks.com

Contents at a GlanceAbout the Authors.xvAbout the Technical Reviewers .xviiAcknowledgments .xixPreface .xxi Chapter 1: Big Data. 1 Chapter 2: NoSQL . 13 Chapter 3: Introducing MongoDB . 25 Chapter 4: The MongoDB Data Model . 29 Chapter 5: MongoDB - Installation and Configuration . 35 Chapter 6: Using MongoDB Shell . 53 Chapter 7: MongoDB Architecture . 95 Chapter 8: MongoDB Explained . 159 Chapter 9: Administering MongoDB . 191 Chapter 10: MongoDB Use Cases. 213 Chapter 11: MongoDB Limitations . 227 Chapter 12: MongoDB Best Practices . 233Index . 243vwww.allitebooks.com

ContentsAbout the Authors.xvAbout the Technical Reviewers .xviiAcknowledgments .xixPreface .xxi Chapter 1: Big Data. 1Getting Started . 1Big Data . 3Facts About Big Data . 3Big Data Sources . 4Three Vs of Big Data . 6Volume . 7Variety. 8Velocity . 8Usage of Big Data . 9Visibility . 9Discover and Analyze Information . 9Segmentation and Customizations . 9Aiding Decision Making . 9Innovation . 9Big Data Challenges . 10Policies and Procedures . 10Access to Data . 10Technology and Techniques . 10viiwww.allitebooks.com

CONTENTSLegacy Systems and Big Data . 10Structure of Big Data . 10Data Storage . 11Data Processing. 11Big Data Technologies . 11Summary . 12 Chapter 2: NoSQL . 13SQL . 13NoSQL. 13Definition . 14A Brief History of NoSQL . 15ACID vs. BASE. 15CAP Theorem (Brewer’s Theorem) . 15The BASE . 16NoSQL Advantages and Disadvantages . 17Advantages of NoSQL . 17Disadvantages of NoSQL . 18SQL vs. NoSQL Databases . 18Categories of NoSQL Databases . 22Summary . 23 Chapter 3: Introducing MongoDB . 25History . 25MongoDB Design Philosophy. 26Speed, Scalability, and Agility . 26Non-Relational Approach . 26JSON-Based Document Store . 26Performance vs. Features . 27Running the Database Anywhere. 27SQL Comparison . 27Summary . 28viiiwww.allitebooks.com

CONTENTS Chapter 4: The MongoDB Data Model . 29The Data Model . 29JSON and BSON . 31The Identifier ( id) . 32Capped Collection . 32Polymorphic Schemas . 32Object-Oriented Programming. 32Schema Evolution . 33Summary . 34 Chapter 5: MongoDB - Installation and Configuration . 35Select Your Version . 35Installing MongoDB on Linux . 36Installing Using Repositories . 36Installing Manually . 36Installing MongoDB on Windows . 37Running MongoDB. 37Preconditions . 37Starting the Service . 38Verifying the Installation. 38MongoDB Shell . 38Securing the Deployment . 39Using Authentication and Authorization . 39Controlling Access to a Network. 44Provisioning Using MongoDB Cloud Manager . 47Summary . 52 Chapter 6: Using MongoDB Shell . 53Basic Querying . 53Create and Insert . 58Explicitly Creating Collections . 60Inserting Documents Using Loop . 60ixwww.allitebooks.com

CONTENTSInserting by Explicitly Specifying id . 60Update . 61Delete . 62Read . 63Using Indexes . 69Stepping Beyond the Basics. 78Using Conditional Operators . 79Regular Expressions . 81MapReduce. 82aggregate( ). 83Designing an Application’s Data Model . 84Relational Data Modeling and Normalization . 84MongoDB Document Data Model Approach . 86Summary . 93 Chapter 7: MongoDB Architecture . 95Core Processes. 95mongod. 95mongo. 96mongos . 96MongoDB Tools . 96Standalone Deployment . 96Replication . 97Master/Slave Replication . 97Replica Set . 98Implementing Advanced Clustering with Replica Sets . 115Sharding . 124Sharding Components . 125Data Distribution Process . 127Data Balancing Process . 130xwww.allitebooks.com

CONTENTSOperations . 133Implementing Sharding . 134Controlling Collection Distribution (Tag-Based Sharding) . 142Points to Remember When Importing Data in a ShardedEnvironment . 151Monitoring for Sharding. 152Monitoring the Config Servers . 152Production Cluster Architecture . 152Scenario 1 . 153Scenario 2 . 154Scenario 3 . 155Scenario 4 . 156Summary . 157 Chapter 8: MongoDB Explained . 159Data Storage Engine. 159Data File (Relevant for MMAPv1) . 161Namespace (.ns File) . 162Data File (Relevant for WiredTiger) . 170Reads and Writes . 172How Data Is Written Using Journaling . 174GridFS – The MongoDB File System . 178The Rationale of GridFS . 178GridFSunder the Hood . 179Using GridFS . 180Indexing . 183Types of Indexes . 184Behaviors and Limitations . 190Summary . 190xiwww.allitebooks.com

CONTENTS Chapter 9: Administering MongoDB . 191Administration Tools . 191mongo. 191Third-Party Administration Tools. 191Backup and Recovery. 192Data File Backup. 192mongodump and mongorestore . 192fsync and Lock. 196Slave Backups . 197Importing and Exporting . 197mongoimport . 198mongoexport. 198Managing the Server . 199Starting a Server . 199Stopping a Server . 200Viewing Log Files. 200Server Status . 200Identifying and Repairing MongoDB . 202Identifying and Repairing Collection Level Data . 203Monitoring MongoDB. 204mongostat. 204mongod Web Interface. 205Third-Party Plug-Ins . 205MongoDB Cloud Manager . 206Summary . 212 Chapter 10: MongoDB Use Cases. 213Use Case 1 - Performance Monitoring . 213Schema Design. 213Operations . 214Sharding . 218Managing the Data . 219xii

CONTENTSUse Case 2 – Social Networking .

The authors can be reached at architectbigdata@gmail.com . xvii About the echnical T Reviewers Pron aso mar Ku is a seasoned technology professional and trainer with more than 18 years of strong experience in build