Docker In Practice - Apphosting.io

2y ago
127 Views
15 Downloads
6.37 MB
370 Pages
Last View : Today
Last Download : 2m ago
Upload by : Albert Barnett
Transcription

INPRACTICEIan MiellAidan Hobson SayersFOREWORD BY Ben FirshmanMANNINGwww.allitebooks.com

Docker in Practicewww.allitebooks.com

www.allitebooks.com

Docker in PracticeIAN MIELLAIDAN HOBSON SAYERSMANNINGSHELTER ISLANDwww.allitebooks.com

For online information and ordering of this and other Manning books, please visitwww.manning.com. The publisher offers discounts on this book when ordered in quantity.For more information, please contactSpecial Sales DepartmentManning Publications Co.20 Baldwin RoadPO Box 761Shelter Island, NY 11964Email: orders@manning.com 2016 by Manning Publications Co. All rights reserved.No part of this publication may be reproduced, stored in a retrieval system, or transmitted, inany form or by means electronic, mechanical, photocopying, or otherwise, without prior writtenpermission of the publisher.Many of the designations used by manufacturers and sellers to distinguish their products areclaimed as trademarks. Where those designations appear in the book, and ManningPublications was aware of a trademark claim, the designations have been printed in initial capsor all caps.Recognizing the importance of preserving what has been written, it is Manning’s policy to havethe books we publish printed on acid-free paper, and we exert our best efforts to that end.Recognizing also our responsibility to conserve the resources of our planet, Manning books areprinted on paper that is at least 15 percent recycled and processed without the use of elementalchlorine.Manning Publications Co.20 Baldwin RoadPO Box 761Shelter Island, NY 11964Development editor: Cynthia KaneTechnical development editors: Alain Couniotand Robert WennerCopyeditor: Andy CarrollProofreader: Melody DolabTechnical proofreader: José San LeandroTypesetter: Gordan SalinovicCover designer: Marija TudorISBN 9781617292729Printed in the United States of America1 2 3 4 5 6 7 8 9 10 – EBM – 21 20 19 18 17 16www.allitebooks.com

brief contentsPART 1PART 2PART 3DOCKER FUNDAMENTALS .11 Discovering Docker32 Understanding Docker—inside the engine room19DOCKER AND DEVELOPMENT .413 Using Docker as a lightweight virtual machine434 Day-to-day Docker5 Configuration management—getting your house inorder 10365DOCKER AND DEVOPS.1436 Continuous integration: speeding up your developmentpipeline 1457 Continuous delivery: a perfect fit for Dockerprinciples 1698 Network simulation: realistic environment testing withoutthe pain 186vwww.allitebooks.com

viPART 4BRIEF CONTENTSDOCKER IN PRODUCTION .2139 Container orchestration: managing multiple Dockercontainers 21510 Docker and security 26211 Plain sailing—Docker in production and operationalconsiderations 29112 Docker in production—dealing with challengeswww.allitebooks.com308

contentsforeword xvpreface xviiacknowledgments xixabout this book xxabout the cover illustrationxxiiiPART 1 DOCKER FUNDAMENTALS .11Discovering Docker1.1The what and why of DockerWhat is Docker?concepts 81.235 5What is Docker good for?Building a Docker application7 Key10Ways to create a new Docker image 11 Writing a Dockerfile 12Building a Docker image 13 Running a Docker container 14Docker layering 16 1.32Summary18Understanding Docker—inside the engine room2.1Docker’s architecture20viiwww.allitebooks.com19

viiiCONTENTS2.2The Docker daemonTECHNIQUETECHNIQUETECHNIQUE2.32.64567Use socat to monitor Docker API traffic 27Using ports to connect to containers 29Linking containers for port isolation 31Using Docker in your browser 33Docker registriesTECHNIQUE2.5Open your Docker daemon to the world 22Running containers as daemons 23Moving Docker to a different partition 26The Docker client ting up a local Docker registryThe Docker HubTECHNIQUE9Summary39343536Finding and running a Docker image37PART 2 DOCKER AND DEVELOPMENT .413Using Docker as a lightweight virtual machine 433.1From VM to container mmary6455The “save game” approach to development 55Docker tagging 57Sharing images on the Docker Hub 59Referring to a specific image in builds 61Environments as processesTECHNIQUE52Managing the startup of your container’sservices 53Saving and restoring your ing your VM to a container 44A host-like container 47Splitting a system into microservice containersManaging services on your containersTECHNIQUE3.310111262The “save game” approach to development 62Day-to-day Docker 654.1Volumes—a persistent problemTECHNIQUETECHNIQUE192066Docker volumes—problems of persistence 66Distributed volumes with BitTorrent Sync 67www.allitebooks.com49

NIQUE4.24.552627282930313287Injecting files into your image using ADDRebuilding without the cache 90Busting the cache 92Staying unning GUIs within Docker 79Inspecting containers 81Cleanly killing containers 83Using Docker Machine to provision Dockerhosts 84Building imagesTECHNIQUETECHNIQUETECHNIQUE4.4Retain your container’s bash history 69Data containers 71Remote volume mounting using sshfs 74Sharing data over NFS 76Dev tools container 78Running 12223242593Running Docker without sudo 93Housekeeping containers 94Housekeeping volumes 95Detaching containers without stopping them 97Using DockerUI to manage your Dockerdaemon 98Generate a dependency graph of your Dockerimages 99Direct action—execute commands on yourcontainer 101Configuration management—getting your house in order5.15.2Configuration management and ETECHNIQUETECHNIQUE4243444546103104Create reliable bespoke tools withENTRYPOINT 104Avoid package drift by specifying versions in yourbuild 106Replacing text with perl -p -i -e 107Flattening images 109Managing foreign packages with alien 111Reverse-engineer a Dockerfile from animage 113Traditional configuration management tools withDocker 116TECHNIQUE88Traditional: using make with Docker 116www.allitebooks.com

xCONTENTSTECHNIQUETECHNIQUE5.35.44748Building images with Chef Solo 118Source-to-image builds 122Small is IQUETECHNIQUE495051525354Summary141128Dockerfile tricks for keeping your build small 128Tricks for making an image smaller 131Tiny Docker images with BusyBox and Alpine 133The Go model of minimal containers 134Using inotifywait to slim containers 137Big can be beautiful 139PART 3 DOCKER AND DEVOPS .1436Continuous integration: speeding up your developmentpipeline 1456.1Docker Hub automated 61Summary168146149Speed up I/O-intensive builds with eatmydata 149Set up a package cache for faster builds 151Running Selenium tests inside Docker 154Containerizing your CI processTECHNIQUE146Using the Docker Hub workflowMore efficient g a complex developmentenvironment 158Running the Jenkins master within a Dockercontainer 162Scale your CI with Jenkins’ Swarm plugin 164Continuous delivery: a perfect fit for Docker principles 1697.1Interacting with other teams during the CD pipelineTECHNIQUE7.27.362Facilitating deployment of Docker ly mirroring registry images 173Delivering images over constrainedconnections 174Sharing Docker objects as TAR files 176Configuring your images for environmentsTECHNIQUE170The Docker contract—reducing friction177Informing your containers with etcd 178170

xiCONTENTS7.47.58Upgrading running containersTECHNIQUE67Summary185180Using confd to enable zero-downtimeswitchover 181Network simulation: realistic environment testing without thepain 1868.1Container communication—beyond manual linkingTECHNIQUETECHNIQUETECHNIQUE8.28.38.4686970A simple Docker Compose cluster 187A SQLite server using Docker Compose 190Finding containers via DNS with Resolvable 194Using Docker to simulate real-world imulating troublesome networks withComcast 197Simulating troublesome networks withBlockade 200Docker and virtual networksTECHNIQUETECHNIQUE187204Setting up a substrate network with Weave 204Docker networking and service features 208PART 4 DOCKER IN PRODUCTION .2139Container orchestration: managing multiple Dockercontainers 2159.1Simple single-host DockerTECHNIQUETECHNIQUE9.29.4Managing your host's containers with systemdOrchestrating your host's containers withsystemd 220Multi-host E9.375767778798081217223Manual multi-host Docker with Helios 223A seamless Docker cluster with Swarm 229Using a Kubernetes cluster 234Building a framework on Mesos 239Micromanaging Mesos with Marathon 247Service discovery: what have we here?TECHNIQUETECHNIQUE8283Summary261217250Using Consul to discover services 251Automatic service registration with Registrator 259

xiiCONTENTS10Docker and security 26210.1Docker access and what it meansDo you care?10.2Security measures in 64Constraining capabilities 264HTTP auth on your Docker instanceSecuring your Docker API 271268Security from outside Docker 275TECHNIQUE87TECHNIQUE88Summary290OpenShift—an application platform as aservice 275Using security options 284Plain sailing—Docker in production and operationalconsiderations 29111.111.211.311.412262263Monitoring 292TECHNIQUETECHNIQUE8990TECHNIQUE91Logging your containers to the host’s syslog 292Sending Docker logs to your host’s outputsystem 295Monitoring containers with cAdvisor 297Resource ting the cores a container canexecute on 298Giving important containers more CPU 299Limiting the memory usage of a container 300Sysadmin use cases for Docker 302TECHNIQUETECHNIQUE9596Summary307Using Docker to run cron jobs 302The “save game” approach to backups305Docker in production—dealing with challenges 30812.1Performance—you can’t ignore the tinTECHNIQUETECHNIQUE12.29798308Accessing host resources from the container 309Device Mapper storage driver and default containersize 313When containers leak—debugging Docker 315TECHNIQUE99Debugging a container’s network withnsenter 315

xiiiCONTENTSTECHNIQUE 100TECHNIQUE 10112.3appendix Aappendix Bappendix CSummaryUsing tcpflow to debug in flight withoutreconfiguring 318Debugging containers that fail on specifichosts 319323Installing and using DockerDocker configuration 329Vagrant 332index 335325

forewordI might be biased, but Docker is a pretty big deal.It wasn’t long ago that applications were large and monolithic, sitting alone insidelumps of steel and silicon. They would stew away for a few years, resisting change, notwanting to move. This was a problem for organizations that wanted to move fast, so it’sno surprise that virtual machines caught on. Applications were no longer tied to thesepieces of hardware, allowing everything to move more quickly and be more flexible.Unfortunately, virtual machines are very complicated. They simulate an entirecomputer inside of another computer, and this virtual computer is still very complexand needs managing. And because virtual machines are smaller and easier to create,there are far more of them around that need managing.How do we manage all of that complexity? With configuration management, ofcourse—another extremely complex system for managing complexity.Docker takes a different approach. If you put your software inside a container, itseparates the complexity of your application from the infrastructure underneath,making the infrastructure simpler and the application easier to ship around. On topof this organizational efficiency, the leap in technical speed and efficiency comparedto virtual machines is dramatic. Containers boot in milliseconds, not minutes. Memory is shared, not allocated. This makes your application much cheaper to run, butalso means that you can architect your application in the way that you want to, not inthe way that fits the constraints of slow, inflexible infrastructure.xv

xviFOREWORDWhen I first saw Solomon Hykes, creator of Docker, talking about Docker and itsanalogy to the shipping container, I knew he was on to something big. The complexstate of the worldwide shipping industry before standardization is an apt analogy for thecomplex state of managing software before containers. Solomon’s insight was so convincing that I started a company building tools around Docker, which was eventuallyacquired by Docker, Inc. and turned into what we now know as Docker Compose.I first met Ian at some of the Docker meetups we organized in London. Back then,we insistently said, “Docker is not ready for production; please don’t use it!” but Ianwas the sort of person who would disregard this sensible advice and go ahead and runit in production anyway. Back then, he was working for the betting services companyOpenBet alongside Aidan, and the amount of money they must have been processingwith the code we had at that time makes me feel a bit light-headed.Ian and Aidan both saw that the value they got from using Docker outweighed theinconveniences of working with it in its beta state. They jumped on the technologyearly, and therefore have a unique perspective on how best to apply it. The toolingthey built at OpenBet has pointed out stuff that was missing in Docker, and our informal chats have had a real influence on the design and direction we’ve taken it.Docker has moved along quite a bit since Ian and Aidan first started using it, andthousands of organizations are now using it to solve real problems: shipping softwarefaster, managing its daunting complexity, improving the efficiency of infrastructure,fixing “works on my machine” problems, and so on. This is causing a huge shift in howwe build, deploy, and manage software, and a whole new landscape of tools and ideasis forming around it. The bright future of containerization is exciting, but is alsointimidatingly different from what we are used to.For you, it might be hard to see how to get from here to there, but this book contains a deluge of practical advice about how to apply Docker to problems you’re having right now. Follow this advice, and your organization will keep on moving quickly.And—perhaps more importantly—building and deploying your applications willbecome a lot more enjoyable.BEN FIRSHMANDIRECTOR OF PRODUCT MANAGEMENT, DOCKER, INC.COCREATOR OF DOCKER COMPOSE

prefaceIn September 2013, while browsing Hacker News, I stumbled across an article in Wiredabout a new technology called “Docker.” 1 As I read it, I became increasingly excited asI realized Docker’s revolutionary potential.The company I’d worked at for over a decade was struggling to deliver softwarequickly enough. Provisioning environments was a costly, time-consuming, manual,and inelegant affair. Continuous integration was barely existent, and setting up development environments was an exercise in patience. As my job title included the words“DevOps Manager,” I was peculiarly motivated to solve these problems!I recruited a couple of motivated coworkers (one of them now my coauthor) via acompany mailing list, and together our skunkworks team labored to turn a beta toolinto a business advantage, reducing the high costs of VMs and enabling new ways ofthinking about building and deploying software. We even built and open sourced anautomation tool (ShutIt) to suit our organization’s delivery needs.Docker gave us a packaged and maintained tool that solved many problems thatwould have been effectively insuperable had we taken it upon ourselves to solve them.This was open source at its best, empowering us to take on a challenge using our sparetime, overcoming technical debt, and learning lessons daily. Lessons not only aboutDocker, but about continuous integration, continuous delivery, packaging, automation, and how people respond to speedy and disruptive technological change.1http://www.wired.com/2013/09/docker/xvii

xviiiPREFACEFor us, Docker is a remarkably broad tool. Wherever you run software using Linux,Docker can impact it. This makes writing a book on the subject challenging, becausethe landscape is as broad as software itself. The task is made more onerous by theextraordinary rate at which the Docker ecosystem is producing solutions to meet theneeds that emerge from such a fundamental change in software production. Overtime, the shape of problems and solutions became familiar to us, and in this bookwe’ve endeavored to pass on this experience. This will enable you to figure out solutions to your particular technical and business constraints.When giving talks at meetups, we’re struck by how quickly Docker has becomeeffective within organizations willing to embrace it. This book mirrors how we usedDocker, going from our desktops, through the DevOps pipeline, and all the way toproduction. As a consequence, this book is sometimes unorthodox, but as engineerswe believe that purity must sometimes give way to practicality, especially when it comesto saving money! Everything in this book is based on real lessons from the field, andwe hope you benefit from our hard-won experience.IAN MIELL

acknowledgmentsThis book couldn’t have been written without the support, sacrifice, and patience ofthose closest to us. Special mention is due to Stephen Hazleton, whose tireless effortswith us to make Docker useful for our customers informed much of the book’s contents.Several Docker contributors and staff were kind enough to review the book at different stages and provided much useful feedback, including the following people whoread the book in manuscript form: Benoit Benedetti, Burkhard Nestmann, ChadDavis, David Moravec, Ernesto Cárdenas Cangahuala, Fernando Rodrigues, José SanLeandro, Kirk Brattkus, Pethuru Raj, Scott Bates, Steven Lembark, Stuart Woodward,Ticean Bennett, Valmiky Arquissandas, and Wil Moore III.Finally, this book also owes a great deal to the Manning editorial team, who wentout of their way to push us into making the book not just good enough, but the best itcould be. We hope the pride they took in their work rubbed off on us.Ian Miell To Sarah, Isaac, and Rachel for putting up with the late-night coding, afather glued to a laptop screen, and the eternal “Docker this, Docker that, Dockerblah, blah,” and to my parents for encouraging me from an early age to question thestatus quo. And buying me that Spectrum.Aidan Hobson Sayers To Mona for the support and encouragement, my parents fortheir wisdom and motivating words, and my coauthor for that fateful “Has anyonetried this Docker thing?” e-mail.xix

about this bookDocker is arguably the fastest-growing software project ever. Open sourced in March 2013,by 2016 it had gained nearly 30,000 GitHub stars and over 7,500 forks. It has accepted significant numbers of pull requests from the likes of Red Hat, IBM, Microsoft, Google, Ciscoand VMware.Docker has hit this critical mass by responding to a critical need for many softwareorganizations: the ability to build software in an open and flexible way and thendeploy it reliably and consistently in different contexts. You don’t need to learn a newprogramming language, buy expensive hardware, or do much in the way of installation or configuration to build, ship, and run applications portably.Docker in Practice takes you through real-world examples of Docker usage usingtechniques we’ve employed in various contexts. Where possible, we’ve tried to elucidate these techniques without requiring kn

viii CONTENTS 2.2 The Docker daemon 21 TECHNIQUE 1 Open your Docker daemon to the world 22 TECHNIQUE 2 Running containers as daemons 23 TECHNIQUE 3 Moving Docker to a different partition 26 2.3 The Docker client 27 TECHNIQUE 4 Use socat to monitor Docker API traffic 27 TECHNIQUE 5 Using ports to connect to containers 29 TECHNIQUE 6 Linking containers for port isolation 31

Related Documents:

Docker Quickstart Terminal Docker Quickstart Terminal Docker . 2. docker run hello-world 3. . Windows Docker : Windows 7 64 . Windows Linux . 1.12.0 Docker Windows Hyper-V Linux 1.12 VM . docker . 1. Docker for Windows 2. . 3. . 1.11.2 1.11 Linux VM Docker, VirtualBox Linux Docker Toolbox .

Exercise: How to use Docker States of a Docker application: – Dockerfile Configuration to create a Docker Image. – Docker Image Image can be loaded by Docker and is used to create Docker Container. – Docker Container Instance of a Docker Image. Dockerfile – Build a Docker Image from Dockerfile wi

Docker images and lauch Docker containers. Docker engine has two different editions: the community edition (Docker CE) and the enterprise edition (Docker EE). Docker node/host is a physical or virtual computer on which the Docker engine is enabled. Docker swarm cluster is a group of connected Docker nodes.

3.Install the Docker client and daemon: yum install docker-engine. 4.Start the Docker daemon: service docker start 5.Make sure the Docker daemon will be restarted on reboot: chkconfig docker on 6. Add the users who will use Docker to the docker group: usermod -a -G docker user .

o The Docker client and daemon communicate using a RESTAPI, over UNIX sockets or a network interface. Docker Daemon(dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. Docker Client(docker) is the primary way that many Docker users interact with Docker. When docker run

Introduction to Containers and Docker 11 docker pull user/image:tag docker run image:tag command docker run -it image:tag bash docker run image:tag mpiexec -n 2 docker images docker build -t user/image:tag . docker login docker push user/image:tag

What is Docker? 5 What is Docker good for? 7 Key concepts 8 1.2 Building a Docker application 10 Ways to create a new Docker image 11 Writing a Dockerfile 12 Building a Docker image 13 Running a Docker container 14 Docker layering 16 1.3 Summary 18 2 Understanding Docker—inside the engine room 19 2.1 architecture 20 www.allitebooks.com

Open docker-step-by-step.pdf document Introduction to Containers and Docker 19. Backup slides. Docker cheatsheet Introduction to Containers and Docker 21 docker pull user/image:tag docker run image:tag command docker run -it image:tag bash docker run image:tag mpirun -n 2