Introduction To Containers And Docker - Hps.vi4io
Introduction to containers and Docker Summer School on Effective HPC for Climate and Weather Alberto Madonna, CSCS August 26, 2021
Table of Contents 1. Introduction to containers and Docker (20 min) 2. Docker tutorial / live demo (50 min) 3. Overview of containers in HPC (15 min) - Break - 4. Lab session (1h30m) Slides and code available at https://github.com/eth-cscs/containers-hands-on Lab tutorial video: https://youtu.be/dv74sFb3cVc Disclaimer: This material reflects only the author’s view and the EU-Commission is not responsible for any use that may be made of the information it contains Introduction to Containers and Docker 2
Containers Isolated environments to run applications/services Images include all software dependencies Prescriptive, portable, easy to build, quick to deploy Image credit: Docker Inc. Introduction to Containers and Docker 3
Key terms Image: standalone, executable package that includes everything needed to run a piece of software (code, runtime libraries, configuration files). Provides the filesystem and metadata (e.g. environment variables, initial working directory) for a container. Container: a process isolated from the rest of the system through abstractions created by the OS. The level of isolation can be controlled, allowing access to host resources. Its filesystem content comes from an image. Can be thought as the runtime instance of an image: what the image becomes in memory when actually executed. Introduction to Containers and Docker 4
Linux containers ecosystem Linux containers rely on abstraction features (namespaces1) provided by the kernel Different design decisions and use cases gave rise to several solutions: HPC focused LXC Singularity 1 “Namespaces in operation, part 1: namespaces overview” at https://lwn.net/Articles/531114/ Introduction to Containers and Docker 5
Docker Extremely popular container implementation Easy to use authoring tools Container images are created from recipe-like files Images can be named, tagged and built on top of other images Cloud-based image distribution strategy Several remote registries available (e.g. Docker Hub) Client includes facilities to authenticate, push and pull images Introduction to Containers and Docker 6
Docker workflow 1. An image is created locally from a Dockerfile 2. Push (i.e. upload) the image to a remote registry DockerHub is the public registry maintained by the Docker company 3. Pull (i.e. download) the image on a target machine and run the container Introduction to Containers and Docker 7
So how are containers useful? Containers give the possibility to create (scientific) applications that are: 1. Portable 2. Prescriptive 3. Easy to deploy 4. Easy to test Introduction to Containers and Docker 8
Live demo!
Using NVIDIA GPUs in Docker GPU-accelerated application GPU driver Included/built in the image, along with its runtime dependencies It is tied to the hardware: cannot be part of a portable image! NVIDIA provides base images for CUDA, featuring compilers, runtime and accelerated libraries: https://hub.docker.com/r/nvidia/cuda Has to be imported upon container creation NVIDIA Container Toolkit to the rescue! https://github.com/NVIDIA/nvidia-docker Quickest way to get a Dockerfile going: FROM nvidia/cuda Introduction to Containers and Docker Docker 19.03 has native support: docker run --gpus all nvidia/cuda nvidia-smi 10
Containers in HPC
Docker and HPC: not a good fit Security model assumes root privileges No integration with workload managers Missing support for diskless nodes Very limited support for kernel bypassing devices (e.g. accelerators and NICs) No adequate parallel storage driver Introduction to Containers and Docker 12
HPC-focused container software Enabling execution of containers complying with the peculiarities of HPC environments Singularity https://sylabs.io/singularity/ https://products.cscs.ch/sarus/ https://hpc.github.io/charliecloud/ https://github.com/NERSC/shifter Introduction to Containers and Docker 13
Sarus container engine Combines container portability with native HPC performance Integrates with HPC infrastructure and software Customizes containers at runtime with standard plugins Pulls regular Docker images Provides a Docker-like command line interface Introduction to Containers and Docker 14
Further material & contact Slides and Lab material: https://github.com/eth-cscs/containers-hands-on Lab intro video: https://youtu.be/dv74sFb3cVc Official Docker documentation: https://docs.docker.com/ Best practices for writing Dockerfiles: kerfile best-practices/ Contact: alberto.madonna@cscs.ch Introduction to Containers and Docker 15
The ESiWACE1/2 projects have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 675191 and No 823988 Disclaimer: This material reflects only the author’s view and the EU-Commission is not responsible for any use that may be made of the information it contains Introduction to Containers and Docker 16
Thank you for your attention.
Break time! Back at 15.30 CEST
Lab session Slides and material: https://github.com/eth-cscs/containers-hands-on Intro video: https://youtu.be/dv74sFb3cVc Quickstart: 1. git clone t 2. cd containers-hands-on/esiwace2-summer-school-2021 3. Open docker-step-by-step.pdf document Introduction to Containers and Docker 19
Backup slides
Docker cheatsheet docker pull user/image:tag docker run image:tag command docker run –it image:tag bash docker run image:tag mpirun –n 2 docker images docker build –t user/image:tag . docker login docker push user/image:tag Introduction to Containers and Docker 21
Open docker-step-by-step.pdf document Introduction to Containers and Docker 19. Backup slides. Docker cheatsheet Introduction to Containers and Docker 21 docker pull user/image:tag docker run image:tag command docker run -it image:tag bash docker run image:tag mpirun -n 2
Introduction to Containers and Docker 11 docker pull user/image:tag docker run image:tag command docker run -it image:tag bash docker run image:tag mpiexec -n 2 docker images docker build -t user/image:tag . docker login docker push user/image:tag
Docker Quickstart Terminal Docker Quickstart Terminal Docker . 2. docker run hello-world 3. . Windows Docker : Windows 7 64 . Windows Linux . 1.12.0 Docker Windows Hyper-V Linux 1.12 VM . docker . 1. Docker for Windows 2. . 3. . 1.11.2 1.11 Linux VM Docker, VirtualBox Linux Docker Toolbox .
o The Docker client and daemon communicate using a RESTAPI, over UNIX sockets or a network interface. Docker Daemon(dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. Docker Client(docker) is the primary way that many Docker users interact with Docker. When docker run
Docker images and lauch Docker containers. Docker engine has two different editions: the community edition (Docker CE) and the enterprise edition (Docker EE). Docker node/host is a physical or virtual computer on which the Docker engine is enabled. Docker swarm cluster is a group of connected Docker nodes.
Exercise: How to use Docker States of a Docker application: – Dockerfile Configuration to create a Docker Image. – Docker Image Image can be loaded by Docker and is used to create Docker Container. – Docker Container Instance of a Docker Image. Dockerfile – Build a Docker Image from Dockerfile wi
3.Install the Docker client and daemon: yum install docker-engine. 4.Start the Docker daemon: service docker start 5.Make sure the Docker daemon will be restarted on reboot: chkconfig docker on 6. Add the users who will use Docker to the docker group: usermod -a -G docker user .
What is Docker? 5 What is Docker good for? 7 Key concepts 8 1.2 Building a Docker application 10 Ways to create a new Docker image 11 Writing a Dockerfile 12 Building a Docker image 13 Running a Docker container 14 Docker layering 16 1.3 Summary 18 2 Understanding Docker—inside the engine room 19 2.1 architecture 20 www.allitebooks.com
American Revolution, students are exposed to academic, domain-specific vocabulary and the names and brief descriptions of key events. Lesson 2 is a simulation in which the “Royal Tax Commissioners” stamp all papers written by students and force them to pay a “tax” or imprisonment.
