Docker

Docker

Docker is an open platform to build, ship, and run distributed applications. Docker consists of Docker Engine, a lightweight runtime and packaging tool that is made up of:

• A daemon (aka server); a server process that manages all the containers
• Clients act as a remote control for the daemon. Clients talk to the docker server/daemon
• Images are the building blocks of Docker, think of it like the source code for your containers. Images are built by adding a file, running a command, opening a port. Example images include Nginx or Postgresql database.
• Containers are launched from images and can contain one or more running processes. A Docker container is:
  • An image format (think of this as the cargo)
  • A set of standard operations (think of it as what it can do with the cargo, e.g. create, start, stop, restart, destroy)
  • An execution environment (think of it as where to ship your container; can build on laptop, deploy to EC2 instance)

The goal of Docker is to enable apps to be quickly assembled from components and eliminates the friction between different environments (dev, QA, production).

Intro Guide

https://training.play-with-docker.com/beginner-linux/

Containers with microservices

Docker Containers are natural for microservices.

* Simple to Model
* Available in any app, any language
* Image is the version
* Test and deploy from same artifact
* Stateless servers decrease change risk

Scheduling Containers

Scheduling a single resource is easy, just run a single container on laptop. However, scheduling a cluster of containers is difficult. * How do I roll out new versions of my software? * What happens if one of my machines dies?

For this, I recommend looking into something like AWS ECS or Kubernetes.

AWS ECS

Amazon EC2 Container Service (ECS) is a container management service. You can use ECS to schedule the placement of containers across your cluster. You can integrate your own scheduler or a third-party scheduler.

Cluster Management: Resource Management

A resource manager is responsible for keeping track of resources (memory, cpu, storage) on a cluster at any given time. E.g. 3 EC2 instances across 2 availability zones, each running Docker and has tasks running on containers.

Cluster Management: Scheduling

Scheduler is responsible for scheduling tasks for execution. Say X node does Y task based on resources above (memory, cpu, storage). Scheduling requests the resource and confirms request done. Scheduler knows if task is alive or dead (if dead, then reschedule).

AWS handles resource management and scheduling with the Cluster Management Engine. You can segment your cluster based on say dev, test, production.

AWS has an Agent Communication Service that talks with an ECS Agent (installed on each ECS container).

AWS has a Key/Value Store to handle state so that we have concurrency control.

You can access a lot of this through an API.

Install Docker and Docker Compose

Note: I’m using a specific older version of docker and docker-compose

# Install basic packages
sudo apt-get install -y vim curl openssh-server ca-certificates python3.6 git

# Install Docker - keep in mind I'm currently using Xenial (16.04, replace as needed, e.g. for 18.04)
wget https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/docker-ce_18.06.1~ce~3-0~ubuntu_amd64.deb /tmp
sudo dpkg -i /tmp/docker-ce_18.06.1~ce~3-0~ubuntu_amd64.deb

# Install Docker-Compose
sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Docker Ecosystem

• Docker Engine, can check with docker --version # Usually this is what people refer to when they say ‘Docker’
• Compose, can check with docker-compose --version # Help orchestrate multiple containers
• Machine, can check with docker-machine --version # I don’t use machine

Docker General Commands

• docker to get a list of docker commands
• docker info to tell you about the current setup

Docker Images

List out Docker images that you have

docker images

Docker Container

docker container to get a list of specific docker commands for containers. E.g. docker container run alpine hostname to (get if not already available the alpine container) and runs the hostname command docker container logs <mycontainer> to get the logs from that container

Docker Linking Containers

You can create a secure tunnel between containers so that they don’t have to expose any ports externally. E.g. web container linked to a db container, we don’t want to expose the db container

The syntax is to use the --link flag to link a source container (e.g. db) to provide information to a recipient container (e.g. web)

docker run -d -P --name web --link db:db training/webapp python app.py

Design Patterns for container-based distributed systems

https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45406.pdf

Building

Say there’s a Dockerfile, you can build that image and tag it with:

docker build . -t <name>:<tag>

Use -f to target a specific Dockerfile

Docker Mounting Files and Directories

docker --volume  # aka docker -v

docker run -d -it --name devtest -v "$(pwd)"/target:/app:z nginx:latest

docker --mount

Often you’ll want to mount your ~/.aws/ configs, e.g. ~/.aws/:/root/.aws:ro

You might often use this with aws sso login && eval $(aws-sso-creds export) to export AWS credentials to the env

Scaling up

Related Posts

• 11 May 2023 DBT
• 30 Oct 2022 Godot
• 16 Oct 2022 AWS Fargate