DevOps #1: Docker for Embedded Engineers

Docker can be a powerful tool for embedded engineers. Here’s a breakdown of how it might be useful for you:

What is Docker?

Docker is a platform for developing, shipping, and running applications in containers. Containers are lightweight, standalone packages that include everything needed to run an application—code, runtime, libraries, and dependencies.

A Docker image is a read-only template that contains the filesystem and configuration needed to run an application. A Docker container is a running instance of a Docker image.

Docker Analogy:  

• Docker Image: Like the recipe for a cake.
• Docker Container: Like the actual cake that you bake using the recipe.

Benefits for Embedded Engineers

1. Consistent Development Environment: Docker helps ensure that all team members are working in the same environment. This consistency can eliminate issues caused by differences in local setups.

2. Isolation: Containers isolate your development environment from the host system. This can be particularly useful if you need to manage different versions of tools or dependencies for various projects.

3. Reproducibility: Docker images can be versioned and shared, making it easier to reproduce specific development environments. This can be beneficial for testing and debugging embedded systems where hardware and software environments are complex.

4. Cross-Platform Compatibility: Docker allows you to build and test software in a controlled environment that can be replicated across different operating systems and hardware setups.

5. Automated Builds and Testing: You can set up automated workflows using Docker to build and test your software in a consistent environment. This can help catch issues early and streamline the CI/CD process.

Typical Usage in Embedded Engineering

1. Toolchain Management: Embedded projects often require specific versions of toolchains and libraries. You can create Docker containers that include the exact toolchain and libraries you need, ensuring compatibility and easing setup for new team members.

2. Simulation and Emulation: For embedded systems that require specific hardware, Docker can simulate or emulate the environment needed for development and testing. This allows you to test software in an environment similar to the target hardware.

3. Dependency Management: Use Docker to manage dependencies and configurations required for building and running your embedded applications. This avoids conflicts and simplifies the setup process.

4. Build and Test Automation: Integrate Docker into your build and test pipelines to automate the process of building your software, running tests, and verifying that everything works correctly in a consistent environment.

Basic Workflow

1. Create a Dockerfile: Define a Dockerfile that describes the environment needed for your project. This includes the base image, dependencies, and any setup commands.

2. Build the Docker Image: Use the Dockerfile to build an image that contains your development environment.

3. Run Containers: Start containers from your image to develop, build, and test your software. You can run multiple containers if needed to simulate different environments or scenarios.

4. Share and Collaborate: Push your Docker images to a container registry, allowing team members to pull and use the same environment.

Can we run so many containers from a single image? 

Yes, exactly! You can run multiple containers from a single Docker image. Each container is an independent instance of that image, running in its own isolated environment. For Example:

docker run -d --name container1 my-image
docker run -d --name container2 my-image

Here, container1 and container2 are two separate containers running from the my-image image.

Creating a Docker Image/Container

Creating Docker images and containers involves a few key steps. Here's a step-by-step guide on how to do it:

To create a Docker image, you need to define the image's configuration and setup using a Dockerfile. The Dockerfile is a text file that contains instructions for building the image.

Example Dockerfile

Here’s a simple example of a Dockerfile for a basic Python application:

# Use an official Python runtime as a parent image
FROM python:3.9-slim
# Set the working directory in the container
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
# Install any needed packages specified in requirements.txt, By using RUN we can run as many commands as want.
RUN pip install --no-cache-dir -r requirements.txt
# Make port 80 available to the world outside this container
EXPOSE 80
# Define environment variable
ENV NAME World
# Run app.py when the container launches
CMD ["python", "app.py"] or ENTRYPOINT ["python", "app.py"]

Building  Example Image

 

docker build -t my-python-app .

-t my-python-app tags the image with the name my-python-app.

. indicates the build context, which is the current directory.

Now it's time to create and run the containers

Use the docker run command to create and start (run command is equivalent to create and start commands) a container from the image:

docker run -d -p 4000:80 --name my-running-app my-python-app

-d runs the container in detached mode (in the background).

-p 4000:80 maps port 80 in the container to port 4000 on your host machine.

--name my-running-app assigns a name to the container.

my-python-app specifies the image to use.

You can also add --rm to remove the container once it finishes it's Job.

If your application exposes a web server, you can access it in a web browser by navigating to http://localhost:4000.

Run bash on an existing image (--entrypoint will override the ENTRYPOINT in the Dockerfile)

sudo docker run -it --entrypoint="/bin/bash" existing-image-name

where i indicated interactive mode, t indicates pseudo-tty terminal.

Restart an existing container

sudo docker start -i container-name

To stop a running container:

docker stop my-running-app

To remove the stopped container:

docker rm my-running-app

To remove all the containers at once:


sudo docker container rm $(sudo docker container ls -aq)


To see logs of a container:

sudo docker logs container-name

To list all the container:

sudo docker container ls -a

To list all the running container:

sudo docker ps 

To list all the created images:

sudo docker images 

To clean out all the dangling images(created due to failed build):

sudo docker image prune -f

To clean out all the images having no containers:

sudo docker image prune -a

Way to search for available images on hub.docker.com having stm32 in it's name:

sudo docker search stm32
 

NOTE: Docker containers are immutable, meaning their underlying image cannot be changed once the container is running. Stop the container and restart with new image is also not possible. You have create a new container with the updated image.

Workflow Summary 

• Modify Code: Update your source code files. 
• Rebuild Image: Rebuild the Docker image to include your changes. 
• Run Container: Run a new container from the updated image to test your changes.

Docker Registry

Docker Registry is a system for storing and distributing Docker images. It serves as a central repository where you can push Docker images for sharing with others or pull Docker images for use in your environment. Registries enable collaboration and efficient management of Docker images across different environments.

Key Concepts of Docker Registry

1. Docker Image: A Docker image is a read-only template that contains the application code, dependencies, libraries, and configurations needed to create a container. Images are built from Dockerfiles and can be versioned using tags.

2. Docker Repository: A repository is a collection of related Docker images, typically representing different versions of the same application. For example, a repository called my-app might have tags like v1.0, v2.0, and latest.

3. Docker Registry: A Docker registry is a service that stores repositories. It allows you to push images to it and pull images from it. The registry can be public or private. 

How Docker Registry Works

1. Pulling an Image from a Registry: When you want to run a Docker container, you typically pull an image from a registry using the docker pull command. For example, to pull the nginx image from Docker Hub:

 

docker pull nginx:latest

 

Docker checks locally if the image is available; if not, it pulls it from the registry (e.g., Docker Hub).

2. Pushing an Image to a Registry: After building an image locally, you can push it to a registry to make it available to others or for later use:

 

docker push myregistry.com/my-app:latest

 

This command uploads the image to the specified registry under the specified repository and tag.

3. Tagging Images: To push an image to a registry, you need to tag it with the registry’s address and repository name:

 

docker tag my-app:latest myregistry.com/my-app:latest

 

Tagging is crucial for organizing and versioning images.

Types of Docker Registries

1. Docker Hub: The default and most widely used public Docker registry. It hosts millions of public images and also allows private repositories for a fee.

   docker pull myrepo/myimage:tag

2. Private Docker Registries: Organizations often run their own private registries to securely store and manage Docker images. This can be done using the open-source Docker Registry software or a managed service like:

   • Amazon Elastic Container Registry (ECR)
   • Google Container Registry (GCR)
   • Azure Container Registry (ACR)

   docker pull myregistry.com/myrepo/myimage:tag

3. Self-Hosted Registry: You can deploy and manage your own Docker Registry using Docker itself:

   docker run -d -p 5000:5000 --name registry registry:2

   This command runs a private registry on localhost:5000.

 Workflow Using Docker Registry

1. Build an Image
2. Tag the Image
3. Push the Image
4. Pull the Image on the same machine or on the other machine
5. Run the Image