The Docker Building Process

I am finishing one new project and before sharing it I want to make sure that everyone can use it without using an IDE installed locally. For that, the best solution is to create a docker container of the app and host it to make it public.

When it comes to building Docker images, we are often faced with the choice between leveraging cloud-based solutions like GitHub Actions or sticking with traditional local builds.

Each approach offers its own set of benefits and drawbacks, making it essential to understand their distinctions in order to determine which method best aligns with your project requirements.

In this blog post, we’ll dive into a comprehensive comparison of GitHub Actions and local builds for creating Docker images, examining factors such as automation, resource usage, build times, and more.

Building Docker Images in the Cloud

Github Actions: x86

By using GitHub Actions, you can create workflows that automatically build, test, and deploy your Docker images whenever changes are pushed to your repository. This automation not only saves time and effort but also helps maintain a consistent deployment process.

  • To setup Github Actions there are some details to consider, among them:
    • You will need: a CI/CD configuration, Dockerfile and Requirements file

Github Runner: x86, ARM32, ARM64…

To create ARM images using Github CI/CD, you will need Github Runner as Github let us use X86 machines only for our builds.

I have included this point under the cloud… but actually the cloud now can be your Raspberry Pi (or other ARM device), maybe GCP, or Oracle cloud (ARM too)… the point is that you will user other machine than the one in which you created the code to make the heavy lifting and make the image building process.

  • Download/Clone the repository and navigate to the new folder:
git clone https://github.com/JAlcocerT/Py_Trip_Planner.git ./Trip_Planner &&
cd Trip_Planner

Requirement File - Python DASH

The requirements.txt will be common for all the architectures.

In the particular case of the Trip Planner App that I am building, I need to include also few more packages.

  • Make sure to specify the version for which the development was successful.
dash==2.7.0 

dash_leaflet==0.1.23

pandas==1.2.4
plotly==5.11.0

meteostat==1.6.5

Dockerfile x86/ARM32/ARM64 - Python DASH

The good news is that the Dockerfile for a Python DASH App is compatible across all these platforms:

#https://hub.docker.com/_/python
FROM python:3.8

# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./

# Install production dependencies.
RUN pip install -r requirements.txt

#We will use it later while running the container
EXPOSE 8050

CMD python ./app/app.py

Building Images Locally: x86, ARM32, ARM64…

As we were already pointing on the docker guide post, in general, with this simple CLI commands you will be able to build and run any container.

Basically, you will need to have the App files, Dockerfile and Requirements File ready and then:

  • Build the Docker Image:

I Will name it as trip_planner, but you can customize it:

#docker build -t trip_planner .
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t trip_planner .

This process will take a while. Depending on the device you are using, you can expect different build times.

  • In my case, for the Trip Planner:

    • A Raspberry Pi 4B (2GB): ~113 min
    • Github Actions CI/CD (x86): ~5 min
    • Laptop i7-1185G7 (32GB): ~5 min

  • Run the Image: The container part will be fixed at 8050, but on the local machine side you can choose any free port.

With CLI:

docker run --name Trip_Planner -p 8057:8050 --detach trip_planner

Or with yml:

---
version: "2"
services:
  tripplanner:
    image: trip_planner #fossengineer/trip_planner:arm64
    container_name: py_trip_planner
    ports:
      - 8057:8050
    networks: #optional
      - cloudflare_tunnel  #optional
    restart: unless-stopped

networks: #optional
  cloudflare_tunnel: #optional
    external: true    #optional

  • You can apply the same logic for building Docker images locally, this will:

    • Involve manually running Docker commands on your machine
    • At the same time, Local builds give you more control over the process, but require additional effort to set up and maintain

  • Another key difference between GitHub Actions and local builds is the resources used during the build process.

    • With GitHub Actions, the build process runs on GitHub’s servers (Actually Microsoft Cloud - Azure), saving your local machine’s resources and allowing you to continue working without performance impact. However, you may encounter limitations on compute resources, depending on your GH plan.
    • Local builds utilize your machine’s resources, which can lead to longer build times and reduced performance if your machine isn’t powerful enough.

Useful CLI Commands for Building Images Locally

docker history my-image

docker build --no-cache --progress=plain -t my-image .

Use all cores and show me the logs - This is the one I use the most:

DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t my-image .

DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain --cpuset-cpus 0,1,2 -t my-image .
#DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain --cpuset-cpus 0-$(($(nproc) - 1)) -t my-image .

Sharing Local Docker Images - DockerHub

In a previous post I was giving a general overview to the Docker push process.

Let’s make it particular for DockerHub and discover how to share with other people the images that we have built locally.

  • You will need an account and login:
docker login

A success message will follow.

  • Then create a tag for the image:
docker tag the_image_name_you_built docker.io/your_username/your_image_name_as_it_will_be_pushed:latest

For our particular example:

docker tag trip_planner docker.io/your_username/trip_planner:latest
  • And finally push it to DockerHub container registry:
docker push docker.io/your_username/your_image_name_as_it_will_be_pushed:latest
#docker push docker.io/your_username/trip_planner:latest

Managing Multi-Arch Images in Dockerhub

What if we want to have different architectures under the same tag in Dockerhub? I wanted the image of this project to be at least working perfectly for: x86, ARM32 and ARM86.

Fortunately, we can use the docker manifest:

  • First, make sure you have created (and pushed) the docker images with the architectures you are interested, for example:
docker build -t fossengineer/trip_planner:latest-arm64 --build-arg ARCH=arm64 .
docker push fossengineer/trip_planner:latest-arm64

docker build -t fossengineer/trip_planner:latest-arm32 --build-arg ARCH=arm32 .
docker push fossengineer/trip_planner:latest-arm32

docker build -t fossengineer/trip_planner:latest-amd64 --build-arg ARCH=amd64  .
docker push fossengineer/trip_planner:latest-amd64
  • Create a manifest list that references the Docker images:
docker manifest create fossengineer/trip_planner:latest \
fossengineer/trip_planner:latest-arm32 fossengineer/trip_planner:latest-arm64 fossengineer/trip_planner:latest-amd64
  • If you need to edit it:
docker manifest create --amend fossengineer/trip_planner:latest \
fossengineer/trip_planner:latest-arm64 fossengineer/trip_planner:latest-amd64 fossengineer/trip_planner:latest-anotherarch
  • Push the manifest list to Docker Hub:
docker manifest push fossengineer/trip_planner:latest

You can check in DockerHub how multi-arch images looks like for the Trip-Planner Python App

Leveraging Local Docker Image Building: Local Free Open Source CI/CD Tools

There are several free and open-source CI/CD tools you can use locally or self-host within your infrastructure. Some popular options include that you can integrate with Gitea (the open source repository that you can self-host) are:

  • Drone: Drone is a container-native CI/CD platform that integrates well with Gitea. It supports a wide range of languages, platforms, and integrations through plugins. To set up the integration, follow the official documentation (https://docs.drone.io/server/provider/gitea/).

  • Jenkins: a popular open-source CI/CD automation server, can be integrated with Gitea using the Gitea plugin (https://plugins.jenkins.io/gitea/). This plugin enables Jenkins to work with Gitea repositories for continuous integration and deployment.

  • GoCD: You can also integrate GoCD with Gitea using a plugin called Gitea PR Builder (https://github.com/ashwanthkumar/gocd-build-github-pull-requests). This plugin allows GoCD to work with Gitea repositories to build and test pull requests.

Another solution to make the local code versioning and image build easier is by using GitLab Community Edition and Gitlab CI/CD.