Run Multi-Step Job inside Docker Container with GitHub Actions

What

This post will explore the possibility to run multiple GitHub Actions steps inside a Docker container that is built in the same workflow.

I will visualize this with a simple example of a PHP project utilizing PHPUnit, PHPStan and PHP-CS-Fixer but this of course works for every language or tooling.

Why

With GitHub Actions you have the possibility to choose from endless available workflows built by the community such as PHP-CS-Fixer or PHPUnit.

The problem with this that - if you're distributing your project as a Docker container (as you should) - your CI environment can be vastly different from your container environment.

You have the option to pin a specific PHP version but nonetheless if you've worked with Docker containers before you will know there are a lot of quirks - such as Alpine images, specific environment variables, multi-step build stages, ...

Why in parallel

Speed.

The idea

TLDR; If you just want to see the full workflow file, scroll down.

We want to firstly build a reusable Docker container and then run any tasks in parallel afterwards.

1. Build Container

In this build step we will build the container and save it as a tarball to a temporary directory /tmp/project.tar. To make it later available to other jobs we will upload it as an artifact named project.

build-image:
  runs-on: ubuntu-latest
  steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Build Container
      uses: docker/build-push-action@v5
      with:
        tags: project-image
        context: .
        push: false
        outputs: type=docker,dest=/tmp/project.tar
        build-args: |
          KEY=value

    - name: Upload artifact
      uses: actions/upload-artifact@v3
      with:
        name: project
        path: /tmp/project.tar

2. use Container in parallel jobs

In any other following job, we attach a needs: [ build-image ] to tell GitHub Actions that we depend on the previous job.

The first step is to download previously uploaded artifact (container) project and load it into the Docker daemon.

phpstan:
  name: PHPStan
  runs-on: ubuntu-latest
  needs: [ build-image ]
  steps:
    - name: Download artifact
      uses: actions/download-artifact@v3
      with:
        name: project
        path: /tmp

    - name: Load image
      run: docker load --input /tmp/project.tar

    - name: Run PHPStan
      uses: addnab/docker-run-action@v3
      with:
        image: project-image
        run: vendor/bin/phpstan analyse

3. Cleanup

Since we made use of artifacts, these files will be persisted after the default cleanup time of 30 days. This will take up storage quota from your GitHub account.

Note to add if: always() to always execute this step - even if previous jobs failed. Also specify all previous build steps in the needs: [] array.

remove-image:
  name: Remove image
  if: always()
  runs-on: ubuntu-latest
  needs: [ build-image, phpstan ]
  steps:
    - uses: geekyeggo/delete-artifact@v2
      with:
        name: project

Notes!

Dependencies

If you're using the same Dockerfile for your production and CI environment, be sure to add a configuration/environment variable which indicates the Docker build to only install production dependencies outside of the CI.

FROM composer:latest AS build-composer

ARG PRODUCTION=true

WORKDIR /app

COPY . /app

RUN if [ "$PRODUCTION" = "true" ]; then \
        composer install --prefer-dist --no-cache --no-scripts --ignore-platform-reqs --no-dev; \
    else \
        composer install --prefer-dist --no-cache --no-scripts --ignore-platform-reqs; \
    fi
RUN composer dump-autoload --optimize

build-image:
  steps:
    - name: Build Container
      uses: docker/build-push-action@v5
      with:
        tags: project-image
        push: false
        outputs: type=docker,dest=/tmp/project.tar
        build-args: |
          PRODUCTION=false

Quota Usage

If your workflows are running for a longer time and GitHub Actions quota is important to you, note that we have observed that running multiple steps in parallel using different jobs counts more to your quota since every job counts against your quota on its own. The following example is not representative of actual execution times since the job logs originate from different points in time but rather visualize the issue.

Comparison using jobs in parallel vs. steps ins sequence

Workflow File

Full GitHub Actions workflow file using PHPUnit (+MySQL), PHPStan and PHP-CS-Fixer.

name: Tests

on: [ push ]

jobs:
  build-image:
    name: Build image
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build Container
        uses: docker/build-push-action@v5
        with:
          tags: project-image
          context: .
          push: false
          outputs: type=docker,dest=/tmp/project.tar
          build-args: |
            KEY=value

      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          name: project
          path: /tmp/project.tar

  phpunit:
    name: PHPUnit
    runs-on: ubuntu-latest
    needs: [ build-image ]
    services:
      database:
        image: mariadb:10.6
        env:
          MYSQL_DATABASE: web
          MYSQL_USER: web
          MYSQL_PASSWORD: web
          MYSQL_ROOT_PASSWORD: root
        ports:
          - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: project
          path: /tmp

      - name: Load image
        run: docker load --input /tmp/project.tar

      - name: Run Tests
        uses: addnab/docker-run-action@v3
        with:
          image: project-image
          options: |
            --add-host=host.docker.internal:host-gateway
            -e APP_KEY=base64:abcdefghijklmnopqrstuvwxyz1234567890
            -e DB_PORT=${{ job.services.database.ports[3306] }}
            -e DB_HOST=host.docker.internal
            -e DB_USERNAME=web
            -e DB_PASSWORD=web
            -e DB_DATABASE=web
          run: |
            php artisan passport:keys
            vendor/bin/phpunit --testdox

  phpstan:
    name: PHPStan
    runs-on: ubuntu-latest
    needs: [ build-image ]
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: project
          path: /tmp

      - name: Load image
        run: docker load --input /tmp/project.tar

      - name: Run PHPStan
        uses: addnab/docker-run-action@v3
        with:
          image: project-image
          run: vendor/bin/phpstan analyse

  php-cs-fixer:
    name: PHP-CS-Fixer
    runs-on: ubuntu-latest
    needs: [ build-image ]
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: project
          path: /tmp

      - name: Load image
        run: docker load --input /tmp/project.tar

      - name: Run PHP-CS-Fixer
        uses: addnab/docker-run-action@v3
        with:
          image: project-image
          run: vendor/bin/php-cs-fixer fix --stop-on-violation --dry-run

  remove-image:
    name: Remove image
    if: always()
    runs-on: ubuntu-latest
    needs: [ build-image, phpunit, phpstan, php-cs-fixer ]
    steps:
      - uses: geekyeggo/delete-artifact@v2
        with:
          name: project