Run Multi-Step Job inside Docker Container with GitHub Actions

What

This post will explore the possibility to run multiple GitHub Actions steps inside a Docker container that is built in the same workflow.

I will visualize this with a simple example of a PHP project utilizing PHPUnit, PHPStan and PHP-CS-Fixer but this of course works for every language or tooling.

Why

With GitHub Actions you have the possibility to choose from endless available workflows built by the community such as PHP-CS-Fixer or PHPUnit.

The problem with this that - if you're distributing your project as a Docker container (as you should) - your CI environment can be vastly different from your container environment.

You have the option to pin a specific PHP version but nonetheless if you've worked with Docker containers before you will know there are a lot of quirks - such as Alpine images, specific environment variables, multi-step build stages, ...

Why in parallel

Speed.

The idea

TLDR; If you just want to see the full workflow file, scroll down.

We want to firstly build a reusable Docker container and then run any tasks in parallel afterwards.

1. Build Container

In this build step we will build the container and save it as a tarball to a temporary directory /tmp/project.tar. To make it later available to other jobs we will upload it as an artifact named project.

 1build-image:
 2  runs-on: ubuntu-latest
 3  steps:
 4    - name: Checkout repository
 5      uses: actions/checkout@v4
 6
 7    - name: Set up Docker Buildx
 8      uses: docker/setup-buildx-action@v3
 9
10    - name: Build Container
11      uses: docker/build-push-action@v5
12      with:
13        tags: project-image
14        context: .
15        push: false
16        outputs: type=docker,dest=/tmp/project.tar
17        build-args: |
18          KEY=value
19
20    - name: Upload artifact
21      uses: actions/upload-artifact@v3
22      with:
23        name: project
24        path: /tmp/project.tar

2. use Container in parallel jobs

In any other following job, we attach a needs: [ build-image ] to tell GitHub Actions that we depend on the previous job.

The first step is to download previously uploaded artifact (container) project and load it into the Docker daemon.

 1phpstan:
 2  name: PHPStan
 3  runs-on: ubuntu-latest
 4  needs: [ build-image ]
 5  steps:
 6    - name: Download artifact
 7      uses: actions/download-artifact@v3
 8      with:
 9        name: project
10        path: /tmp
11
12    - name: Load image
13      run: docker load --input /tmp/project.tar
14
15    - name: Run PHPStan
16      uses: addnab/docker-run-action@v3
17      with:
18        image: project-image
19        run: vendor/bin/phpstan analyse

3. Cleanup

Since we made use of artifacts, these files will be persisted after the default cleanup time of 30 days. This will take up storage quota from your GitHub account.

Note to add if: always() to always execute this step - even if previous jobs failed. Also specify all previous build steps in the needs: [] array.

1remove-image:
2  name: Remove image
3  if: always()
4  runs-on: ubuntu-latest
5  needs: [ build-image, phpstan ]
6  steps:
7    - uses: geekyeggo/delete-artifact@v2
8      with:
9        name: project

Notes!

Dependencies

If you're using the same Dockerfile for your production and CI environment, be sure to add a configuration/environment variable which indicates the Docker build to only install production dependencies outside of the CI.

 1FROM composer:latest AS build-composer
 2
 3ARG PRODUCTION=true
 4
 5WORKDIR /app
 6
 7COPY . /app
 8
 9RUN if [ "$PRODUCTION" = "true" ]; then \
10        composer install --prefer-dist --no-cache --no-scripts --ignore-platform-reqs --no-dev; \
11    else \
12        composer install --prefer-dist --no-cache --no-scripts --ignore-platform-reqs; \
13    fi
14RUN composer dump-autoload --optimize

 1build-image:
 2  steps:
 3    - name: Build Container
 4      uses: docker/build-push-action@v5
 5      with:
 6        tags: project-image
 7        push: false
 8        outputs: type=docker,dest=/tmp/project.tar
 9        build-args: |
10          PRODUCTION=false

Quota Usage

If your workflows are running for a longer time and GitHub Actions quota is important to you, note that we have observed that running multiple steps in parallel using different jobs counts more to your quota since every job counts against your quota on its own. The following example is not representative of actual execution times since the job logs originate from different points in time but rather visualize the issue.

Comparison using jobs in parallel vs. steps ins sequence

Workflow File

Full GitHub Actions workflow file using PHPUnit (+MySQL), PHPStan and PHP-CS-Fixer.

  1name: Tests
  2
  3on: [ push ]
  4
  5jobs:
  6  build-image:
  7    name: Build image
  8    runs-on: ubuntu-latest
  9    steps:
 10      - name: Checkout repository
 11        uses: actions/checkout@v4
 12
 13      - name: Set up Docker Buildx
 14        uses: docker/setup-buildx-action@v3
 15
 16      - name: Build Container
 17        uses: docker/build-push-action@v5
 18        with:
 19          tags: project-image
 20          context: .
 21          push: false
 22          outputs: type=docker,dest=/tmp/project.tar
 23          build-args: |
 24            KEY=value
 25
 26      - name: Upload artifact
 27        uses: actions/upload-artifact@v3
 28        with:
 29          name: project
 30          path: /tmp/project.tar
 31
 32  phpunit:
 33    name: PHPUnit
 34    runs-on: ubuntu-latest
 35    needs: [ build-image ]
 36    services:
 37      database:
 38        image: mariadb:10.6
 39        env:
 40          MYSQL_DATABASE: web
 41          MYSQL_USER: web
 42          MYSQL_PASSWORD: web
 43          MYSQL_ROOT_PASSWORD: root
 44        ports:
 45          - 3306
 46        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
 47    steps:
 48      - name: Download artifact
 49        uses: actions/download-artifact@v3
 50        with:
 51          name: project
 52          path: /tmp
 53
 54      - name: Load image
 55        run: docker load --input /tmp/project.tar
 56
 57      - name: Run Tests
 58        uses: addnab/docker-run-action@v3
 59        with:
 60          image: project-image
 61          options: |
 62            --add-host=host.docker.internal:host-gateway
 63            -e APP_KEY=base64:abcdefghijklmnopqrstuvwxyz1234567890
 64            -e DB_PORT=${{ job.services.database.ports[3306] }}
 65            -e DB_HOST=host.docker.internal
 66            -e DB_USERNAME=web
 67            -e DB_PASSWORD=web
 68            -e DB_DATABASE=web
 69          run: |
 70            php artisan passport:keys
 71            vendor/bin/phpunit --testdox
 72
 73  phpstan:
 74    name: PHPStan
 75    runs-on: ubuntu-latest
 76    needs: [ build-image ]
 77    steps:
 78      - name: Download artifact
 79        uses: actions/download-artifact@v3
 80        with:
 81          name: project
 82          path: /tmp
 83
 84      - name: Load image
 85        run: docker load --input /tmp/project.tar
 86
 87      - name: Run PHPStan
 88        uses: addnab/docker-run-action@v3
 89        with:
 90          image: project-image
 91          run: vendor/bin/phpstan analyse
 92
 93  php-cs-fixer:
 94    name: PHP-CS-Fixer
 95    runs-on: ubuntu-latest
 96    needs: [ build-image ]
 97    steps:
 98      - name: Download artifact
 99        uses: actions/download-artifact@v3
100        with:
101          name: project
102          path: /tmp
103
104      - name: Load image
105        run: docker load --input /tmp/project.tar
106
107      - name: Run PHP-CS-Fixer
108        uses: addnab/docker-run-action@v3
109        with:
110          image: project-image
111          run: vendor/bin/php-cs-fixer fix --stop-on-violation --dry-run
112
113  remove-image:
114    name: Remove image
115    if: always()
116    runs-on: ubuntu-latest
117    needs: [ build-image, phpunit, phpstan, php-cs-fixer ]
118    steps:
119      - uses: geekyeggo/delete-artifact@v2
120        with:
121          name: project

Blog