From 1fe35a2ea6a3d5198356d3baad5dd74dd035d9bd Mon Sep 17 00:00:00 2001 From: Aaron Po Date: Mon, 31 Aug 2026 15:58:45 -0400 Subject: [PATCH] Use compose wait for CI test containers (#270) --- .github/workflows/tests.yml | 4 +- .gitignore | 1 + docs/website/testing.md | 89 +++++++++++++++++++++++++++++++++---- 3 files changed, 84 insertions(+), 10 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index ba812fb1..e6630881 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -24,7 +24,9 @@ jobs: run: docker compose --env-file .env.test -f docker-compose.test.yaml build - name: Run tests - run: docker compose --env-file .env.test -f docker-compose.test.yaml up --abort-on-container-exit + run: | + docker compose --env-file .env.test -f docker-compose.test.yaml up -d + docker compose --env-file .env.test -f docker-compose.test.yaml wait api.specs unit.tests frontend.tests || true - name: Check test container exit codes if: always() diff --git a/.gitignore b/.gitignore index 44912412..3e4c26ab 100644 --- a/.gitignore +++ b/.gitignore @@ -502,3 +502,4 @@ FodyWeavers.xsd *storybook.log storybook-static web/test-results/frontend/vitest-storybook.xml +web/test-results/frontend/playwright-storybook.xml diff --git a/docs/website/testing.md b/docs/website/testing.md index 06164cc8..9c9167c1 100644 --- a/docs/website/testing.md +++ b/docs/website/testing.md @@ -25,7 +25,8 @@ The easiest way to run all tests is using Docker Compose, which sets up an isolated test environment: ```bash -docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests ``` This command: @@ -36,7 +37,13 @@ This command: 4. Executes all test suites in parallel, including the frontend's Storybook Vitest and Playwright suites (`frontend.tests`, which needs no database) 5. Exports results to `./test-results/` -6. Exits when tests complete +6. Returns once all three test containers have exited + +`wait` blocks until `api.specs`, `unit.tests`, and `frontend.tests` have all +stopped, regardless of which one finishes first. Long-running services +(`sqlserver`, `seaweedfs`) and the one-shot `database.migrations` / +`database.seed` jobs keep running in the background alongside them and are +torn down separately (see below) — `wait` doesn't touch them. ### View test results @@ -338,16 +345,25 @@ containerized environment in CI as they do on a developer machine: ```bash # CI/CD command docker compose --env-file web/.env.test -f web/docker-compose.test.yaml build -docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v ``` -Because `--abort-on-container-exit` stops the stack as soon as any one-shot -test container exits (success or failure), the workflow separately inspects -the exit code of each test container (`test-env-api-specs`, -`test-env-unit-tests`, `test-env-frontend-tests`) after `up` returns and fails -the job if any of them is non-zero. `./test-results/` is uploaded as a build -artifact regardless of outcome. +The workflow inspects the exit code of each test container +(`test-env-api-specs`, `test-env-unit-tests`, `test-env-frontend-tests`) +after `wait` returns and fails the job if any of them is non-zero. +`./test-results/` is uploaded as a build artifact regardless of outcome. + +Earlier versions of this workflow used +`up --abort-on-container-exit`, which stops the entire stack the instant +*any* watched container exits. That's wrong here: the one-shot +`database.migrations`/`database.seed` jobs are expected to exit 0 partway +through, and even scoping the flag to just the three test services doesn't +help, since `frontend.tests` (no database dependency) reliably finishes +before `api.specs`/`unit.tests` even start, aborting them prematurely. +`up -d` + `wait` avoids both races by not tying container teardown to any +single container's exit. Exit codes: @@ -366,6 +382,61 @@ npm run test:storybook npm run test:storybook:playwright ``` +### Running the workflow locally with `act` + +[`act`](https://github.com/nektos/act) replays `.github/workflows/*.yml` on +your own machine using Docker, so you can reproduce a CI run (or a fix for +one) without pushing a branch. + +**Install** (macOS): + +```bash +brew install act +``` + +**Run the workflow** from the repo root: + +```bash +act push \ + -j containerized-tests \ + -W .github/workflows/tests.yml \ + -P ubuntu-latest=catthehacker/ubuntu:act-latest \ + --container-architecture linux/amd64 +``` + +- `-j containerized-tests` runs only that job (there's currently just the + one). +- `-P ubuntu-latest=catthehacker/ubuntu:act-latest` pins the runner image + act uses to impersonate `ubuntu-latest`. Without it, act's first run + prompts interactively to choose a default image size, which hangs/fails + under a non-interactive shell. +- `--container-architecture linux/amd64` is required on Apple Silicon: the + `sqlserver` service's image is amd64-only, and act needs to know to + emulate that platform for the whole job container, not just that one + service. +- No secrets or `.actrc` are needed — `generate-env.sh` creates its own + `.env.test` with freshly randomized values, same as in real CI. + +**Known act-only limitation**: the "Upload test results" step +(`actions/upload-artifact@v4`) fails locally with `Unable to get the +ACTIONS_RUNTIME_TOKEN env variable`. act doesn't provide a real Actions +artifact backend, so this step — and only this step — is expected to fail +under act even when everything else passes. It works normally on +GitHub-hosted runners. + +**Stale local state**: `docker-compose.test.yaml` uses fixed container +names and named volumes (`sqlserverdata-test`, `seaweedfsdata-test`) scoped +to the `web` compose project. If a previous local run (via `act` or a +manual `docker compose` invocation) didn't get torn down, its SQL Server +volume can persist with an old `SA_PASSWORD` baked in, while a fresh +`.env.test` generates a new one each run — causing `Login failed for user +'sa'` errors that look like a test bug but are really a leftover volume. +Clear it before re-running: + +```bash +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v +``` + ## Troubleshooting ### Tests failing due to database connection