Skip to content

CI workflows

What each workflow in .github/workflows/ does, by stage. How the stages connect: delivery pipeline. How to write a workflow: GitHub Actions style guide. The rule that a workflow supplies credentials rather than build logic: BUILD-2.

Every workflow's name: starts with its stage. These are the following: 1 Check, 2 Build, 3 Deploy, 4 Release, Ops. A required check matches a job name, not a workflow name.

1 Check

Workflow Trigger Does
check-lint-and-test.yml pull request, push to main The sweep. See below
check-pre-commit.yml pull request, push to main Runs the pre-commit hooks and the commit-message lint. Its pre-commit job is a required check on main
check-image-pins.yml pull request on image paths //infra/docker:check-pinned-tags and //infra/docker/go-service:check-base-image. The checks that need a registry
check-code-review.yml dispatch, with a pull request Automated review

check-workflow-status.yml turns a workflow's jobs into one named check, which GitHub can require. build-stream-execlet-images.yml uses it.

Sweep

check-lint-and-test.yml runs these on every pull request and push to main (MERGE-1):

  1. mise run '//...:lint'
  2. mise run '//...:test-fast'
  3. mise run '//...:test-integration'. Only the Go projects define it.

It covers every config root in the root mise.toml, with no affected set. Within a stage every task runs to the end, so the stage reports all of its failures. A failed stage skips the stages after it, so a lint failure reports quickly. Its Lint and test job is a required check on main (MERGE-1).

The runner supplies what the integration tests need. These are the following: Docker for testcontainers, redis-server and dockerlyceum/lyceum_cpu for stream_execlet, and Ansible from the root mise.toml for hydra-autoscaler.

2 Build

These four are the current build path. Each calls a Mise task, so the same command runs on your machine. See bootstrapping platform images.

Workflow Trigger Does
build-service-images.yml push to main, dispatch from any branch Caller matrix over the reusable workflow, one entry per service with an image, or per service in the services input
build-reusable-service-image.yml workflow_call Reusable: WIF auth, buildx, Mise, then //<service_dir>:push-image. Takes gcp_project_id, gcp_project_number, gcp_region, service_dir
build-base-images.yml push on base-image paths, dispatch Builds and pushes builder, go-runtime, python-runtime, then opens the digest-repin pull request as a GitHub App

Every push to main builds every service image, so every main commit has images to pin. push-image skips an image already pushed for its commit. Pinning stays manual. There is no nightly base-image rebuild, by design: see BUILD-14.

Legacy image workflows

build-legacy-croesus-images.yml, build-legacy-inference-images.yml, build-legacy-metric-server-image.yml, build-legacy-test-dummy-images.yml. Manual dispatch, pushing to the per-project US registries, with inline Docker steps rather than Mise tasks.

They still run production's builds, and are left alone until the consolidation reaches them. Do not copy them, and do not add to them (BUILD-2).

build-stream-execlet-images.yml builds the stream_execlet CPU and GPU images and pushes them to ghcr.io. It runs on a pull request, on a push to main and on dispatch. It does not go through a Mise task (BUILD-2).

3 Deploy

deploy-staging.yml applies the staging service units through //infra/neph:deploy-staging as the github-deploy account. It runs on every push to main, and on dispatch. See deploying staging.

4 Release

  1. release-trigger.yml is dispatched by hand. It pushes a release-* tag as the lyceum-release GitHub App.
  2. release-create.yml runs on that tag. It calls release-infra-deploy.yml against production.
  3. release-infra-deploy.yml runs Terraform, builds the binaries, then runs Ansible.

This path is broken. release-infra-deploy.yml is not maintained. Every deploying job needs its Fail on purpose job, so a release tag or a dispatch stops before it changes anything.

Ops

Workflow Trigger Does
ops-deploy-admin.yml dispatch Applies the admin unit
ops-db-migrate.yml dispatch Database migrations, run separately from release-infra-deploy.yml
ops-publish-lyceum-cli.yml dispatch Publishes lyceum-cli to PyPI
ops-deploy-dev-docs.yml pull request, push, dispatch Builds this wiki with //docs/wiki:build and publishes it with //docs/wiki:deploy
ops-api-monitoring.yml hourly, dispatch Probes the public API
ops-auto-reapprove-stacked-prs.yml pull request, review Hymenaeus re-approves a stacked pull request whose diff has not changed