Skip to content

Bootstrapping platform images

Build and publish every image a platform project holds, from your machine, without triggering CI. These are the following: the three shared base images (builder, go-runtime, python-runtime), and the service images. Everything CI does is a mise task or a terragrunt component, so locally you authenticate as yourself and run the same commands.

Container images explains how these images fit together. This page is the runbook for creating them.

platform-staging is built first, as the greenfield source of truth for images during the migration. platform-production is set up the same way. The steps take the platform environment as a variable and are identical for either. Only the project and its registries differ, and in platform-production only workflows on main may use the CI identity.

env=platform-staging                 # or platform-production
project=platform-staging-506908      # LYC_GCP_PROJECT_ID from infra/inventory/<env>/config.yml

Prerequisites

mise install                                   # from the repo root

# Authenticate: your identity for gcloud, application-default for terraform
gcloud auth login
gcloud auth application-default login
gcloud config set project "$project"

# Enable the APIs the platform project assumes are already on — it does not enable them
gcloud services enable \
  iam.googleapis.com iamcredentials.googleapis.com sts.googleapis.com \
  artifactregistry.googleapis.com cloudresourcemanager.googleapis.com \
  storage.googleapis.com \
  --project "$project"

# Let Docker push to the europe Artifact Registry
gcloud auth configure-docker europe-docker.pkg.dev --quiet

# mise resolves tool metadata from the GitHub API. Unauthenticated that is 60
# requests an hour, and past it mise warns repeatedly and cannot refresh the
# lockfile; a token raises the limit to 5000. CI sets the same variable.
export GITHUB_TOKEN="$(gh auth token)"   # or any token with public read access

You also need Docker with buildx (the tasks run docker buildx build). No ghcr.io login is required. Every image lives in Artifact Registry.

The builder image is multi-arch (linux/amd64,linux/arm64), so buildx needs the QEMU emulators registered for whichever architecture is not your own. This is a one-time step:

docker run --privileged --rm tonistiigi/binfmt --install all

CI does this with docker/setup-qemu-action. Locally the docker preflight checks for it and prints this command if the emulators are missing.

A platform project keeps its Terraform state in its own bucket, not the shared central bucket, and so does every environment it serves (INFRA-7). LYC_TERRAFORM_STATE_BUCKET in envs/<env>.env points the backend at <project>-terraform-state, a bucket in $project. You create it once in step 1, because this Terragrunt does not auto-provision it. So you need rights to create a GCS bucket in the project, which Owner grants, rather than access to any shared state bucket.

1. Apply the platform project

First create the Terraform state bucket. The platform/state_bucket unit owns it, and its own state lives inside it, so the bare bucket comes first:

LYC_DEPLOY_ENV="$env" mise run //infra/terraform/platform/state_bucket:apply

apply runs terragrunt backend bootstrap first, which creates the bare bucket in the location LYC_TERRAFORM_STATE_BUCKET_LOCATION names in envs/<env>.env. The unit then adopts it and sets the rest. platform-staging-506908-terraform-state is the reference, and every platform state bucket has its settings:

  • EU multi-region, standard storage class.
  • Uniform access, public access prevention enforced.
  • Object versioning on, soft delete 30 days.
  • Two lifecycle rules. Keep 30 noncurrent versions, and delete a noncurrent version after 10 days.

A plan of the unit shows where a bucket has drifted from them.

Then apply. This creates the twelve europe repositories, which are the two base-image repos plus one per service image, and the CI identity. It reads the project, number and region from infra/inventory/<env>/config.yml:

uv run neph terraform --env "$env" --component platform
Driving terragrunt directly instead of neph

neph applies this secret-free, VM-less platform project natively. It reads the plain inventory, skips the devops SSH key, and authenticates Terraform with your ADC. If you would rather run terragrunt yourself, then the component needs only five variables. --backend-bootstrap creates the bare state bucket, and LYC_TERRAFORM_STATE_BUCKET_LOCATION puts it in eu. A bucket's location cannot change afterwards. Apply platform/state_bucket first, so the bucket takes the reference settings before anything else stores state in it:

cd infra/terraform/platform
export TF_VAR_lyceum_env="$env" \
       TF_VAR_lyceum_project_id="$project" \
       TF_VAR_lyceum_project_number=<LYC_GCP_PROJECT_NUMBER> \
       TF_VAR_gcp_default_region=<LYC_GCP_DEFAULT_REGION> \
       LYC_TERRAFORM_STATE_BUCKET="$project-terraform-state" \
       LYC_TERRAFORM_STATE_BUCKET_LOCATION="eu"
terragrunt run --all --non-interactive --backend-bootstrap -- init -reconfigure
terragrunt run --all --non-interactive -- plan
terragrunt run --all --non-interactive -- apply

cleanup_dry_run defaults to true, so the registry cleanup policies are created in dry-run mode. It MUST stay there. See image retention.

2. Build and push the base images

Order matters

The base-image repositories must exist first, so step 1 has to precede this. The service images then build FROM these, so this step precedes step 4.

DOCKER_GCP_PROJECT="$project" mise run //infra/docker/builder:push-image
DOCKER_GCP_PROJECT="$project" mise run //infra/docker/go-runtime:push-image
DOCKER_GCP_PROJECT="$project" mise run //infra/docker/python-runtime:push-image

The tasks default to the platform-staging project. The explicit DOCKER_GCP_PROJECT keeps the runbook the same for every platform project. The images publish to europe-docker.pkg.dev/$project/{builder,go-runtime,python-runtime}.

3. Pin the base-image digests, then commit

The push tasks recorded the digests under infra/build_artifacts/docker/. The repin task rewrites every reference to them. These are the following: the FROM pins in both shared service Dockerfiles, and the builder the amd64 wheel build runs in.

mise run //infra/docker:repin-base-images
git add infra/docker/go-service/Dockerfile infra/docker/python-service/Dockerfile \
        mise/scripts/py-build.bash
git commit -m "[build] Repin base-image digests"

Locally you just commit. The open-base-image-repin-pr task is only for the automated CI pull request.

4. Build and push the service images

Each service's identity (image name, binary, directory) lives in its own mise.toml, so the only thing to supply is where to push. You push as yourself (Owner implies artifactregistry.writer). WIF is not involved.

# every service image, Go and Python
DOCKER_GCP_PROJECT="$project" mise run '//...:push-image'

Push a single service the same way with its project path, e.g. DOCKER_GCP_PROJECT="$project" mise run //croesus:push-image.

5. Verify

gcloud artifacts docker images list \
  "europe-docker.pkg.dev/$project/inference-proxy" --include-tags

How CI runs the same steps

The build-base-images.yml and build-service-images.yml workflows call these identical tasks. They add only the runner's credentials and buildx. The set-up-gcloud action derives the WIF provider from the project number and assumes github-actions@<project>, which step 1 creates. Nothing is set outside the repository.

A deploy of staging assumes a separate account. See Deploying staging.