Skip to content

tofu-deploy-generate

The engine of the selective-environment promotion model (proven end-to-end in phpboyscout/infra before extraction — see the component spec): a release tag's pipeline is a thin parent — this job emits one concrete child-pipeline YAML per target from your environment catalog, and your manual trigger:deploy:<target> jobs include those artifacts as dynamic child pipelines. A child plans from the tag checkout and applies its same-pipeline artifact — never a plan banked on another ref, which eliminates the stale-plan/rebase race class entirely.

The generator engine is embedded in the component template itself (template + engine are one immutable file per release — no runtime fetching, no version skew); generator_path overrides it with a consumer-local variant. Two modes, selected by mode:

mode What it does Runs on
generate (default) Validates the catalog fail-closed and emits deploy-child-<target>.yml per target into out_dir, uploaded as long-lived artifacts (audit trail). Strict-semver release tags only (^v[0-9]+\.[0-9]+\.[0-9]+$hardcoded, deliberately not an input, so a component bump can never loosen the ref gate).
verify-fixtures Regenerates in --test mode and byte-diffs against your committed golden fixtures — any engine or catalog change shows its emitted-DAG delta in the MR diff. MR + default-branch pipelines.

The catalog (schema version 2 — the public interface)

Committed JSON at catalog_path. This schema is versioned deliberately; unknown versions — including v1 — are refused: migrate the catalog and the component pin in the same MR.

{
  "version": 2,
  "targets": {
    "dev": {
      "deployment_tier": "development",
      "working_directory": "src",
      "tfvars": "env/dev/src.tfvars",
      "backend_config": "env/dev/src.backend.hcl",
      "role_arn": "arn:aws:iam::111111111111:role/my-automation",
      "resource_group": "src/dev",
      "manual": false,
      "confirmation": null,
      "stoppable": true,
      "stop_tfvars": "env/dev/stop.tfvars"
    },
    "prod": {
      "deployment_tier": "production",
      "working_directory": "src",
      "tfvars": "env/prod/src.tfvars",
      "backend_config": "env/prod/src.backend.hcl",
      "role_arn": "arn:aws:iam::222222222222:role/my-automation",
      "resource_group": "src/prod",
      "manual": true,
      "confirmation": "Apply this release tag's plan to PRODUCTION? Review the plan job's output in this child pipeline first.",
      "stoppable": false,
      "stop_tfvars": null
    }
  }
}

Migrating from v1: set "version": 2 and add "stoppable": false, "stop_tfvars": null to every target, then regenerate fixtures; opt individual non-prod targets in when they have a stop overlay to apply.

Engine-enforced, fail-closed:

  • Tier guard: production and other tiers must be manual: true with non-empty confirmation — a catalog cannot make a production or control-plane target automatic. Any manual: true target needs confirmation text.
  • deployment_tierproduction | staging | testing | development | other (GitLab's tiers).
  • role_arn is a literal, format-validated IAM role ARN — never a CI variable reference (on GitLab Free an unprotected variable is overridable by whoever runs a pipeline; the account-side OIDC trust and allowed_account_ids are the real boundary).
  • tfvars / backend_config / working_directory must exist and resolve inside the repository root — no absolute paths or ../symlink escapes.
  • Stop-lane guard: stoppable: true is accepted only on development/staging tiers, and requires stop_tfvars (an existing, repo-contained file); every other target must carry stoppable: false, stop_tfvars: null. A production or control-plane target can never grow a stop lane.
  • Generated deploy jobs read no operator variables (TARGET_ENV, ENV, ROLE_ARN, TFVARS never appear); target names match ^[a-z][a-z0-9-]*$; output is deterministic sorted-JSON (valid YAML).

Emitted child pipelines

Per target: plan:<target>:tag (tofu plan from the tag checkout, artifact + terraform MR report) and deploy:<target>:tag (needs: the same-pipeline plan artifact; records environment with the catalog's tier; sets resource_group). Manual targets get a manual apply with manual_confirmation — the second gate, sitting after the tag-local plan output exists (GitLab disallows manual_confirmation on the parent's trigger jobs, and reviewing a real plan is the better gate anyway) — and allow_failure: false so the child (and a strategy: mirror trigger) stays incomplete until decided. Every job and the child's workflow are scoped to $CI_PIPELINE_SOURCE == "parent_pipeline"; the child cannot run standalone. Mechanics mirror tofu-plan / tofu-apply: tofu-tools image, OIDC id-token (aud: sts.amazonaws.com), GitLab HTTP state auth via $CI_JOB_TOKEN, plugin cache, partial-backend init via TF_CLI_ARGS_init.

Jobs

Job What it runs
tofu-deploy-generate Fetches the bundled engine at the component version (or uses generator_path), then per mode: generate — validate + emit + upload artifacts; verify-fixtures--test regenerate + diff -ru against fixtures_path.

Inputs

Input Type Default Description
image_version string "v0.1.0" tofu-tools image tag (python3 ships in it). Also embedded as the image of the emitted child plan/apply jobs — bumping it changes the emitted DAG, so regenerate your fixtures.
stage string generate GitLab CI stage.
job_name string "tofu-deploy-generate" Override to include the component twice (one generate + one verify-fixtures instance) without a collision.
mode string (generate|verify-fixtures) "generate" See the mode table above.
catalog_path string ".gitlab/deploy-catalog.json" Your deploy catalog (schema v2, above).
out_dir string ".generated" Where generate mode emits deploy-child-<target>.yml; uploaded as artifacts.
fixtures_path string ".gitlab/deploy-fixtures" Committed golden fixtures for verify-fixtures mode.
generator_path string "" Repo-relative consumer-local engine override. Empty = bundled engine at the component version. Overriding shifts engine/fixture-divergence responsibility to you.
artifacts_expire_in string "never" Generate-mode artifact retention — forever by default, so the exact deployed config stays auditable.

Usage

The component provides the generate/verify jobs; trigger jobs stay in your pipeline (component templates cannot emit one per catalog entry) — about 12 lines per target:

stages: [test, generate, plan, apply, deploy]

include:
  # MR/main: prove committed fixtures match the engine + catalog.
  - component: gitlab.com/phpboyscout/cicd/tofu-deploy-generate@<version>
    inputs:
      stage: test
      job_name: verify-deploy-fixtures
      mode: verify-fixtures

  # Release tags: emit the child configs the triggers include.
  - component: gitlab.com/phpboyscout/cicd/tofu-deploy-generate@<version>
    inputs:
      stage: generate

trigger:deploy:prod:
  stage: deploy
  needs: ["tofu-deploy-generate"]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: never
    - if: '$CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+$/'
      when: manual
  allow_failure: true          # an unpromoted tag pipeline stays green
  trigger:
    include:
      - artifact: .generated/deploy-child-prod.yml
        job: tofu-deploy-generate
    strategy: mirror           # the trigger job reflects the child's status

Regenerate fixtures locally after a catalog (or component-version) change — fetch the engine at your pinned version so local output matches CI:

curl -fsSL "https://gitlab.com/api/v4/projects/phpboyscout%2Fcicd/repository/files/scripts%2Ftofu-deploy-generate%2Fgenerate.py/raw?ref=<version>" -o /tmp/tdg.py
python3 /tmp/tdg.py --test --catalog .gitlab/deploy-catalog.json --out .gitlab/deploy-fixtures

Stop lanes (stoppable targets)

A stoppable: true target's child config gains a third job, stop:<target>:tagmanual, allow_failure: true, the same resource_group as the deploy (a stop and a deploy can never interleave) — and its deploy job's environment gains on_stop: stop:<target>:tag, so the Environments-page Stop button becomes real for that environment. The stop job strips the environment to its baseline: an overlay apply (tofu apply -var-file=<tfvars> -var-file=<stop_tfvars>, later file wins) that forces your fail-closed enable flags off — never tofu destroy (audit buckets, account-alias imports, and alert subscriptions survive a stop). The next normal deploy restores full configuration and reactivates the environment.

Because production/other targets never get an on_stop job, their Stop button stays cosmetic on GitLab Free — pressing it marks the environment stopped without running anything, and the next deployment reactivates it.

GitLab plays a stop action from the pipeline of the environment's latest deployment. If your stoppable environment also auto-deploys from the default branch, pair the trunk apply with the tofu-stop component (tofu-apply's environment_mode: stoppable + on_stop inputs wire it) so the trunk lane carries the same stop job — otherwise the button would only work when a tag deploy happens to be latest. See spec 0056-stoppable-environments.