There is one line in the bootstrap that decided how CI authenticates to this platform, and it was written before any CI existed:

resource "google_org_policy_policy" "disable_sa_key_creation" {
  name   = "${local.oneclub_folder}/policies/iam.disableServiceAccountKeyCreation"
  parent = local.oneclub_folder

  spec {
    rules {
      enforce = "TRUE"
    }
  }
}

Enforced at the folder, inherited by every project underneath. No service account in the organisation can have a JSON key minted for it. Not by me, not in a hurry, not "temporarily, for this one script".

So when it came time to let GitHub Actions deploy, federation was not a preference I had to justify against a simpler option. It was the only route that exists. That turns out to be a much better way to arrive at it than reading a best-practices page and feeling vaguely guilty.

The thing you are not doing

The default shape, the one every tutorial still shows, is: create a service account, download its key, paste the JSON into a GitHub repository secret, google-github-actions/auth with credentials_json.

What you have then is a bearer credential that never expires, sitting in a system that is not GCP, readable by anyone who can add a workflow file to that repository — which on most teams is anyone who can open a pull request, because a workflow triggered on pull_request_target runs with the repository's secrets. It does not rotate. Nothing tells you it leaked. Revoking it means finding every place it was pasted, and the honest answer to "where else is this key" is usually "I do not know."

Federation replaces the long-lived secret with a proof of identity that GitHub already issues.

What the exchange actually is

Every GitHub Actions job can ask GitHub for an OIDC token describing itself: which repository, which ref, which workflow, which actor. It is signed by GitHub, it is good for a few minutes, and it is useless to anyone who is not the audience it was minted for.

GCP's side is two resources. A pool, which is a namespace for external identities, and a provider inside it that says which issuer to trust and how to read its claims:

resource "google_iam_workload_identity_pool_provider" "github" {
  project                   = var.project_id
  workload_identity_pool_id = google_iam_workload_identity_pool.github.workload_identity_pool_id
  attribute_condition       = "assertion.repository_owner == '${var.github_org}'"

  attribute_mapping = {
    "google.subject"             = "assertion.sub"
    "attribute.repository"       = "assertion.repository"
    "attribute.repository_owner" = "assertion.repository_owner"
    "attribute.ref"              = "assertion.ref"
    "attribute.actor"            = "assertion.actor"
  }

  oidc {
    issuer_uri = var.issuer_uri
  }
}

The workflow hands that token to GCP's STS, gets a short-lived federated credential back, and uses it to impersonate a service account. No key is created at any point, which is exactly why the org policy does not object.

Two parts of that resource do the actual security work, and they are easy to get backwards.

attribute_condition decides who may enter the pool at all. Without it, any GitHub repository in the world can present a valid token to your provider. It is a valid token — GitHub signed it — and the provider will happily mint a federated credential for it. Whether that credential can do anything is a separate question, but the door is open. Ours requires assertion.repository_owner == '1club-ai'.

attribute_mapping decides what you can write bindings against. Mapping repository is what makes this possible on the service account:

principalSet://iam.googleapis.com/projects/<n>/locations/global/workloadIdentityPools/<pool>/attribute.repository/1club-ai/1club

That is a principal set, not a principal: it names every identity that entered the pool with attribute.repository equal to that string. The roles/iam.workloadIdentityUser binding on the deployer service account is scoped to it. The condition on the provider is the coarse filter — one organisation. The binding on each service account is the real one — one repository, one environment.

Three identities, not one

The instinct is to create "the CI service account" and grant it what CI needs. What exists instead is three, and the split is deliberate.

A CI identity, in the admin project. It builds the container image and pushes it to Artifact Registry, tagged with the commit SHA. That is all it does. It lives next to the registry, not in a workload project, because the registry is central and building an image has nothing to do with any particular environment.

A deployer identity, one per environment. It updates the Cloud Run service in its own project. roles/run.admin there, plus roles/iam.serviceAccountUser on that environment's runtime account — because deploying a service that runs as another identity requires permission to act as it, which is its own small gate and a good one.

A runtime identity, one per environment. This is what the container actually runs as. Its whole grant today is roles/cloudsql.client.

resource "google_service_account_iam_member" "deployer_act_as_runtime" {
  service_account_id = google_service_account.runtime.name
  role               = "roles/iam.serviceAccountUser"
  member             = "serviceAccount:${google_service_account.deployer.email}"
}

The reason to separate the last two is the one I would defend hardest. A deployment identity is held by CI, which executes code from pull requests. A runtime identity can reach the database. If they are the same account, then every workflow in the repository can reach production data, and the review process for "can this branch read the members table" is the review process for "can this branch change a GitHub Action" — which is to say, none.

Staging and production are separate accounts in separate projects for the same reason, one level up. A staging deploy that has gone wrong cannot reach production, because the token it holds was scoped at the exchange and there is nothing to escalate to.

The permissions you discover by their absence

This is the part that costs you an afternoon each, so here they are written down.

The Cloud Run service agent needs reader on the registry, and it is not the account you are thinking of. When Cloud Run pulls an image it does not use your runtime service account. It uses the project's Cloud Run service agent, service-<PROJECT_NUMBER>@serverless-robot-prod.iam.gserviceaccount.com, and if that principal lacks roles/artifactregistry.reader on the repository, the deploy fails with a message about the image not being found.

It is found. It is right there. You can pull it yourself from your laptop. The error tells you nothing about permissions because Cloud Run will not confirm the existence of an image it is not allowed to see, which is correct behaviour and completely unhelpful the first time. The registry module therefore takes the project number from a data source and constructs the principal rather than letting anyone paste it:

reader_members = [
  "serviceAccount:service-${data.google_project.prod.number}@serverless-robot-prod.iam.gserviceaccount.com",
  "serviceAccount:${data.terraform_remote_state.prod_iam.outputs.runtime_sa_email}",
]

Moving a floating tag needs a permission writer does not have. After a successful deploy the pipeline re-tags the deployed digest as :staging, so there is always a name for "what is currently deployed". roles/artifactregistry.writer covers tags.create and tags.update but not tags.delete, and moving an existing tag deletes it from where it was. The role that does cover it is repoAdmin, which also permits deleting versions and editing the repository's IAM — far too much for a retag. So there is a custom role with exactly one permission in it:

resource "google_project_iam_custom_role" "tag_deleter" {
  project     = var.project_id
  role_id     = var.tag_deleter_role_id
  title       = "Artifact Registry Tag Deleter"
  permissions = ["artifactregistry.tags.delete"]
}

One permission. It felt absurd to write and it is the correct size.

A public Cloud Run service collides with a default you did not set. The global load balancer's serverless network endpoint group carries no caller identity, so the Cloud Run service needs roles/run.invoker granted to allUsers or the load balancer returns a 403 before a request ever reaches Express. But new GCP organisations enforce Domain Restricted Sharing by default, and allUsers is not a member of your customer ID, so the binding is rejected with a message about the member not belonging to a permitted domain.

The fix is a per-project exception to iam.allowedPolicyMemberDomains, scoped to the one workload project that hosts a public service. Not at the folder. Every other project keeps the restrictive default. It is worth being precise about what this is: it is not "the service is unauthenticated". Ingress is still pinned to internal-and-cloud-load-balancing by org policy, so the only path in is the load balancer, and authorisation is the application's job. The allUsers binding is plumbing that lets the load balancer talk to its own backend.

Wiring it without copying strings

The pool lives in one Terraform root and the service accounts live in six others. The temptation is to terraform output the principal-set prefix once and paste it into each environment's tfvars, which works and then quietly rots the first time anything is recreated.

Instead each environment's IAM root reads it:

data "terraform_remote_state" "iam_wif" {
  backend = "gcs"
  config = {
    bucket = "1club-tf-state"
    prefix = "admin/global/iam-wif"
  }
}

locals {
  deployer_principal_members = [
    for repo in var.deployer_repos :
    "${data.terraform_remote_state.iam_wif.outputs.principal_set_prefix}/attribute.repository/${repo}"
  ]
}

Each root keeps its own state under its own prefix in the same bucket, so this is a read across a boundary rather than a merge of two stacks into one. The environment declares which repositories may deploy it; the prefix that makes that a valid principal comes from the root that owns the pool.

On the GitHub side there are two repository variables — GCP_WIF_PROVIDER and GCP_DEPLOYER_SA — and they are variables rather than secrets, because neither is one. A provider resource name and a service account email are not confidential; what protects them is that presenting them without a token GitHub signed for the right repository gets you nothing.

What this does not fix

Short-lived credentials are not the same as small credentials. The federated token expires in minutes, but for those minutes a workflow holds everything the deployer service account can do. If the deployer had roles/editor the federation would be theatre. The role list is the security boundary; the token lifetime only bounds the theft window. That is why the deployer's grants are run.admin on one project, serviceAccountUser on one account, and artifactregistry.writer plus one custom permission on one repository.

The organisation-wide attribute condition is genuinely coarse. Any repository under 1club-ai can enter the pool. What stops a new repository from deploying production is only the workloadIdentityUser binding, which names specific repositories. That is the correct place for the fine-grained rule, but it means the provider condition is a backstop, not a gate, and I would rather say so than let the assertion.repository_owner line look like more protection than it is.

And the admin project is now a concentration. The pool, the registry and the Terraform state all live in it, which is a direct consequence of the four-project ceiling the organisation is built under. A separate CI/CD project was in the original design and did not survive.

But the specific failure this all exists to prevent — a credential sitting in a system that is not GCP, with no expiry and no revocation story — is not mitigated here. It is unavailable. Nobody has a key because nobody can make one.