Artifact Immutability as a Release Safety Primitive: Go Binary Signing, Digest Pinning, and the Promotion Pipeline

Most CI/CD pipelines treat the artifact as a byproduct rather than a contract. A tag like v1.4.2 points to whatever image the registry resolves it to today—and that resolution can change silently through a docker push overwrite, a registry garbage-collection race, or a compromised build step. For Go microservices operating at production scale across multiple environments, this silent mutability is not a theoretical concern; it is the mechanism through which supply-chain compromises propagate and through which "works in staging" stops meaning anything verifiable.

This article treats artifact immutability as a first-class release safety primitive and walks through the mechanics of enforcing it in a Go promotion pipeline: what to sign and when, how digest pinning interacts with Kubernetes admission, and where the gates belong architecturally.

Why Tags Are Not a Safety Primitive

An OCI image tag is a mutable pointer. The registry stores a content-addressable manifest identified by its SHA-256 digest; a tag is just a named alias over that digest. When you kubectl set image deployment/api api=registry/api:v1.4.2, Kubernetes stores the tag string, resolves it to a digest at pull time, and records nothing durable about which digest it actually ran. Two deployments of the same tag across different nodes in a rolling update can pull different digests if a push races the rollout—and Kubernetes will happily run both without surfacing the discrepancy.

The fix is always deploying by digest: registry/api@sha256:<digest>. This converts the mutable pointer into a content-addressed reference. But digest pinning is only useful if you can prove which digest corresponds to which build, which requires signing.

The Go Binary as the Root of Trust

In a Go microservice pipeline, the immutability chain starts at the binary, not the container image. The container is assembled from a binary; if the binary is not attested before packaging, you cannot reason about the image's provenance.

Reproducible Builds

Go's toolchain produces reproducible binaries when you control the build environment: same Go version, same GOFLAGS, same CGO_ENABLED=0, same module graph, no embedded timestamps or random salts. The buildinfo package exposes what went into a binary at runtime:

import "runtime/debug"

func printBuildInfo() {
    info, ok := debug.ReadBuildInfo()
    if !ok {
        return
    }
    for _, s := range info.Settings {
        fmt.Printf("%s = %s\n", s.Key, s.Value)
    }
}

The output includes vcs.revision, vcs.time, GOARCH, GOOS, and whether CGO was enabled. This is your binary's fingerprint. A CI step that hashes the resulting ELF and compares it against a reproducible rebuild is a strong tamper-detection gate without any external key infrastructure.

Signing with Cosign and SLSA Provenance

For production pipelines, hash comparison alone is insufficient because it only proves the binary is internally consistent—it does not prove it came from your CI system. Cosign's keyless signing mode, backed by a Sigstore transparency log, anchors the signature to a short-lived OIDC identity issued by your CI provider (GitHub Actions, GitLab, etc.). The signature is stored in the OCI registry alongside the manifest as a referrer artifact.

The signing step belongs immediately after the binary is built and before the container image is assembled:

# Build the Go binary with build info
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
  -ldflags "-X main.version=$(git rev-parse HEAD)" \
  -trimpath \
  -o ./bin/api ./cmd/api

# Hash and record the binary digest
sha256sum ./bin/api > ./bin/api.sha256

# Build and push the OCI image, capture the digest
IMAGE_DIGEST=$(docker buildx build \
  --push \
  --platform linux/amd64 \
  --iidfile /tmp/iid \
  -t registry/api:${GIT_SHA} . 2>&1 | \
  grep 'digest:' | awk '{print $2}')

# Sign the image digest, not the tag
cosign sign --yes registry/api@${IMAGE_DIGEST}

GLSA provenance is generated alongside: the provenance attestation records the exact build inputs, builder identity, and output digest as a verifiable document attached to the image as a second referrer. Consumers verify before running:

cosign verify registry/api@${IMAGE_DIGEST} \
  --certificate-identity-regexp="https://github.com/your-org/" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com"

This verification step is where most pipelines stop. The harder problem is making verification mandatory at admission time in Kubernetes.

Kubernetes Admission as the Enforcement Plane

Verifying a signature in CI is advisory unless something downstream enforces it. A policy engine deployed as a Kubernetes validating admission webhook—Kyverno or OPA/Gatekeeper—can reject any pod whose image reference is a mutable tag or whose image's Cosign signature does not verify against your known issuer and subject patterns.

A Kyverno ClusterPolicy enforcing digest-only references looks like:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-image-digest
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-image-digest
      match:
        any:
          - resources:
              kinds: [Pod]
      validate:
        message: "Image must be referenced by digest, not tag."
        pattern:
          spec:
            containers:
              - image: "*@sha256:*"

Combine this with a Cosign verification policy and you have a closed enforcement loop: CI signs, the pipeline substitutes the digest into the Kubernetes manifest, and admission rejects anything that bypasses the process.

The tradeoff is operational: digest-pinned manifests require active management. When you promote an image from staging to production, you are promoting a specific digest, not rebuilding. Your promotion tooling must update the manifest digest in the deployment repository—GitOps repositories, Helm values.yaml, or Kustomize overlays—atomically with the promotion event.

The Promotion Pipeline Architecture

The architectural pattern that makes all of this tractable is treating promotion as a controlled state transition rather than a re-deployment:

Build Stage
  go build → binary digest
  docker buildx → image digest
  cosign sign → attestation stored in registry
  ↓
Staging Gate (automated)
  deploy by digest to staging
  run integration + load tests
  record test results as attestation on the digest
  ↓
Promotion Gate (policy-controlled)
  verify: image signature present
  verify: staging test attestation present and passed
  verify: no CVEs above threshold (Grype/Trivy scan attestation)
  verify: SLSA provenance chain intact
  → if all pass: write digest to production manifests via PR or direct GitOps commit
  ↓
Production Deployment
  ArgoCD/Flux syncs digest-pinned manifest
  Kyverno admission enforces digest + signature
  Rollout proceeds; previous digest retained for rollback

The promotion gate is a pipeline job that programmatically assembles the attestation evidence before allowing any manifest mutation. In Go, querying the Sigstore transparency log and verifying attestations is done via the github.com/sigstore/cosign/v2/pkg/cosign library, which lets you build this gate as a standalone verifier binary that your pipeline executes rather than shelling out to the CLI—giving you structured error handling and auditability.

Failure Modes and Operational Tradeoffs

Registry unavailability during verification. If the Sigstore transparency log or your registry is unavailable during admission, Kyverno's webhook will fail depending on its failurePolicy. Set failurePolicy: Fail for security-sensitive workloads; accept that a Sigstore outage can block deployments and plan for an emergency bypass procedure protected by break-glass controls.

Digest staleness in long-lived branches. If a feature branch deployment runs for days, its pinned digest may diverge significantly from main. Establish a maximum digest age policy: any digest older than N days must be rebuilt. This forces periodic revalidation of base image CVE posture.

Multi-architecture images. When building for linux/amd64 and linux/arm64, the image reference is a manifest list with its own digest. Sign and pin the manifest list digest, not the per-platform digests—otherwise platform-specific pulls will bypass verification on some nodes.

Rollback semantics. Digest-pinned deployments make rollback precise: you are reverting to a known-good, previously-verified digest rather than re-building and hoping for reproducibility. Keep a digest history in your deployment record; automated rollback triggered by SLO breach should reference this history rather than re-resolving a tag.

Decision Framework

Before implementing this pipeline, evaluate your threat model and operational maturity against these questions:

1. Is your current deployment system mutable-tag-based? If yes, digest pinning is the highest-leverage first step. Signing and admission can follow.
2. Do you have a GitOps repository? Promotion as a manifest-write PR is significantly cleaner than imperative kubectl pipelines and gives you a durable audit log.
3. What is your acceptable blast radius from a compromised build? If a single compromised image can reach customer data, mandatory admission verification is non-negotiable. If environments are strongly isolated, advisory verification may be proportionate.
4. Can your team operate break-glass procedures? Every enforcement layer adds a potential deployment blocker. Define, document, and access-control the bypass path before you need it at 2 AM.
5. Are your base images from a controlled registry? Signing your application layer is undermined if the base image (scratch, distroless) is pulled from an unverified source. Pin base image digests in your Dockerfile and include them in the SLSA provenance.

Artifact immutability is not a compliance checkbox. It is the mechanism that makes "deployed the same thing we tested" a verifiable statement rather than an assumption. For Go services where the binary itself is the primary deliverable, the chain from go build output to running container is short enough to close completely—and closing it removes an entire class of subtle, high-impact production failures.