k8wiz
Back to Articles

Ready to test your Kyverno knowledge? Try our Kyverno quiz!

Cosign explained: sign and verify container images, step by step

What cosign is, how keyless signing with Fulcio and Rekor works, and how to sign, attest and verify images, then enforce signatures in Kubernetes with Kyverno.

15 min read
By k8wiz Team
  • cosign
  • sigstore
  • supply-chain
  • kyverno
  • oci
  • security

A registry tells you what an image contains. It doesn't tell you who built it. If someone pushes a new image under your tag, every cluster that pulls the tag runs it, and nothing in Kubernetes notices. Cosign closes that gap. It signs the exact bytes of an image, records who signed them, and lets anyone check that before the image runs.

This guide explains the concepts first, because most cosign mistakes come from trusting the wrong thing rather than from typing the wrong command. Then it goes step by step: sign an image from your laptop and from CI, verify it, attach an SBOM, and make Kubernetes refuse images your release pipeline didn't sign. The commands are for cosign 3. The interactive graph of the same setup shows how the pieces connect.

What cosign is

Cosign is the command-line tool of Sigstore, an OpenSSF project. It signs and verifies container images and anything else stored in an OCI registry, such as Helm charts and Flux artifacts. It can also sign plain files (cosign sign-blob) and attach signed statements about an image, called attestations.

Cosign works with three public services that Sigstore runs for free:

  • Fulcio, a certificate authority. It issues short-lived code-signing certificates to whoever can prove an identity with an OpenID Connect (OIDC) token, such as a Google account or a GitHub Actions job.
  • Rekor, a transparency log. It's an append-only public record of signatures. Anyone can search it, and its entries prove when a signature was made.
  • The TUF trust root. It distributes the Fulcio root certificates and the Rekor public keys that verifiers should trust, through The Update Framework, so keys can rotate without a new cosign release.

You can use cosign without Fulcio and Rekor, with your own key pair. Keyless signing with the public services is the default and the common choice in CI.

Concepts to get right first

Sign digests, not tags

An image has a tag, such as shop:1.4.2, and a digest, such as sha256:5e1a…. The digest is a hash of the image manifest, so it names exact content. A tag is a pointer anyone with push access can move.

Cosign signs a digest. If you give it a tag, it resolves the tag first and warns you, because the tag may already point somewhere else than the image you built. Always sign the digest your build step printed. Verification works the other way around: you can verify by tag, and cosign resolves it to a digest and looks for signatures of that digest.

For a multi-platform image, the digest you push is an image index that lists one manifest per platform. Sign the index digest. cosign sign --recursive also signs each platform manifest, for tools that look up a platform digest directly.

Key-based or keyless

With a key pair, you sign with a private key and people verify with the public key. The hard parts are the usual key problems: storing the private key safely, rotating it, and revoking it after a leak.

Keyless signing removes the long-lived key. Each time you sign, cosign:

  1. generates a fresh key pair in memory,
  2. gets an OIDC token that proves your identity,
  3. sends the public key and the token to Fulcio, which returns a certificate binding that key to your identity, valid for 10 minutes,
  4. signs the image digest with the private key and throws the key away,
  5. records the signature and certificate in Rekor.

The certificate expires minutes later, which sounds like it would break verification. It doesn't, because the Rekor entry proves the signature was made while the certificate was valid. A stolen key is useless: it was never written to disk, and its certificate expired long ago.

An identity is a subject and an issuer

A keyless signature tells you two things from the certificate: the subject, such as alice@example.com or a GitHub workflow URL, and the issuer, the OIDC provider that vouched for it. You need both to decide whether to trust a signature.

The issuer matters as much as the subject. An email address is only meaningful together with the provider that checked it. A rule that accepts the subject alice@example.com from any issuer is weaker than it looks.

For GitHub Actions, the issuer is https://token.actions.githubusercontent.com and the subject is the workflow file and the ref it ran from:

https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main

So the question a verifier asks is precise: was this digest signed by the release workflow of acme/shop, running on main? A signature from a fork, from another workflow or from a feature branch has a different subject and fails.

Where the signature lives

A signature can't go inside the image, because adding it would change the image's digest. Cosign 3 stores it next to the image as a separate OCI artifact, a Sigstore bundle. The bundle holds the signature, the Fulcio certificate and the Rekor entry with its inclusion proof, so a verifier has everything in one place.

The bundle is pushed as a manifest whose subject field points at the image digest. Registries that support the OCI 1.1 referrers API index it, and anyone can list everything that refers to an image. Registries without that API get a fallback tag named after the digest, sha256-5e1a…. The OCI images guide covers referrers in detail.

Older cosign versions used a different layout: a tag named sha256-5e1a….sig holding a signature manifest. You'll still see it on images signed before cosign 3.

Attestations

A signature says "this identity vouches for this digest". An attestation says something more specific about it: "this is the SBOM of this digest", or "this digest was built from commit 4f2c8e1 by this workflow". Cosign wraps the statement in the in-toto format, signs it the same way as an image, and attaches it as another referrer.

Each attestation has a predicate type, such as https://spdx.dev/Document for an SPDX SBOM or https://slsa.dev/provenance/v1 for SLSA provenance. Verifiers filter on it.

Step 1: install cosign

On macOS or Linux with Homebrew:

brew install cosign
cosign version

With Go:

go install github.com/sigstore/cosign/v3/cmd/cosign@latest

Release binaries for every platform are on the cosign GitHub releases page, signed with cosign itself. In GitHub Actions, use the sigstore/cosign-installer action, shown in step 4.

Step 2: build, push and keep the digest

Build and push the image, and ask buildx to write the result to a file:

docker buildx build --push \
  -t ghcr.io/acme/shop:4f2c8e1 \
  --metadata-file meta.json .

DIGEST=$(jq -r '."containerimage.digest"' meta.json)
echo "$DIGEST"   # sha256:5e1a…

For an image that's already pushed, crane digest ghcr.io/acme/shop:4f2c8e1 prints its digest. Either way, from here on you work with ghcr.io/acme/shop@$DIGEST.

You also need to be logged in to the registry with push rights, since cosign pushes the bundle there. Cosign uses the same credentials as Docker.

Step 3: sign keylessly from your laptop

cosign sign ghcr.io/acme/shop@$DIGEST

Cosign tells you the signature will be recorded in a public transparency log, which includes your identity, and asks you to confirm. Then it opens a browser so you can log in with GitHub, Google or Microsoft. The account's email address becomes the certificate's subject, and the provider becomes the issuer.

Think about that notice before you sign from a personal account. The Rekor log is public and permanent, so the email address and the image name stay in it.

Add --yes to skip the confirmation in scripts. In CI there's no browser: cosign picks up the job's own OIDC token, as the next step shows.

Step 4: sign in GitHub Actions

This is the setup to aim for: images signed by the pipeline that built them, with no secret to manage.

name: release
on:
  push:
    branches: [main]
permissions:
  contents: read
  packages: write
  id-token: write   # lets the job request an OIDC token for cosign
jobs:
  image:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: sigstore/cosign-installer@v4
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - id: build
        uses: docker/build-push-action@v6
        with:
          push: true
          tags: ghcr.io/acme/shop:${{ github.sha }}
      - name: Sign the image
        run: cosign sign --yes ghcr.io/acme/shop@${{ steps.build.outputs.digest }}

Three details matter:

  • id-token: write is what lets cosign request the token. Without it, signing fails with an error about getting an OIDC token.
  • The image is signed by steps.build.outputs.digest, the digest this job pushed, never by tag.
  • Pin actions to a commit SHA in real pipelines. A release workflow is exactly what an attacker wants to change.

The certificate subject for this job is https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main. GitLab CI and other providers work the same way with their own issuer and subject formats.

Step 5: verify the signature

cosign verify \
  --certificate-identity https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/shop:4f2c8e1

On success, cosign prints the checks it made on stderr:

Verification for ghcr.io/acme/shop:4f2c8e1 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates

and a JSON array on stdout, one entry per valid signature. "Verified offline" means cosign checked the inclusion proof stored in the bundle, without calling Rekor.

Both identity flags are required. Cosign refuses to verify a keyless signature without knowing whose signature you expect, which is the right default. When a signature exists but from someone else, the error names the identity it found:

Error: no matching signatures: none of the expected identities matched what was in the certificate, got subjects [https://github.com/acme/shop/.github/workflows/test.yaml@refs/heads/main] with issuer https://token.actions.githubusercontent.com

To accept releases from version tags instead of main, match the subject with a regular expression, anchored at both ends:

cosign verify \
  --certificate-identity-regexp '^https://github\.com/acme/shop/\.github/workflows/release\.yaml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/shop:1.4.2

Escape the dots and keep the anchors. https://github.com/acme/.* would also match a repository called acme/shop-fork that anyone in the organisation can create.

Step 6: sign with a key instead

Keys still make sense when the signer can't get an OIDC token, or when a security team wants signing tied to a key it controls in a KMS.

cosign generate-key-pair

This writes cosign.key, the private key encrypted with a password you choose, and cosign.pub. Set COSIGN_PASSWORD to use it from scripts. Then sign and verify:

cosign sign --key cosign.key ghcr.io/acme/shop@$DIGEST
cosign verify --key cosign.pub ghcr.io/acme/shop:4f2c8e1

Better than a key file is a key that never leaves a KMS. Cosign accepts KMS URIs wherever it takes a key:

cosign sign --key gcpkms://projects/acme/locations/global/keyRings/release/cryptoKeys/cosign ghcr.io/acme/shop@$DIGEST

awskms://, azurekms:// and hashivault:// work the same way. cosign generate-key-pair k8s://cosign-system/signing-key stores a new key pair in a Kubernetes Secret.

Key-based signatures are also recorded in the public Rekor log by default. If image names must not appear in a public log, the clean answer is a private Sigstore deployment, not turning the log off.

Step 7: attach an SBOM as an attestation

Generate an SBOM, for example with Syft, then attest it:

syft ghcr.io/acme/shop@$DIGEST -o spdx-json > sbom.spdx.json

cosign attest --yes \
  --type spdxjson \
  --predicate sbom.spdx.json \
  ghcr.io/acme/shop@$DIGEST

Verify it with the same identity flags, plus the type:

cosign verify-attestation \
  --type spdxjson \
  --certificate-identity https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/shop:4f2c8e1 | jq -r .payload | base64 -d | jq .predicateType

The payload is the in-toto statement, base64-encoded. Its subject names the image digest and its predicate holds the SBOM. Build provenance works the same way, with a SLSA provenance document as the predicate and --type slsaprovenance1.

Step 8: see what's attached to an image

Signatures and attestations are referrers of the image. List them with ORAS:

oras discover ghcr.io/acme/shop:4f2c8e1

You should see the signature bundle and the attestation bundle, both with the Sigstore bundle artifact type. If you copy images between registries, copy their referrers too: oras cp -r does that. A mirror that copies only the image drops the signatures, and verification at the destination fails.

Step 9: enforce signatures in Kubernetes with Kyverno

Verifying by hand protects nobody. The goal is a cluster that refuses to run an image your pipeline didn't sign. Kyverno does that at admission with an ImageValidatingPolicy. It replaces the verifyImages rule of the legacy ClusterPolicy, which Kyverno 1.20 no longer enforces.

apiVersion: policies.kyverno.io/v1
kind: ImageValidatingPolicy
metadata:
  name: require-signed-shop
spec:
  validationActions: [Deny]
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  matchImageReferences:
    - glob: "ghcr.io/acme/*"
  attestors:
    - name: release
      cosign:
        keyless:
          identities:
            - issuer: https://token.actions.githubusercontent.com
              subject: https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main
  validationConfigurations:
    mutateDigest: true
    verifyDigest: true
    required: true
  validations:
    - expression: >-
        images.containers.map(image, verifyImageSignatures(image, [attestors.release])).all(e, e > 0)
      message: shop images must be signed by the release workflow on main

How it reads:

  • matchImageReferences limits the policy to your images. Public images such as nginx aren't signed by your workflow, so a policy that matched everything would block them.
  • The attestor holds the same subject and issuer you passed to cosign verify. They're the whole trust decision, so review changes to them like code changes.
  • verifyImageSignatures returns how many signatures passed for an image. The expression requires at least one for every container.
  • mutateDigest: true rewrites the Pod's image to the verified digest, so the node pulls what was checked, even if the tag moves a second later. verifyDigest checks that images are pinned to a digest, and required rejects matched images no attestor verified.
  • Kyverno reads the referrers from the registry itself, so it needs its own credentials for a private registry. A pull secret that only the kubelet uses doesn't help it.

The same policy kind can require attestations too, for example an SBOM, with an attestations list and the verifyAttestationSignatures CEL function. Check the Kyverno documentation for your version before you copy field names.

Test it:

kubectl create deployment shop -n shop --image=ghcr.io/acme/shop:4f2c8e1
kubectl get pods -n shop -o jsonpath='{.items[0].spec.containers[0].image}'
# ghcr.io/acme/shop:4f2c8e1@sha256:5e1a…

Then try an image the workflow didn't sign. The Deployment is created, but its Pods aren't. kubectl describe rs -n shop shows FailedCreate events with the policy's message. Kyverno can also apply the same check to Deployments and other Pod controllers directly, so the error shows at kubectl apply time.

Roll out in steps. Start with validationActions: [Audit], read the policy reports to find unsigned images you didn't know about, sign or exempt them, then switch to Deny.

What changed in cosign 3

Cosign 3.0 turned on by default features that earlier releases only offered behind flags:

  • Signatures and attestations are stored as Sigstore bundles, attached through the OCI referrers API, instead of the sha256-….sig tag layout.
  • Cosign reads the URLs of the Sigstore services from a signing config distributed through the TUF trust root, instead of hard-coded defaults.

To produce the old layout, for a verifier that can't read bundles yet, pass --new-bundle-format=false together with --use-signing-config=false. The signing config requires the new format, so the first flag alone isn't enough.

Before upgrading the signing side, check every consumer: admission controllers, deploy tools and registry mirrors. A verifier that only looks for .sig tags reports "no signatures" for images signed with cosign 3.

Troubleshooting

no signatures found. Usually the verifier and the signer disagree about the digest. Common causes: you signed a platform manifest and verify the index, or the other way round (--recursive covers both); the image was copied to another registry without its referrers; or the verifier only understands the old tag layout.

none of the expected identities matched. The signature exists, but from another identity. The error prints the subject it found. Compare it character by character with what you expect: workflow file name, .yaml against .yml, and refs/heads/main against refs/tags/….

Signing fails in CI with an OIDC error. The job lacks id-token: write, or the workflow runs from a fork's pull request, which doesn't get an ID token for the base repository.

Kyverno denies a signed image. Run the same cosign verify command with the policy's subject and issuer. If that passes, check that Kyverno can reach the registry with credentials, and that its version reads bundles attached as referrers.

Checklist

  • Sign digests from the build output, never tags, and sign multi-platform indexes with --recursive.
  • Sign in CI with keyless signing and id-token: write. Keep keys for cases that really need them, and keep those keys in a KMS.
  • Verify with an exact subject and issuer. Use regular expressions only with anchors and escaped dots.
  • Attach SBOMs and provenance as attestations, so they travel with the image.
  • Copy referrers when you mirror images.
  • Enforce at admission with an ImageValidatingPolicy, scoped to your own images, with mutateDigest on. Start in Audit.
  • Before moving to cosign 3, make sure every verifier reads Sigstore bundles from referrers.

Test yourself

Ready to test your Kyverno knowledge? Try our Kyverno quiz!