cert-manager graph

cert-manager with Let's Encrypt: Certificate, CertificateRequest, Order and Challenge

Every object cert-manager creates to get a Let's Encrypt certificate through an HTTP-01 challenge: ClusterIssuer, Certificate, CertificateRequest with its CSR, ACME Order, Challenge, solver Pod and the final TLS Secret.

You write two things: a ClusterIssuer that knows how to talk to Let's Encrypt, and an annotation on your Ingress. From there cert-manager creates a chain of objects, each owned by the one before it, until a signed certificate lands in a Secret.

Knowing that chain is how you debug a certificate that never becomes ready. Start at the Certificate, follow it down to the CertificateRequest, the Order and the Challenge, and the first one that isn't ready has the error in its status.

Click a resource to see what it does, or follow the setup step by step.

You write itCreated by a controllerOutside the clusterreferencestrafficcreates

How it works

All 10 resources and what they do

ClusterIssuer letsencrypt-prodcert-manager.io/v1: Cluster-scoped, so Certificates in any namespace can use it. It names the ACME server, the email for expiry notices, the Secret for the account key, and how to solve challenges: HTTP-01 through Ingresses of class traefik.
Secret letsencrypt-prod-account-keyv1: The ACME account's private key. cert-manager generates it and registers the account the first time the ClusterIssuer is used. If it's deleted, cert-manager registers a new account on the next use, which is harmless.
ACME server Let's Encrypt: Issues the certificate once you prove you control the domain. For HTTP-01 it fetches http://shop.example.com/.well-known/acme-challenge/<token> over the public internet, on port 80.
Ingress shopnetworking.k8s.io/v1: Your app's Ingress. The cert-manager.io/cluster-issuer annotation tells cert-manager's ingress-shim to create a Certificate for every host listed under tls, stored in the named Secret.
Certificate shop-tlscert-manager.io/v1: Created by ingress-shim from the Ingress, and owned by it. You can also write Certificates yourself. It says which names you want, which issuer to use and which Secret to fill, and it renews the certificate when two thirds of its lifetime have passed.
CertificateRequest shop-tls-1cert-manager.io/v1: One per issuance, numbered by revision. spec.request holds a PEM certificate signing request (CSR) made from a new private key. It has to be approved before an issuer signs it, and cert-manager's built-in approver does that for its own issuers.
Order shop-tls-1-2389721634acme.cert-manager.io/v1: The ACME order for the names in the CSR. Let's Encrypt answers with one authorization per name, and cert-manager creates a Challenge for each. When they all pass, the Order is finalized with the CSR and the signed certificate comes back.
Challenge shop-tls-1-2389721634-310…acme.cert-manager.io/v1: One per domain. It records the token and key the ACME server expects to find, and its status.reason is where most failures show up, for example a 404 or a timeout from Let's Encrypt's side.
Pod cm-acme-http-solver-x7k2pv1: A tiny web server that answers the challenge. cert-manager also creates a Service and an Ingress of class traefik that route /.well-known/acme-challenge/<token> for shop.example.com to it, and deletes all three when the challenge is done.
Secret shop-tlsv1: Written by cert-manager with tls.crt (the certificate and chain) and tls.key. The Ingress controller reads it to serve HTTPS. cert-manager updates it in place on every renewal, and the controller picks up the change.

Common mistakes

Use the staging server, https://acme-staging-v02.api.letsencrypt.org/directory, while you get things working. Production has strict rate limits, such as 5 duplicate certificates per week for the same set of names.
HTTP-01 needs port 80 open to the internet and DNS for the name already pointing at your load balancer. Behind a private load balancer, or for a wildcard name, use DNS-01 instead.
A Certificate that never becomes ready says why somewhere down the chain. Run kubectl describe certificate, then certificaterequest, order and challenge in the same namespace. With the cmctl plugin, kubectl cert-manager status certificate shop-tls -n shop shows the whole chain at once.
A ClusterIssuer and an Issuer are different kinds. An Ingress annotated with cert-manager.io/issuer looks for a namespaced Issuer, and fails if only a ClusterIssuer of that name exists.
ingress-shim only acts on Ingresses with a tls section. The annotation alone, without spec.tls and a secretName, creates nothing.

The manifests you write

Everything above that is marked "You write it", in one file. The rest comes from add-ons and controllers.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ops@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  namespace: shop
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: traefik
  tls:
    - hosts: [shop.example.com]
      secretName: shop-tls
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: shop
                port:
                  number: 80

Keep going

cert-manager quizTest yourself on Certificates, Issuers, ACME challenges, renewal and trust distribution.ArticleHow to troubleshoot a cert-manager Certificate that never becomes ready