> For the complete documentation index, see [llms.txt](https://docs.platform9.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.platform9.com/private-cloud-director/getting-started/self-hosted/using-custom-certificates.md). # Using Custom Certificates Starting from this release, you can configure DU-specific SSL/TLS certificates for your Self-hosted Private Cloud Director deployment. Previously, the system only used a single self-signed wildcard certificate generated during the deployment process. `airctl provision-certs` now supports four certificate modes, only one of which is ever active at a time: | Mode | Flag | Description | | -------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | Self-signed (default) | `--revert` | The automatically generated wildcard certificate, shared across every DU namespace | | cert-manager | `--cluster-issuer ` | A certificate per DU namespace, issued and auto-renewed by `cert-manager` via an issuer of your choice (for example, Let's Encrypt) | | BYOC wildcard | `--cert-path` / `--key-path` | A single certificate/key pair you supply, written to every DU namespace | | BYOC per-namespace (DU-specific) | `--cert-dir ` | A distinct certificate/key pair you supply for each DU namespace | You can switch directly between any two modes — including cert-manager to BYOC, or one BYOC flavor to the other — without reverting to self-signed as an intermediate step first. {% hint style="warning" %} `airctl provision-certs` replaces the previous `airctl renew-certs` command, which has been removed as of this release. If you are running an earlier release where `provision-certs` is not available, see [Configuring DU-Specific Certificates Manually](/private-cloud-director/getting-started/self-hosted/using-du-specific-certificates-manual.md) instead — it walks through the same `cert-manager` setup by hand. That page also has BYOC-specific guidance for upgrading onto a release with `provision-certs`. {% endhint %} ### Prerequisites (cert-manager mode) Before running `airctl provision-certs --cluster-issuer`, `cert-manager` must already be installed in the cluster with a working issuer — `provision-certs` does not install `cert-manager` or create the issuer for you. If you plan to use a BYOC mode instead, skip ahead to [Configure a BYOC Wildcard Certificate](#configure-a-byoc-wildcard-certificate) or [Configure DU-Specific (Per-Namespace) BYOC Certificates](#configure-du-specific-per-namespace-byoc-certificates) — their prerequisites are much lighter (just a valid certificate and key). | Requirement | How to confirm | | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `cert-manager` is installed in the cluster | `kubectl get pods -n cert-manager` | | A `ClusterIssuer` exists and is `Ready` | `kubectl get clusterissuer ` | | A DNS-01 solver is configured on the issuer if the node has no publicly reachable port 80 | `kubectl get svc -n ingress-nginx -o wide` — if only `443`/`444`/`10254` are exposed, ACME HTTP-01 cannot work and the issuer must use a DNS-01 solver (for example, Route53) instead | The example below sets up a `ClusterIssuer` using Let's Encrypt with a Route53 DNS-01 solver — the common choice for nodes without inbound port 80 reachability. Any `Ready` `cert-manager` `ClusterIssuer` works; the solver and ACME server are entirely up to you. {% stepper %} {% step %} #### Create the Route53 credentials secret Only the AWS secret access key is treated as sensitive — the access key ID is a plain field on the `ClusterIssuer` spec. ```bash kubectl create secret generic route53-credentials-secret -n cert-manager \ --from-literal=secret-access-key='' \ --dry-run=client -o yaml | kubectl apply -f - ``` {% endstep %} {% step %} #### Create the ClusterIssuer (staging first) Validate against Let's Encrypt's staging environment before spending a production rate-limit slot. ```yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: spec: acme: server: https://acme-staging-v02.api.letsencrypt.org/directory email: privateKeySecretRef: name: -account-key solvers: - dns01: route53: region: hostedZoneID: accessKeyID: secretAccessKeySecretRef: name: route53-credentials-secret key: secret-access-key ``` ```bash kubectl apply -f clusterissuer.yaml kubectl get clusterissuer ``` {% hint style="info" %} `READY: True` here only confirms the ACME account registered — it needs a syntactically valid email and a private key, nothing more. It does **not** prove the AWS credentials or Route53 permissions are correct; that's only exercised when a certificate actually triggers a DNS-01 challenge. {% endhint %} {% hint style="warning" %} `kubectl patch --type=merge` on the `solvers` list replaces the entire list entry, not just the fields you pass — patching in only `region`/`hostedZoneID` can silently wipe `accessKeyID` and `secretAccessKeySecretRef`. Always `kubectl apply -f` the full `ClusterIssuer` spec when changing anything inside `solvers`. {% endhint %} {% endstep %} {% step %} #### Validate with a throwaway certificate Before running `provision-certs`, confirm the DNS-01 → Route53 → issuance pipeline works end-to-end with a disposable certificate that doesn't share a secret name with anything real: ```yaml apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: route53-dns01-test namespace: spec: commonName: dns01test. dnsNames: - dns01test. issuerRef: name: kind: ClusterIssuer secretName: route53-dns01-test-tls ``` ```bash kubectl apply -f test-cert.yaml kubectl get certificate route53-dns01-test -n -w ``` Issuance typically takes a couple of minutes end to end — the ACME challenge usually reports `valid` well before the `Certificate` itself flips to `Ready`; give it time past that point before assuming it's stuck. Clean up once confirmed: ```bash kubectl delete certificate route53-dns01-test -n kubectl delete secret route53-dns01-test-tls -n ``` {% endstep %} {% endstepper %} ### Configure DU-Specific Certificates via cert-manager Once you have a `Ready` `ClusterIssuer`, hand it to `airctl` to provision certificates for every DU namespace: ```bash airctl provision-certs --cluster-issuer ``` This command: * Validates that `` exists and is `Ready` — it fails fast, before touching any namespace, if the issuer is missing or not ready. * Removes the internal `cert-manager-tls` replication label from each DU namespace, so the shared self-signed wildcard certificate is not re-synced over the new certificate. * Creates (or updates) a `cert-manager` `Certificate` resource in every DU namespace, requesting `.` from the issuer, and waits for it to become `Ready`. * Restarts any deployment in the namespace that mounts the certificate secret, so the new certificate is picked up immediately. * Persists `clusterIssuerName` to `/opt/pf9/airctl/conf/airctl-config.yaml`, so subsequent `airctl upgrade` runs know this namespace's certificate is managed by `cert-manager` and skip overwriting it with the self-signed wildcard cert. Renewal after this point is handled entirely by `cert-manager` — no further manual steps are needed. #### Switch from staging to production Once staging issuance is proven end-to-end, re-apply the `ClusterIssuer` with `acme.server` pointed at the production endpoint (`https://acme-v02.api.letsencrypt.org/directory`) — always a full `apply`, never a partial patch (see the `solvers` warning above). cert-manager does not proactively reissue an already-`Ready` certificate just because the issuer's server field changed, so re-run: ```bash airctl provision-certs --cluster-issuer ``` {% hint style="info" %} Let's Encrypt's production environment enforces a rate limit of 5 duplicate certificates per exact domain set per 7 days. Confirm staging issuance is fully working before switching. {% endhint %} #### Verify the certificate is actually being served A `Ready` certificate only means `cert-manager` wrote the secret — confirm the ingress is serving it, and confirm real trust (not `curl -k`, which skips verification and proves nothing): ```bash echo | openssl s_client -connect :443 -servername 2>/dev/null \ | openssl x509 -noout -subject -issuer -dates curl -v https:/// -o /dev/null 2>&1 | grep -E "SSL certificate verify|issuer:|subject:" ``` Staging certificates only verify successfully on a host that explicitly trusts the Let's Encrypt staging root — that's expected and not a sign anything is broken. ### Configure a BYOC Wildcard Certificate If you already hold a certificate/key pair — self-signed, from an internal CA, or from a public CA — and want to use the same one across every DU namespace, hand it to `airctl` directly instead of going through `cert-manager`: ```bash airctl provision-certs --cert-path /path/to/wildcard.crt --key-path /path/to/wildcard.key ``` Before writing anything, `airctl` validates that: * The private key matches the certificate. * The certificate has not already expired. * The certificate covers the DU's domain (via its Common Name or Subject Alternative Names). If any check fails, `provision-certs` exits without touching any namespace — validation runs before any existing certificate (cert-manager or otherwise) is torn down, so a bad certificate can't leave you in a half-migrated state. This command: * Writes the supplied certificate and key into every DU namespace's certificate secret, retrying with verification if `kubernetes-replicator` races the write. * Removes the internal `cert-manager-tls` replication label from each DU namespace, so the shared self-signed wildcard certificate is not re-synced over it. * Restarts any deployment in the namespace that mounts the certificate secret. * Persists `certMode: byoc-wildcard` to `/opt/pf9/airctl/conf/airctl-config.yaml`, so subsequent `airctl upgrade` runs know not to overwrite it. You are responsible for renewing this certificate yourself before it expires — unlike cert-manager mode, there is no automatic renewal. ### Configure DU-Specific (Per-Namespace) BYOC Certificates If you want a distinct certificate per DU namespace (for example, one per region) rather than a single shared wildcard, use `--cert-dir` instead. The directory must contain a `.crt`/`.key` pair for **every** DU namespace — including the `bork3` companion namespace: ``` byoc-certs/ ├── .crt ├── .key ├── -region1.crt ├── -region1.key ├── -bork3.crt └── -bork3.key ``` ```bash airctl provision-certs --cert-dir /path/to/byoc-certs ``` The same validation runs per file (key matches certificate, not expired, covers that namespace's domain) before any namespace is touched — if any single namespace's cert/key pair fails validation, none of the namespaces are written. This command: * Writes each namespace's certificate and key into that namespace's own certificate secret. * Removes the `cert-manager-tls` replication label from each DU namespace. * Restarts any deployment in each namespace that mounts the certificate secret. * Persists `certMode: byoc-per-ns` to `/opt/pf9/airctl/conf/airctl-config.yaml`. {% hint style="info" %} `--cert-path`/`--key-path`, `--cert-dir`, `--cluster-issuer`, and `--revert` are all mutually exclusive — pick exactly one per invocation. {% endhint %} ### Reverting to the Self-Signed Certificate To restore the automatically generated self-signed wildcard certificate at any time — regardless of which mode (cert-manager, BYOC wildcard, or BYOC per-namespace) is currently active — run: ```bash airctl provision-certs --revert ``` This command: * Deletes the `cert-manager` `Certificate` resource for each DU namespace, if one exists (a no-op if the active mode was BYOC). * Restores the original self-signed certificate and key directly into each namespace's certificate secret. * Re-adds the `cert-manager-tls` replication label to each namespace, so it resumes receiving the shared self-signed wildcard certificate. * Restarts any deployment that mounts the certificate secret. * Clears `clusterIssuerName` and resets `certMode` to `self-signed` in `/opt/pf9/airctl/conf/airctl-config.yaml`. {% hint style="info" %} Important Notes * Ensure that your `DU_FQDN` environment variable or the `duFqdn` field in the airctl configuration file matches the domain specified in your certificates. * The user running `provision-certs` must have access to a `Ready` `ClusterIssuer` in the cluster if using `--cluster-issuer`. * `--cluster-issuer`, `--cert-path`/`--key-path`, `--cert-dir`, and `--revert` are all mutually exclusive. * Only one mode is ever active at a time. You can switch directly between any two modes — self-signed, cert-manager, BYOC wildcard, or BYOC per-namespace — using `airctl provision-certs`; you do not need to `--revert` first as an intermediate step. {% endhint %} ### Monitoring Certificate Expiry Once a certificate is active — cert-manager, BYOC wildcard, or BYOC per-namespace — the DU's built-in Grafana includes a **TLS Certificate Expiry** dashboard so you can check expiry status without logging into the cluster: * **Days Until Expiry** — a live countdown, color-coded green/yellow/red * **Certificate Expiry Date** — the certificate's actual `notAfter` date * **Probe Status** — whether the DU's TLS endpoint is reachable at all * Trend graphs for both of the above over time This is backed by a dedicated probe (via `blackbox-exporter`) against the DU's externally-facing TLS endpoint, scraped hourly by Prometheus. {% hint style="info" %} The dashboard checks **one representative DU endpoint**, not every namespace individually. For cert-manager and BYOC wildcard mode this is sufficient, since every namespace shares the same certificate (and therefore the same expiry). If you are using BYOC per-namespace mode with certificates that have **different** expiry dates across namespaces, be aware that only one namespace's expiry is reflected here — track the others separately. {% endhint %} {% hint style="warning" %} This dashboard is for manual/visual monitoring only — automated alerting on certificate expiry is not currently configured. You are responsible for periodically checking this dashboard (or the certificate directly) and renewing before it expires. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.platform9.com/private-cloud-director/getting-started/self-hosted/using-custom-certificates.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.