Skip to content

Testing

cmp-issuer validation spans unit tests, envtest, OpenSSL interoperability, Kind e2e and manual verification against real CMP servers.

Unit and envtest

make test

Coverage includes:

  • Protocol adapter encoding and verification
  • Signer and controller reconciliation
  • Deferred enrollment over real CMP messages with transaction state on a real API server
  • Delayed confirmation and nonce echo rules
  • Transaction identifier pinning across a retry and recovery of an already issued chain

The run writes cover.out and reports one figure for api/ and internal/ together, set by COVER_PACKAGES. Go otherwise credits coverage only to the package a test binary lives in, which makes a package holding no test file of its own read as 0% even when every run executes it. Inspect the result per function with go tool cover -func=cover.out or in a browser with go tool cover -html=cover.out. cmd/ and test/utils/ are reported as having no test files, which is accurate: the first is manager wiring and the second is scaffolding for the e2e suite. CI uploads cover.out to Codecov, which leaves out the generated deepcopy functions.

make test regenerates the CRDs, the RBAC rules and the DeepCopy code before it runs anything, so a stale generated file is repaired on the spot rather than reported. CI therefore runs go mod tidy and the generation targets in a step of their own and fails when the working tree changed, which is what turns a forgotten make manifests generate into a red run instead of a green one on a tree that no longer matches the commit. Run go mod tidy and make manifests generate fmt locally and commit what they change.

test/workflows runs in the same command and checks the GitHub Actions workflow definitions rather than Go code, so it reports no coverage either. It fails a change that references an action by tag or branch instead of a commit SHA, that drops the version comment which makes a pinned SHA reviewable, that interpolates a ${{ }} expression into a run: block instead of passing the value through env, or that adds a workflow with no top-level permissions key. These are the conventions every workflow here already follows, and nothing else enforces them.

test/repository checks the repository itself: governance and support files, citation identity and its released version, EditorConfig settings, tracked-file whitespace, Git line-ending and binary rules, the toolchain-free pre-commit hooks plus the DCO sign-off check behind make dco-check. The workflow suite separately requires release source archives, checksums and signed build provenance. These checks keep metadata changes reviewable through the same make test command as the Go behavior.

OpenSSL CMP interoperability

CI runs delayed transaction, delayed confirmation, pinned transaction, IR and KUR flows against the OpenSSL CMP mock. The KUR flows have the mock check the oldCertID the request carries, which needs the -ref_cert option that OpenSSL added in 3.2, so the workflow installs a current OpenSSL rather than using the runner's 3.0 package. IR covers both protection modes with and without a requested validity. KUR covers new-key and same-key CRMF proof of possession. The pinned transaction test asserts that the identifier recorded before sending is the identifier OpenSSL saw on the wire, which is what a retry after an interruption reuses. The workflow fails if the test is silently skipped when OpenSSL is available.

Locally set OPENSSL_BIN to select a non-default OpenSSL binary. The test skips when OpenSSL is missing, and the KUR flows skip on their own when the build predates -ref_cert while the other flows still run.

The mock uses one transaction per TCP connection. A pooling proxy forwards tests without changing CMP bytes.

Kind e2e suite

make test-e2e

Requires Colima or Docker for Kind. The suite:

  • Creates an isolated Kind cluster
  • Installs cert-manager and cmp-issuer
  • Verifies issuer readiness and Secret naming on failure
  • Validates accepted p10crResponseCertReqId values
  • Confirms denied requests send no CMP traffic and store no certificate
  • Confirms connection failures store no partial material
  • Confirms crafted private-key annotations do not expose Secrets
  • Confirms Secret access requires the documented RoleBinding

Runtime is about one minute after the cluster is ready. Do not run this suite against a production cluster.

Other Kubernetes and cert-manager versions

Two variables select the versions, so one suite covers the whole supported range:

Variable Effect
KIND_NODE_IMAGE Kubernetes version of the Kind cluster, for example kindest/node:v1.34.11. Empty uses the default of the pinned Kind release
CERT_MANAGER_VERSION cert-manager release the suite installs, for example v1.19.6
make test-e2e KIND_NODE_IMAGE=kindest/node:v1.34.11 CERT_MANAGER_VERSION=v1.19.6

CI runs the suite once per supported combination after a change lands on dev or main. Pull requests omit this expensive matrix, so a change that only works on the newest Kubernetes or cert-manager is reported on the development branch before release:

Kubernetes cert-manager
v1.37, the default of the pinned Kind release v1.21.3
v1.36 v1.21.3
v1.35 v1.20.5
v1.34 v1.19.6

The three older rows pin a digest, taken from the node images published for the Kind release pinned as KIND_VERSION in the Makefile, so they have to be updated with it. The newest row pins no image and follows the Kind default.

Enrollment against a CMP server in the cluster

make test-e2e-ejbca

This runs a second set of specs that enroll real certificates from EJBCA Community Edition running in the same Kind cluster, through cert-manager Certificate resources and the full controller path:

Spec What it enrolls
Shared secret over HTTP PasswordBasedMac protected P10CR against a plain endpoint
Shared secret over HTTPS the same request over TLS, with the endpoint certificate verified against a pinned authority
Certificate signature over HTTP Signature protected P10CR using a registration certificate
KUR with key rotation P10CR revision one then certificate-authenticated KUR with a new requested key
KUR with key reuse P10CR revision one then certificate-authenticated KUR with the existing key
IR then KUR PasswordBasedMac protected IR with a requested validity, then the KUR an omitted renewal selects
Renewal fallback IR revision one, then IR revision two under renewalFallback: InitialEnrollment after the Secret is deleted, with the recorded reason and the Warning Event checked
Transaction records one CMPTransaction per enrollment, each reporting Issued

Each issued certificate is checked against the authority that signed it and against the chain stored in its Secret. The CMP response trust anchor and the endpoint TLS trust anchor are two different authorities, so a run also proves the two trust decisions stay separate.

The server image already carries its certification authorities, its CMP aliases and a TLS certificate issued for the Service name the suite uses, so a run costs a container start rather than a certificate authority setup. It is pulled when it has been published and built locally otherwise:

make ejbca-test-image

EJBCA_VERSION in the Makefile selects the upstream release, and EJBCA_IMAGE_REVISION republishes the image after a configuration change. test/e2e/ejbca/README.md describes what the image contains and why the aliases are configured the way they are.

CI runs these specs in their own job after a change lands on dev or main and rebuilds the server image only when the upstream release it is built from is republished under a new digest. Pull requests omit the EJBCA job.

Chart installation

make helm-lint
make helm-deploy IMG=<image>

make helm-lint renders the chart and asserts the value combinations that change what is rendered, and it needs no cluster. test-chart.yml installs the chart into a Kind cluster on top of that, and it installs each mode that changes the arguments the manager is started with or the permissions it is granted, not only the default values:

Mode What an install proves
Default values The chart installs and the manager becomes available
rbac.namespaced=true The namespaced Role and its bindings are the ones the manager is granted
metrics.secure=false The plain HTTP metrics endpoint starts instead of failing at flag parsing
metrics.tls.certManager.enabled=true The manager mounts the certificate cert-manager issues for the metrics endpoint

Every install waits for the Deployment to become available, which is how a mode that renders correctly but leaves the manager unable to start is caught. The release is uninstalled between modes, because a RoleBinding roleRef is immutable and upgrading an existing release into namespaced RBAC fails for a reason that says nothing about the chart. The workflow runs on a pull request only when the chart or a build input changes.

Interoperability against real CMP servers

Enrollment against Nokia NCM is verified against a dedicated CMP server, because that product is not distributable as a test image. Outcomes are summarized in Tested PKIs.

Enrolling from a hosted NCM instance in CI

interop-ncm.yml performs a complete enrollment against a hosted Nokia NCM instance: it creates a Kind cluster, installs cert-manager and cmp-issuer, stores the endpoint credentials, creates a CMPIssuer, enrolls a Certificate and inspects the certificate that comes back, then renews it through P10CR re-enrollment and renews two further certificates through KUR.

It is the only workflow that reaches a PKI outside the repository. It runs automatically for trusted pushes to dev and main and once a week, but never for pull requests. Start it from the Actions tab to choose different Kubernetes or cert-manager versions. Automatic runs use the default Kind node and cert-manager v1.21.3 and cover every enrollment and renewal without further input.

What each run covers

One job runs per protection mechanism and transport, so a fully configured endpoint exercises four independent enrollments:

HTTP HTTPS
PasswordBasedMac shared secret, plain transport shared secret, TLS transport
Signature request signing certificate, plain transport request signing certificate, TLS transport

Each job builds its own Kind cluster, so the four run in parallel and a failure in one does not mask the others. Configure only the transports the server exposes: a missing NCM_CMP_HTTP_URL or NCM_CMP_HTTPS_URL skips that column rather than failing it. Every job asserts that the endpoint it received actually uses the scheme of its own row, so a secret pasted into the wrong box fails loudly instead of quietly enrolling over HTTP twice.

Every job then renews the enrolled certificate through P10CR re-enrollment and two further certificates through KUR, over the transport of its own row. KUR uses a second CMPIssuer carrying protocol.renewal: KUR and the enrollment endpoint, with no endpoint.renewalUrl, because NCM answers a key update there:

Renewal Rotation policy What it proves
P10CR re-enrollment Always A fresh PKCS #10 for an already certified identity returns a new certificate
KUR with a new key Always The update returns a new serial number on a different public key
KUR with the existing key Never The update returns a new serial number on the exact existing public key, which needs a profile that certifies the same key twice

Configuration

Configure it under Settings, Secrets and variables, Actions. A repository without these values reports what is unconfigured and skips the enrollment rather than failing:

Secret Required Contents
NCM_CMP_HTTP_URL one URL at least Plain HTTP CMP endpoint, for example http://ncm.example:8080/pkix/
NCM_CMP_HTTPS_URL one URL at least HTTPS CMP endpoint, for example https://ncm.example/pkix/
NCM_CMP_RECIPIENT_DN yes Distinguished name of the issuing CA, sent as spec.protocol.recipient
NCM_CMP_RESPONSE_TRUST yes PEM anchor that signs the CMP responses and the issued chain
NCM_CMP_PBM_REFERENCE for PasswordBasedMac Reference number identifying the shared secret
NCM_CMP_PBM_SECRET for PasswordBasedMac Shared secret value
NCM_CMP_BOOTSTRAP_CERT for Signature PEM certificate that signs the CMP requests
NCM_CMP_BOOTSTRAP_KEY for Signature PEM private key for that certificate
NCM_CMP_BOOTSTRAP_CHAIN no PEM issuers of that certificate, sent in extraCerts, which also sets chainKey
NCM_CMP_HTTPS_TRUST no PEM anchor for the HTTPS server certificate, when it is not publicly trusted
Variable Required Contents
NCM_CMP_COMMON_NAME no Common name to enroll. Left unset it becomes cmp-issuer-test-<UTC YYMMDD-HHMMSS>, so every run enrolls a distinct identity
NCM_CMP_COUNTRY no Country requested in the subject, for example PL. Omitted entirely when unset
NCM_CMP_ORGANIZATION no Organization requested in the subject, for example cmp-issuer. Omitted entirely when unset
NCM_CMP_KUR no false skips both KUR renewals. Any other value, including unset, runs them
NCM_CMP_KUR_SAME_KEY no false skips only the KUR renewal that reuses the existing key
NCM_CMP_REENROLL no false skips the P10CR re-enrollment. Any other value, including unset, runs it

Leave NCM_CMP_COMMON_NAME unset unless you need a fixed name. A server profile that authorizes an identity to enroll only once refuses every run after the first, and a timestamped name sidesteps that without any server-side cleanup between runs.

The KUR certificates enroll under a generated name and have no variable of their own, because each is a second identity alongside the one NCM_CMP_COMMON_NAME names. A renewal needs no credential beyond the certificate being updated, so all three run wherever an endpoint is configured. Turn one off only for a profile that refuses it: NCM_CMP_REENROLL where an identity is certified once, NCM_CMP_KUR where key update is disabled, NCM_CMP_KUR_SAME_KEY where the same key cannot be certified twice.

Requesting a country or an organization is a request, not an instruction. A CA that enforces its own subject answers grantedWithMods, which the issuer rejects under policy.grantedModifications: Reject, so an enrollment that fails only after these variables are set points at the server profile rather than at the issuer. Clear both to fall back to a bare common name.

NCM_CMP_RESPONSE_TRUST and NCM_CMP_HTTPS_TRUST are two different anchors and are frequently confused. The first is CMP trust, which verifies the signature protecting the response message and the chain that comes back in it. The second is transport trust, which verifies the TLS server certificate and is used only on the HTTPS rows. A server may present a publicly trusted TLS certificate while issuing from a private CA, in which case only the first is needed. See HTTP and HTTPS transport.

Every value is written from the environment into a file and read back with --from-file, so no credential reaches a command line, and the endpoint URLs and the recipient are secrets rather than variables so a run against a private instance does not print them.

Lint and supply chain

make lint
make govulncheck
make gitleaks
make sbom
make go-patch-check
make scan-image IMG=<image>

Pull requests run lint, govulncheck and the credential scan. Pushes to dev and main additionally generate the SBOM and build and scan the container image. The weekly schedule repeats the full set so a newly disclosed vulnerability is found without a source change.

.gitleaks.toml extends the default rule set. It allowlists local/, bin/, dist/ and site/ by path, and it allowlists Go identifier names so that the generic credential rule does not report a camelCase or snake_case parameter such as enableRFC9483 that happens to sit next to a word like key. That allowlist is scoped to Go files and to the identifier shape alone, so a high-entropy literal in the same file is still reported. Add a new entry only for a confirmed false positive, and keep it narrow enough that a real credential in the same position would still fail the scan.

make go-patch-check reports whether go.mod still names the newest Go patch in the release series it targets and make go-patch-update moves it there. Standard library security fixes ship in patch releases, so a module left on an older patch has govulncheck report them even though no dependency changed. go-patch.yml runs the same check weekly and opens the bump as a pull request against dev. The release series is never advanced automatically, because a minor bump is a compatibility decision rather than a security one.

golangci-lint is built with the logcheck module plugin, which .custom-gcl.yml pins to an exact version so that a fresh run analyses the code with the same reviewed plugin as the last one. Editing that file rebuilds the custom binary on the next make lint.

make lint runs golangci-lint over the Go code and actionlint over .github/workflows. actionlint delegates run: blocks to shellcheck when it is on PATH, which it is on the GitHub-hosted runners, so install shellcheck locally to see the same findings CI does. codeql.yml additionally runs CodeQL static analysis over the Go code on pushes to dev and main and weekly. It skips without starting a runner while the repository is private, then reports under Security, Code scanning after the repository becomes public.