# Security and enrollment
Provisioner talks to servers that have no credentials yet. It checks every value it accepts from a
node before using it, and signs every credential it returns with keys that the node cannot reach.
{{% notice style="note" title="Where it lives" %}}
`host`: `host.go` holds `validateJWT`, `validateCSR`, `generateServerJWT`, and `encryptPrivateKey`.
`rpc.go` holds `fetchJWT`, `fetchCSR`, and `fetchEd25519PubKey`. Key material paths come from
`config.Config`.
{{% /notice %}}
## Enrollment models
`features.pki` and `features.ed25519` select mutually exclusive models, and `config.Load` rejects a
configuration that sets both. `features.jwt` is independent, and enabling `features.ed25519` turns it
on implicitly.
| Feature | What Provisioner fetches | What it gives back |
|---------|--------------------------|--------------------|
| `jwt` | The node's `provisioning.jwt`. | The node already holds this token, so Provisioner returns none. |
| `pki` | A CSR generated by the node. | A signed certificate and CA from the helper. |
| `ed25519` | The node's ed25519 public key, with a signed nonce. | A server JWT signed by Provisioner. |
## Provisioning token verification
Runs when `features.jwt` is set. Setting `features.ed25519` sets it too, so this step also runs in
Organization Issuer deployments.
`validateJWT` (`host/host.go:405`) refuses to run without `jwt_verify_cert`. `os.Stat` decides whether
that value is a key file path or a hex ed25519 public key:
```go
if _, err = os.Stat(h.cfg.JWTVerifyCert); os.IsNotExist(err) {
pk, err = hex.DecodeString(h.cfg.JWTVerifyCert)
claims, err = tokens.ParseProvisioningToken(h.rawJWT, ed25519.PublicKey(pk))
} else {
claims, err = tokens.ParseProvisioningTokenWithKeyfile(h.rawJWT, h.cfg.JWTVerifyCert)
}
```
A path that exists on disk is read as a key file. Anything else is decoded as a hex ed25519 public
key, the form of an Organization Issuer public key. `validateJWT` stores the parsed claims on
`h.JWT`, and `getConfig` sends them to the helper as the verified `jwt` field.
## The ed25519 challenge
Runs when `features.ed25519` is set. `features.pki` deployments skip it, and `config.Load` rejects a
configuration that sets both.
`fetchEd25519PubKey` (`host/rpc.go:228`) generates a nonce
with `choria.NewRequestID()`, sends it in the `gen25519` request, and verifies the node's signature
over it:
```go
if !ed25519.Verify(pk, []byte(h.nonce), sig) {
err = fmt.Errorf("invalid nonce signature")
return
}
```
Without the nonce, a node could present any public key, including one belonging to a different
machine, and receive a server JWT bound to that key. The signature proves the node holds the matching
seed. The call also records the directory that the node created the seed in, which becomes the SSL
directory in the generated configuration.
## Server token issuance
Runs when `features.ed25519` is set, immediately after the helper returns. It also needs
`jwt_signing_key`, and `server_jwt_validity` or its one year default.
`generateServerJWT` (`host/host.go:251`) builds the claims in increasing order of precedence:
built-in defaults, the configuration that the helper returned, and the helper's explicit
`server_claims`.
org
Defaults to choria. Overridden by server_claims.ou.
collectives
Defaults to mcollective. Becomes choria when the returned configuration sets plugin.security.provider to choria, then a comma-split of the collectives configuration key, then server_claims.collectives.
validity
Defaults to server_jwt_validity, or one year. A server_claims expiry is converted with time.Until.
permissions
Null unless server_claims.permissions is set.
additional publish subjects
Empty unless server_claims.additional_publish_subjects is set.
A computed validity of one hour or less is discarded and replaced with the configured default, which
covers both an expiry already in the past and a claim that failed to parse.
{{% notice style="warning" title="Signing files are re-read for every server" %}}
When `jwt_signing_token` is set, `generateServerJWT` reads the signing token and the seed file from
disk for every server. The code does this deliberately: "a bunch of redundant repeated reading happens
here of the same files but I prefer to do that so just updating the secrets will update the running
instance". Rotating the issuer credentials takes effect on the next server rather than on the next
restart. Provisioning a thousand servers therefore performs two thousand small file reads.
{{% /notice %}}
With a signing token present, the token must carry a `TrustChainSignature` claim, and
`claims.AddChainIssuerData` attaches the chain before signing. `tokens.SignTokenWithKeyFile` signs the
result, and the `configure` call sends it to the node as `server_jwt`.
## CSR checks
Runs when `features.pki` is set. `features.ed25519` deployments skip it, because no x509 certificate
is issued.
`validateCSR` (`host/host.go:435`) parses the PEM that the node sent. The common name must equal the
identity that Provisioner discovered, and no name on the request may match the denylist. `validateCSR`
checks the common name and every DNS SAN.
The default denylist blocks the certificate names that Choria reserves for privileged clients:
```yaml
cert_deny_list:
- "\\.privileged.mcollective$"
- "\\.privileged.choria$"
- "\\.mcollective$"
- "\\.choria$"
```
`matchAnyRegex` (`host/host.go:468`) treats a pattern wrapped in forward slashes as a bare regular
expression and strips them, then matches with `regexp.MatchString`. Matching is unanchored, so a
pattern without `^` or `$` matches anywhere in the name.
{{% notice style="warning" title="Replacing cert_deny_list replaces the defaults" %}}
A node in provisioning mode is unauthenticated, so `validateCSR` is the only check on the name it
receives, and that name can request a certificate with privileged access to the whole fleet. Setting
`cert_deny_list` replaces the defaults rather than adding to them, so a custom list must repeat these
four patterns.
{{% /notice %}}
## Private key protection in transit
Runs when the helper returns a `key`, which no feature flag controls. It also needs `features.jwt`,
because the server's ECDH public key arrives on the `jwt` reply.
Some certificate authorities generate the key pair themselves rather than signing a CSR. For those,
the helper can return a `key`, and it has to cross the network to a node that holds no credentials.
`encryptPrivateKey` (`host/host.go:354`) uses an ephemeral Diffie-Hellman exchange over the two RPC
calls that already happen.
Each secret covers one key on one node, and neither side writes it to disk.
The server's ECDH public key arrives on the `jwt` reply as `EcdhPublic`, so key encryption requires
`features.jwt`. Without it `encryptPrivateKey` fails with "private key received from helper but server
did not start Diffie-Hellman exchange". `encryptPrivateKey` stores Provisioner's own public key on
`h.provisionPubKey`, and the `configure` call sends it as `EcdhPublic`, so the node can derive the
same secret and decrypt the PEM block.
`x509.EncryptPEMBlock` is deprecated in the standard library and the call carries a
`//lint:ignore SA1019 there is no alternative` comment. The format is fixed by what Choria Server can
decrypt.
{{% notice style="tip" title="Next" %}}
[RPC, retries, and upgrades]({{% relref "rpc-and-upgrades" %}}) covers the wrapper that every one of
these calls runs through.
{{% /notice %}}