Skip to main content

Migrate off Deployment Keys

Nothing will break. Existing deployment keys keep working exactly as they do today. There is no forced migration and no deadline. New self-managed deployments should prefer keyless, which means setting controlPlane.auth.k8sToken.enabled: true in the Helm chart. Keyless is off by default.

A deployment can have both a deployment key and a TrustDomainServerAttestation policy configured at once, so you can migrate at your own pace:

  • If either method works, the server is allowed to join. Your keyless configuration can be wrong at first while you debug it, with no outage.
  • If either method fails, the failure is logged at warn level, so you can tell whether it is yet safe to remove the key. On Defakto-monitored deployments, falling back to the key escalates to error level.

Starting Point​

Assume a Trust Domain Server running today with a deployment key:

trustDomainDeployment:
  id: tdd-t6rml5zfg5
  trustDomainName: "on-prem.example.com"
  trustDomainID: "td-ro6d997e6n"
  name: "us-west-2"
controlPlane:
  auth:
    key:
      id: "tdk-pfze0ki4lt"
      pem: |
        -----BEGIN PRIVATE KEY-----
        MC4CAQAwBQYDK2VwBCIEIAJpHMcC1rJhcI0fPHTRoFGlazXYI0K/aIL3UkVe0p0w
        -----END PRIVATE KEY-----

Step 1 — Register the allowed identity​

Register the TrustDomainServerAttestation policy with the Control Plane using spirlctl or Terraform. See Keyless Server Authentication for the full document and how to find each value.

# Check the document first. Nothing is stored.
spirlctl config validate trust-domain-deployment \
  --id tdd-t6rml5zfg5 trust-domain-server-attestation.yaml

spirlctl config set trust-domain-deployment \
  --id tdd-t6rml5zfg5 trust-domain-server-attestation.yaml

# Read back what is stored.
spirlctl config get trust-domain-deployment \
  --id tdd-t6rml5zfg5

This leaves the Control Plane ready to accept the new method. The server has not changed yet.

The policy usually reaches the Control Plane's authentication path within about 30 seconds, and can take up to five minutes if the change misses a sync cycle. Register it before rolling out Step 2.

Step 2 — Turn on keyless, keeping the key​

Add k8sToken alongside the existing key:

trustDomainDeployment:
  id: tdd-t6rml5zfg5
  trustDomainName: "on-prem.example.com"
  trustDomainID: "td-ro6d997e6n"
  name: "us-west-2"
controlPlane:
  auth:
    key:
      id: "tdk-pfze0ki4lt"
      pem: |
        -----BEGIN PRIVATE KEY-----
        MC4CAQAwBQYDK2VwBCIEIAJpHMcC1rJhcI0fPHTRoFGlazXYI0K/aIL3UkVe0p0w
        -----END PRIVATE KEY-----
    k8sToken: # <--- this is new
      enabled: true

Roll out the new values. The Helm release name is the deployment ID, and the chart names the Kubernetes Deployment after it:

# Apply the values change.
helm upgrade tdd-t6rml5zfg5 oci://ghcr.io/spirl/charts/spirl-server \
  -n tdd-t6rml5zfg5-example -f values.yaml

# Watch the roll, which the new flag triggers automatically.
kubectl -n tdd-t6rml5zfg5-example rollout status deploy/tdd-t6rml5zfg5-spirl-server

The rollout can take a few minutes to report success.

Step 3 — Confirm keyless is working​

Every login attempt names the credential it used in the authMethod field, so look for k8s_token in the server logs:

kubectl -n tdd-t6rml5zfg5-example logs deploy/tdd-t6rml5zfg5-spirl-server \
  | grep -E "Authentication succeeded|Authentication failed|fallback auth method"

Failure:

{"level":"warn","msg":"Authentication failed","authMethod":"k8s_token","error":"<reason>"}

Success:

{"level":"info","msg":"Authentication succeeded","authMethod":"k8s_token"}

While the deployment key is still configured, a rejected token does not take the server down. The server falls back to the key and logs Established session with a fallback auth method.

The signal that it is safe to remove the key is the absence of Authentication failed for authMethod=k8s_token, alongside the success line.

The session metric gives the same answer as a count. A non-zero k8s_token value means the login was authenticated with the Kubernetes token:

kubectl -n tdd-t6rml5zfg5-example port-forward deploy/tdd-t6rml5zfg5-spirl-server 19090:9090

# In a second terminal:
curl -s localhost:19090/metrics | grep spirl_session
spirl_session_established_total{auth_method="k8s_token"} 1

Metrics have to be enabled on the deployment. See Trust Domain Server Metrics.

Step 4 — Remove the key​

Once success is confirmed, drop the key from the Helm values:

trustDomainDeployment:
  id: tdd-t6rml5zfg5
  trustDomainName: "on-prem.example.com"
  trustDomainID: "td-ro6d997e6n"
  name: "us-west-2"
controlPlane:
  auth:
    k8sToken:
      enabled: true

Re-run the helm and rollout commands from Step 2 to apply the change.

Then retire the key in the Control Plane:

# Find the key ID if you do not have it.
spirlctl trust-domain deployment key list --trust-domain on-prem.example.com

# Reversible off-switch. The Control Plane stops accepting this key for auth.
spirlctl trust-domain deployment key disable tdk-pfze0ki4lt \
  --trust-domain on-prem.example.com

# Permanent removal.
spirlctl trust-domain deployment key delete tdk-pfze0ki4lt \
  --trust-domain on-prem.example.com

You can confirm the deployment's stored attestation policy in the Defakto Console under Trust Domains → your Trust Domain → Settings → Deployment Configurations → the deployment → Server Attestation.

Rolling Back​

If anything goes wrong mid-migration, redeploy with the deployment key. The key stays valid until you remove it.