Gateway-to-CP mTLS
For: Platform engineers and security teams operating self-hosted gateways.
Security Overview | mTLS | Bundle Signing | Credential Encryption
Axemere Gateway uses mutual TLS (mTLS) to authenticate self-hosted gateways to the Axemere Control Plane. Every gRPC call from a gateway to the CP carries a client certificate; the CP rejects connections from any node that cannot present a valid cert issued by the Axemere CA.
Managed Gateway: mTLS is handled automatically for Axemere-hosted gateways. This guide applies to self-hosted deployments only.
Table of Contents
- How It Works
- Enrollment Flow
- Certificate Lifecycle
- PKI Architecture
- Revocation
- Key Security Properties
- Environment Variables
- Troubleshooting
- See Also
How It Works
The private key is generated on the gateway and never transmitted. The control plane receives only a Certificate Signing Request (CSR) and returns a signed certificate.
Enrollment Flow
Enrollment happens automatically on first startup when MVGC_BOOTSTRAP_TOKEN is set.
Step 1 — Generate a bootstrap token in the Console
- Sign in at console.axemere.ai
- Go to Gateways in the left sidebar
- Click Enroll Node → Generate Enrollment Token
- Choose an expiry (default: 24 hours)
- Copy the token; it is shown only once
Step 2 — Start the gateway with the token
Via environment variables:
export MVGC_BOOTSTRAP_TOKEN="<token from console>" export MVGC_CP_ADDR="us.cp.axemere.ai:9090" export MVGC_ORG_ID="<your org_id>" export MVGC_KEY_DIR="/var/lib/mvgc/keys" ./mvgc-gateway
Or equivalently in mvgc.yaml (see Config File):
control_plane: addr: "us.cp.axemere.ai:9090" bootstrap_token: "<token from console>" gateway: org_id: "<your org_id>" security: key_dir: "/var/lib/mvgc/keys"
bootstrap_token is consumed on first successful enrollment and cleared automatically; it does not need to be removed from the config file manually.
Step 3 — Verify enrollment
curl -s http://localhost:7080/healthz | jq '{cp_status, node_id}' # → {"cp_status":"connected","node_id":"gw-node-prod-01"}
cp_status must be "connected". If it shows "unconfigured" or "offline", check the environment variables and that the bootstrap token has not expired.
Certificate Lifecycle
Initial Enrollment
- Certificate validity: 90 days
- Algorithm: ECDSA P-256
- Subject CN: the node ID. Org ID and node ID are also embedded in a SPIFFE URI SAN
(
spiffe://axemere.ai/org/<org_id>/gateway/<node_id>) and a DNS SAN, so the CP can identify the org and node a connection belongs to without a separate lookup.
Automatic Renewal
The gateway monitors its certificate expiry. When fewer than 14 days remain:
- A new key pair + CSR is generated
- A renewal request is sent to the CP, authenticated with the current cert
- The CP issues a new 90-day cert
- The old cert remains valid until it expires to avoid disruption during rollout
No operator action required.
Manual Renewal
To renew early (e.g., after a security event):
- Generate a new bootstrap token in the console
- Set
MVGC_BOOTSTRAP_TOKENto the new token - Restart the gateway; enrollment runs again and issues a new cert
PKI Architecture
| Level | Key type | Storage | Validity |
|---|---|---|---|
| Root CA | ECDSA P-256 | GCP Certificate Authority Service, HSM-backed; signing access restricted to a dedicated human-custody group, never to Axemere's own service infrastructure | 10 years |
| Subordinate CA | ECDSA P-256 | GCP Certificate Authority Service, HSM-backed | 3 years |
| Gateway node cert | ECDSA P-256 | MVGC_KEY_DIR on gateway | 90 days |
Axemere's control plane uses a genuine two-tier PKI via GCP Certificate Authority Service (CAS). The root CA's signing authority is held only by a dedicated human-custody group, not by any Axemere service or infrastructure, so a compromised control plane pod has no path to it. Day-to-day issuance is handled entirely by the subordinate CA, whose private key never leaves GCP's HSM-backed infrastructure at all: the control plane submits a certificate signing request and receives a signed certificate back, without ever touching key material, not even as an encrypted blob. If the subordinate's issuance capability were ever compromised, a fresh subordinate can be minted from the still-untouched root without redistributing trust across the gateway fleet. The same subordinate also issues shorter-lived certificates (1 year) for Axemere's own internal admin services, a separate use case from gateway enrollment.
Migrating a gateway onto this PKI is a dual-trust transition, not a forced cutover: whatever certificate authority a gateway already trusts stays valid alongside the new root, so existing certificates keep authenticating normally and move onto the new chain the next time they renew.
Revocation
To revoke a gateway certificate (e.g., decommissioning a node or suspending an org):
From the Console:
- Go to Gateways in the left sidebar
- Find the node and click Revoke Certificate
After revocation, the gateway's next gRPC call will be rejected with codes.PermissionDenied. The gateway falls back to cached policy and continues serving requests from cache until the cache TTL expires or the gateway is restarted.
Key Security Properties
| Property | Guarantee |
|---|---|
| Private key confidentiality | Private key generated on gateway, never transmitted |
| CP CA key custody | Root and subordinate signing keys held exclusively in GCP CAS's HSM-backed infrastructure; the control plane never receives or holds key material, only signed certificates |
| CP authentication | Only certs signed by Axemere CA accepted |
| Org isolation | Org identity bound to certificate, cannot be spoofed at the application layer |
| Revocation | Certificate immediately rejected on next connection |
| Forward secrecy | TLS 1.3 with ECDHE, session keys not derived from cert key |
Environment Variables
| Variable | Default | Description |
|---|---|---|
MVGC_BOOTSTRAP_TOKEN | — | Single-use enrollment token from console. Used only on first startup. |
MVGC_CP_ADDR | — | Control plane gRPC address, e.g. us.cp.axemere.ai:9090 |
MVGC_KEY_DIR | ./keys | Directory where the gateway stores its cert, private key, and CA bundle |
MVGC_CP_CA_CERT | — | Override: path to CA cert file (skips auto-provisioned CA bundle) |
MVGC_NODE_CERT | — | Override: path to node cert file (skips auto-provisioned cert) |
MVGC_NODE_CERT_KEY | — | Override: path to node private key file |
Setting MVGC_CP_CA_CERT, MVGC_NODE_CERT, and MVGC_NODE_CERT_KEY together disables automatic enrollment and uses the provided files instead.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
cp_status: "unconfigured" | MVGC_CP_ADDR not set or MVGC_BOOTSTRAP_TOKEN missing | Set both env vars and restart |
cp_status: "offline" after enrollment | Network blocked or cert validation failing | Check firewall rules on port 9090; inspect gateway logs for TLS errors |
codes.PermissionDenied on gRPC | Certificate revoked or org suspended | Generate a new bootstrap token; check org status in console |
cert expiry warning in logs | Certificate within 14-day renewal window | Renewal is automatic; check that CP is reachable |
Gateway stuck in "enrolling" | Bootstrap token expired | Generate a new token and restart |