Gateway overview

The gateway is the only door into and out of the appliance. It runs as a co-located Go process on the appliance chassis, terminates TLS, authenticates virtual API keys, evaluates OPA policy, applies rate limits, and emits audit events before a single token is generated.

Request lifecycle

1. TLS terminate. A per-appliance certificate signed by the fleet root.
2. Key lookup. The virtual-key prefix identifies the tenant, room, and allowed models. The hashed secret authenticates the caller.
3. OPA decision. The gateway fetches the latest signed policy bundle and evaluates the request against it. A deny is a 403.
4. Rate check. Token-bucket counters enforce per-key and per-tenant budgets.
5. Audit emit. A pre-request event is queued. If the request is denied, we emit the denial and stop.
6. Route. The request is forwarded to the Workspace Runtime on localhost.
7. Stream. Response tokens are streamed back to the caller while being tapped into the audit tree.
8. Post-audit. A post-response event captures size, latency, and any policy annotations.

What’s in audit

Only metadata: the key fingerprint, the model ID, counts, latency, the OPA decision, and the Merkle leaf of the request body. Bodies themselves stay in the appliance-side store with per-tenant encryption.

Why this split

It matters that the audit writer, the policy engine, and the inference engine are in different processes with different users and different capabilities. If the inference runtime is ever exploited, the attacker still has to cross a unix-socket boundary to get anywhere near the gateway or audit state.