Using OPA to enforce AI policy at the gateway layer

AI governance conversations usually end with the same unsatisfying conclusion: “We need policies.” Everyone agrees. Nobody specifies what that means in code. The result is governance-by-documentation — PDF policies that developers read once and forget, enforced by hope and quarterly audits. Using OPA to enforce AI policy at the gateway layer turns those PDFs into executable rules that run on every request.

Open Policy Agent is already the standard for infrastructure policy in Kubernetes, service meshes, and cloud IAM. Applying it to AI gateways is a natural extension — and one that finally makes AI governance a technical control rather than a human one.

Why policy belongs at the gateway

AI requests flow through a gateway before they reach the model. The gateway authenticates the caller, selects the model, assembles context, and forwards the request. This is the natural enforcement point for policy because it sees every request, has access to caller identity, and can reject or modify requests before they consume inference resources.

Enforcing policy in application code does not work at scale. Every application team implements its own checks, with its own interpretation of the rules, tested to its own standard. Gaps are inevitable. A centralised OPA policy AI gateway ensures that every request, from every application, is evaluated against the same rule set.

The gateway also has access to metadata that application code typically does not: the caller’s team, budget allocation, data classification level, and historical usage patterns. OPA policies can reference all of this context when making decisions.

How OPA evaluates AI requests

OPA evaluates policies written in Rego, a declarative language designed for policy decisions. A typical AI gateway integration sends OPA a JSON input document containing the request metadata — caller identity, target model, prompt hash, token estimate, data classification — and receives a decision: allow, deny, or allow-with-modifications.

A simple example. Your organisation has a policy that the 70B parameter model is reserved for production workloads. In Rego:

deny["model not authorised for this environment"] {
    input.model == "llama-70b"
    input.environment != "production"
}

A more nuanced example. Requests containing data classified as “confidential” must use on-premise models only:

deny["confidential data must use local models"] {
    input.data_classification == "confidential"
    not input.model_location == "local"
}

These policies are version-controlled, tested, reviewed, and deployed through the same CI/CD pipeline as any other infrastructure configuration. When the auditor asks “what is your AI usage policy?”, you show them the Rego files and the git history, not a PDF.

Policy composition for AI workloads

Real AI governance requires composing multiple policies. A single request might need to pass model-access rules, data-classification rules, rate-limit rules, content-safety rules, and budget rules. OPA handles this naturally through its package system — each concern lives in its own policy package, and the gateway evaluates all of them.

The composition also enables graduated enforcement. Instead of a binary allow/deny, the OPA policy AI gateway can return a decision object that instructs the gateway to downgrade the model (use the 7B instead of the 70B), strip certain context fields, add a monitoring flag, or require human approval before forwarding.

This is where OPA’s declarative model shines. Adding a new policy concern — say, a new regulation requires logging the justification for using a large model — means adding a new Rego file, not modifying application code across twenty services.

Audit and compliance benefits

Every OPA decision can be logged as a structured event: the input, the decision, the policies that contributed to it, and the policy version. This creates an audit trail that shows not just what happened, but why — which policy allowed or denied each request, and what version of that policy was active at the time.

For regulated industries, this is transformative. Compliance is no longer about proving that you have a policy. It is about proving that the policy was enforced on every request, with evidence that cannot be retroactively altered.

Where Operayde fits

Operayde’s gateway integrates policy evaluation into the request path. Policies are declarative, version-controlled, and enforced on every inference request before the model processes a single token. Policy decisions are captured in the Merkle-signed audit trail alongside the request and response. The entire stack — gateway, policy engine, model, audit log — runs on-premise, so policy enforcement does not depend on connectivity to an external service.