What Is an Agentic Control Plane?

A few months ago I was building an MCP server to integrate an app into ChatGPT. It took me days just to get user authentication working. The LLM could call my backend, but my backend had no idea which user had initiated the request. There was no shared identity layer between the user, the model, and my code.

I kept thinking: this can’t just be my problem.

Turns out it isn’t. Every team that moves an AI project from pilot to production hits the same wall. And the solution isn’t a single library or one clever middleware. It’s an entire missing layer in the stack.

That layer is the Agentic Control Plane.

The Short Version

An Agentic Control Plane (ACP) is the trust, identity, and governance layer that sits between LLM runtimes — ChatGPT, Claude, Cursor, custom agents — and your backend services. It ensures that every AI-initiated action is:

• Identified — bound to a real, verified user
• Authorized — checked against your policies
• Safe — sensitive data is detected and redacted before reaching the model
• Constrained — rate limits and budget caps are enforced per user
• Routed — sent to the right backend with the user’s identity intact
• Auditable — logged with full context: identity, policy decisions, and cost

Think of it as the governance checkpoint between the AI and everything it can touch. Not the model layer. Not the app layer. The trust layer in between.

The Three-Party Problem

To understand why this layer needs to exist, you need to understand the fundamental architectural difference between traditional apps and AI apps.

Traditional web apps have two parties: a user and a backend. The user logs in, the backend verifies their identity, and every subsequent request carries that identity context. Authentication is solved. Access control follows naturally. We’ve been doing this for decades.

AI apps break this model. They introduce a third party: the LLM runtime.

Here’s what actually happens:

1. The user authenticates with the LLM (signs into ChatGPT, opens Claude, launches Cursor)
2. The user sends a prompt
3. The LLM decides it needs to call a tool — your API, your database, your internal service
4. The LLM calls your backend with… a shared API key

Your backend receives the request. But it has no idea who made it. The identity link between user and backend is severed. The LLM sits in the middle, holding one credential on each side, bridging neither.

I call this the Three-Party Problem. And it’s the root cause of nearly every governance failure in production AI systems.

The Parrot at the Bank

Imagine you have a really smart parrot. You send it to the bank to withdraw money from your account.

The parrot flies in and says: “I’d like to withdraw $200 from David’s account.”

But you can’t just trust a parrot. It could be lying. Confused. Intercepted on the way. The bank has no way to verify that this parrot is actually acting on your behalf.

The parrot is the LLM. Your backend is the bank. And right now, most AI systems are trusting the parrot.

The fix isn’t smarter parrots. It’s a security checkpoint at the front door of the bank — one that verifies a sealed, tamper-proof letter from a trusted authority before the parrot gets anywhere near a teller. The checkpoint verifies the seal, checks the rules, counts how many trips the parrot has already made today, stamps the request with the verified identity, and writes the whole transaction in an immutable ledger.

That checkpoint is the Agentic Control Plane.

Why This Matters Now

For the past two years, most AI integrations were conversational. Chatbots that answer questions, summarize documents, draft emails. The backend risk was limited because the AI wasn’t doing anything meaningful. It was reading and writing text.

That’s changing. Fast.

Three platform shifts are happening simultaneously — and none of them include a governance layer.

Protocol standardization. OpenAI’s Apps SDK lets ChatGPT call any HTTP endpoint on behalf of a user. Anthropic’s Model Context Protocol (MCP) gives LLMs a standard way to discover and invoke tools. The infrastructure for agents to take real actions now exists as an open standard.

Enterprise agent rollouts. Every large company is deploying internal copilots and agent workflows. These aren’t experiments anymore — they’re production systems that query patient records, create Jira tickets, process refunds, pull credit reports, and modify infrastructure. Real actions. Real consequences.

Regulatory pressure. HIPAA, SOC 2, GDPR, and PCI DSS all have direct implications for AI-mediated access to protected data. Regulators aren’t going to accept “we don’t know which user triggered that model call” as an answer. The era of “move fast and figure out compliance later” is ending.

The protocols are ready. The demand is here. The compliance requirements are real. The governance layer is the missing piece.

What Goes Wrong Without One

The consequences of deploying agents without a control plane are already visible across industries.

Shadow AI

Teams integrate AI tools without security review. Shared API keys get passed around Slack channels. Nobody knows which users are making which requests. When something goes wrong — a data breach, a compliance violation, an unexpected $15,000 invoice — there’s no way to trace it back to a person or a policy gap.

Can you believe what Oscar from accounting sent to ChatGPT? Neither can your security team — because they have no visibility into it.

Data Leakage

Without content filtering at the gateway, sensitive data flows into LLM prompts unchecked. Patient names, social security numbers, credit card numbers, legal case details — all sent to third-party models with no detection, no redaction, and no record of what was shared.

A user pastes a spreadsheet into ChatGPT. It contains customer SSNs. Without a control plane scanning content before it reaches the model, that data is now sitting in a third party’s infrastructure. You can’t un-send it.

No Audit Trail

A compliance officer asks: “Who accessed patient data through the AI assistant last Tuesday?”

Without identity binding and structured logging at the gateway, the honest answer is: “We don’t know.”

Generic API logs with no user attribution are useless for regulated workloads. You can’t answer data access requests, can’t prove compliance, can’t investigate incidents, and can’t satisfy an auditor. This is the kind of gap that fails SOC 2 reviews and triggers HIPAA investigations.

Runaway Costs

An agent loop fires 10,000 API calls in a minute. A single user burns through the team’s monthly LLM budget overnight. Without per-user rate limits and budget caps enforced at the gateway, there’s no guardrail until the invoice arrives.

And it’s not just loops. Agents retry failed calls. They spawn sub-agents that spawn more sub-agents. They hallucinate tool calls they shouldn’t make. Rate limiting by API key counts requests but doesn’t track cost — a single GPT-4 call can cost 100x more than a call to a cheaper model. By the time you notice, the damage is done.

Architecture of an Agentic Control Plane

An Agentic Control Plane implements six governance concerns. Each is composable. Each addresses one piece of the trust problem. Together, they form a pipeline that every AI-initiated request passes through before reaching your backend.

1. Identity — Who Is Calling?

Every request must be bound to a verified user. This means validating OAuth tokens (RS256 JWTs) against your identity provider’s JWKS endpoint and extracting the user’s identity — their sub claim, scopes, roles, and tenant context. The control plane normalizes this into a consistent user object that flows through the rest of the pipeline.

This is the foundation. Without verified identity, nothing else works. You can’t enforce policies if you don’t know who’s asking. You can’t attribute costs if you don’t know who spent them. You can’t audit actions if you don’t know who took them.

Identity isn’t a first-order design principle in most AI systems today. I view this as a fundamental architectural mistake.

2. Content Safety — Is the Data Clean?

Before a request reaches the model or your backend, the control plane inspects it for sensitive content. PII detection catches social security numbers, email addresses, credit card numbers, phone numbers, and other regulated data patterns. Depending on your policy, the control plane can redact, mask, flag, or block the request entirely.

This is the layer that prevents the “user pastes an SSN into ChatGPT” problem. And critically, it runs before the data leaves your perimeter — not after it’s already been sent to a third-party model.

3. Policy Enforcement — Is This Allowed?

A deny-by-default policy engine checks whether the authenticated user has permission to perform the requested action. Can this user call this tool? Access this model? Query this data source? Policies are defined by role, scope, or custom claims — the control plane enforces them consistently across every AI client and every model provider.

This is where you implement the principle of least privilege for AI interactions. Doctors can access patient records. Analysts can query financial data. Interns can use the general knowledge base. The policy layer draws those lines — and enforces them at the gateway, not scattered across your application code.

4. Usage Governance — Within Budget?

Per-user rate limits prevent runaway loops. Budget caps track cumulative spend and reject requests that would exceed the user’s allocation. Agent guard detects anomalous patterns — 500 identical tool calls in 30 seconds is almost certainly a loop, not a legitimate user request.

The key insight: rate limiting by request count isn’t enough. The control plane needs to track actual cost, not just volume. A pre-flight check before execution and a usage accounting step after the model responds gives you real budget enforcement, not just request counting.

5. Secure Routing — Where Does It Go?

The control plane routes the validated request to the appropriate backend with the user’s verified identity injected. Your backend receives x-user-uid — the verified sub claim — on every request. It never sees a shared API key. It always knows exactly which user the LLM is acting for.

Routing also includes SSRF protection (preventing the LLM from tricking the control plane into accessing internal services it shouldn’t reach), auth mode selection (forwarding user tokens vs. injecting service credentials), and scope enforcement per outbound destination.

6. Audit Trail — What Happened?

Every action is logged with structured metadata: who made the request, what tool was called, what policy decision was made, whether PII was detected, what it cost, whether it was approved or denied.

This isn’t generic HTTP logging. It’s a purpose-built audit stream for AI governance. When the compliance officer asks who accessed what through the AI assistant last Tuesday, you have the answer. One log, one format, one place to look — across every model and every client.

Inside the Identity Flow

This is the part that took me days to get right the first time. The identity flow is the most critical piece of the control plane — and the most misunderstood.

Here’s the end-to-end flow when a user triggers a tool call through ChatGPT or an MCP client:

What’s in the JWT

When the user authenticates with your identity provider (Auth0, Okta, Entra ID, Firebase — any OIDC provider), they receive an RS256-signed JWT. This is the sealed letter the parrot carries. Here’s what it looks like decoded:

{
  "header": {
    "alg": "RS256",
    "typ": "JWT",
    "kid": "NjVBRjY5MDlCMUIwNzU4RTA2QzZFMD..."
  },
  "payload": {
    "sub": "auth0|8f3a2b1c9d4e5f6a",
    "aud": "https://api.yourapp.com",
    "iss": "https://yourcompany.auth0.com/",
    "scope": "tool:crm:read tool:jira:write",
    "org_id": "org_acme_corp",
    "exp": 1739145600,
    "iat": 1739142000
  }
}

The critical claims:

• sub — the user’s unique identifier. This is how the control plane knows who is making the request. Not an API key. Not an email. A cryptographically verified user ID.
• aud — the intended audience. The control plane rejects tokens meant for a different service.
• scope — what the user is allowed to do. tool:crm:read means they can read from the CRM connector. tool:jira:write means they can create Jira tickets. No scope, no access.
• org_id — tenant isolation. Users in one organization can’t access another organization’s data.
• kid — the key ID used to sign the token. The control plane fetches the matching public key from the IdP’s JWKS endpoint and verifies the signature locally.

What Your Backend Receives

Without a control plane, your backend gets this:

POST /api/crm/contacts
Authorization: Bearer sk-shared-api-key-for-everyone
Content-Type: application/json

{"query": "show me all contacts in the pipeline"}

No user identity. No scopes. No way to filter results per user.

With a control plane, your backend gets this:

POST /api/crm/contacts
x-user-uid: auth0|8f3a2b1c9d4e5f6a
x-user-scope: tool:crm:read
x-user-org: org_acme_corp
x-request-id: req_7f8a9b0c
Content-Type: application/json

{"query": "show me all contacts in the pipeline"}

Verified identity. Scoped permissions. Tenant context. Full traceability. Your backend code can now do SELECT * FROM contacts WHERE org_id = req.headers['x-user-org'] and return only the data this specific user in this specific organization is allowed to see.

That’s the difference.

How an ACP Fits in Your Stack

An Agentic Control Plane occupies a distinct layer. It complements — not replaces — your existing infrastructure.

It’s not an LLM routing gateway. Tools like Portkey, LiteLLM, and OpenRouter focus on model selection and load balancing — routing prompts to the cheapest or fastest provider. An ACP doesn’t choose which model to use. It governs who can use it and what they’re allowed to do. LLM gateways optimize cost and latency. An ACP enforces trust and governance.

It’s not an agent framework. LangChain, CrewAI, and AutoGen help you build agents — defining tool chains, memory, and reasoning loops. An ACP doesn’t build agents. It governs them. The agent framework decides what to do. The ACP decides whether it’s allowed.

It’s not a traditional API gateway. Kong, Apigee, and AWS API Gateway handle HTTP traffic management — routing, TLS, basic rate limiting. They don’t understand the Three-Party Problem. They can verify a token, but they can’t bind LLM-forwarded requests to the originating user or produce identity-attributed audit trails for AI actions.

Use all of them. Your API gateway handles TLS and global routing. Your LLM gateway handles model selection and fallback. Your agent framework builds the workflows. Your Agentic Control Plane governs the trust boundary between the AI and your backend. Full comparison →

When You Need One

If any of these sound familiar, you need an Agentic Control Plane:

You’re moving an AI pilot to production. The demo worked with a shared API key and a single test user. Production needs per-user identity, access controls, and audit trails. This is where every team discovers the Three-Party Problem.

You’re in a regulated industry. Healthcare, financial services, legal, government — if you need to prove who accessed what data through an AI system, you need identity binding and structured audit logs. HIPAA, SOC 2, GDPR, and PCI DSS all require this, and “we’re working on it” doesn’t pass an audit. Compliance guide →

You’re deploying agents that take real actions. Not chatbots that answer questions — agents that create tickets, modify records, process transactions, and access sensitive data. Real actions need real governance.

You use multiple AI models. ChatGPT for one thing, Claude for another, Cursor for development. Governance that only covers one provider leaves the rest unmonitored. The control plane needs to be model-agnostic — governance follows the user, not the model. Why governance can’t live inside the model provider →

Your team is asking about cost controls. Someone’s LLM spend is out of control. Agent loops are burning budget. You need per-user rate limits and budget caps, not just a monthly API key limit shared across the whole org. The cost of running agents without budget controls →

Compliance has started asking questions. “Who accessed patient data through the AI last month?” “Can you prove that PII isn’t being sent to third-party models?” “Where’s the audit trail for AI actions?” If you can’t answer these today, a control plane is how you’ll answer them tomorrow.

What an Implementation Looks Like

The pattern is vendor-agnostic — you can implement an ACP in any language or framework. To make this concrete, here’s what the six-layer pipeline looks like as composable middleware:

import express from "express";
import { identifiabl } from "@gatewaystack/identifiabl";
import { transformabl } from "@gatewaystack/transformabl";
import { validatabl } from "@gatewaystack/validatabl";
import { limitabl } from "@gatewaystack/limitabl";
import { createProxyablRouter, configFromEnv } from "@gatewaystack/proxyabl";
import { explicablLoggingMiddleware, createConsoleLogger } from "@gatewaystack/explicabl";

const app = express();

// Identity — verify RS256 JWT, extract user
app.use(identifiabl({
  issuer: process.env.OAUTH_ISSUER!,
  audience: process.env.OAUTH_AUDIENCE!,
}));

// Content safety — detect and redact PII
app.use("/tools", transformabl({ blockThreshold: 80 }));

// Policy — deny-by-default authorization
app.use("/tools", validatabl({ requiredPermissions: ["tool:read"] }));

// Usage — per-user rate limits and budget caps
app.use("/tools", limitabl({
  rateLimit: { windowMs: 60_000, maxRequests: 100 },
  budget: { maxSpend: 500, periodMs: 86_400_000 },
}));

// Routing — forward to backends with identity intact
app.use("/tools", createProxyablRouter(configFromEnv(process.env)));

// Audit — structured logging of every request
app.use(explicablLoggingMiddleware(createConsoleLogger()));

app.listen(8080, () => console.log("ACP running on :8080"));

No request reaches your backend without a verified identity, a policy check, a PII scan, a budget check, and an audit log entry. The code above uses GatewayStack, an open-source (MIT) implementation of this pattern — but the architecture is what matters. The six layers, the pipeline order, the deny-by-default stance. Those hold regardless of implementation.

Getting Started

The fastest way to get an ACP running depends on where you are:

Start with identity. Install a JWT verification library, point it at your IdP’s JWKS endpoint, and require a verified user on every AI-initiated request. This alone eliminates the Three-Party Problem. Quickstart guide →

Go deeper on the architecture. The six-layer pipeline has design decisions at every layer — deployment topology, caching strategy, async audit patterns. Architecture deep dive →

Use a managed solution. If you’d rather not operate the infrastructure yourself, Agentic Control Plane Cloud is one option — a managed multi-tenant gateway with a dashboard for policy management and audit review.

The Bigger Picture

Every infrastructure category goes through the same evolution. Early cloud computing was monolithic — one provider, one stack, everything bundled. The market eventually separated into layers: compute, networking, observability, identity, security. Each layer became its own category with its own leaders.

AI infrastructure is following the same trajectory. Model providers will provide models. Agent frameworks will build agents. LLM gateways will handle routing and cost optimization. And governance will be its own layer — one that works across all of them, owned by the teams that deploy AI rather than the vendors that sell it.

The companies that build governance into their AI stack now will have a structural advantage over those that bolt it on after the first compliance failure or the first runaway invoice. The term “Agentic Control Plane” may be new. But the pattern it describes — trust, identity, and governance enforced at the infrastructure boundary — is as old as computing itself.

We’re just applying it to the newest layer of the stack.

Get started → · Architecture deep dive → · ACP vs API gateway → · View on GitHub →