Agentic Data Plane vs Agentic Control Plane
If you’ve spent time in networking or cloud infrastructure, the split between “control plane” and “data plane” is second nature. The control plane decides what should happen. The data plane makes it happen.
AI agent infrastructure is developing the same split — but most teams don’t realize it yet. They’re building both planes in the same codebase, tangled together, and wondering why governance is so hard to enforce.
What is an agentic data plane?
The agentic data plane is the execution layer. It’s everything involved in actually doing the work an AI agent is asked to do: sending prompts to models, receiving completions, calling tools, reading from databases, writing to APIs, and returning results to the user.
If an agent creates a Jira ticket, the data plane is the HTTP request to Jira’s API. If an agent queries a CRM, the data plane is the SQL query and the result set. If an agent routes a prompt to Claude instead of GPT-4, the data plane is the model call itself.
Concretely, the agentic data plane includes:
- LLM routing — selecting a model provider, managing failover, optimizing for cost or latency
- Tool execution — calling APIs, reading files, querying databases on behalf of the user
- Context management — maintaining conversation history, RAG retrieval, memory
- Response streaming — delivering model output back to the client
- Orchestration — multi-step agent workflows, chain-of-thought, sub-agent coordination
Products in the data plane category include LLM gateways (Portkey, LiteLLM, OpenRouter), agent frameworks (LangChain, CrewAI, AutoGen), and MCP servers that expose tool capabilities. These tools decide what to do and how to do it.
What they don’t decide is whether it should be allowed.
What is an agentic control plane?
The agentic control plane is the governance layer. It doesn’t execute actions — it decides whether actions are permitted, by whom, under what conditions, and records what happened.
Before any tool call reaches your backend, the control plane answers five questions:
- Who is making this request? — JWT verification, identity binding to a real user
- Are they allowed to do this? — role-based policy enforcement, deny-by-default
- Is the data safe? — PII detection and redaction before content reaches the model
- Are they within budget? — per-user rate limits and spend caps
- What happened? — structured, identity-attributed audit logging
The control plane doesn’t know or care which model you’re using, which agent framework runs your workflows, or whether you’re using MCP or a custom integration. It operates at the trust boundary between the AI layer and your backend — enforcing governance regardless of the data plane implementation beneath it.
Why the distinction matters
In traditional infrastructure, nobody argues about whether you need both planes. A network switch (data plane) and a network controller (control plane) do fundamentally different jobs. Combining them creates systems that are hard to secure, hard to audit, and hard to scale.
The same is true for AI infrastructure, but teams keep making this mistake.
Governance scattered across application code
Without a dedicated control plane, every team implements governance differently. One service checks permissions in a middleware function. Another validates tokens in the route handler. A third relies on the API gateway to handle auth but doesn’t propagate user identity through the LLM layer. A fourth doesn’t check permissions at all — it was a hackathon prototype that somehow made it to production.
The result: inconsistent enforcement, gaps in audit trails, and no single place to answer “who accessed what through which AI agent?”
Data plane tools can’t solve control plane problems
LLM gateways optimize model selection and cost. They don’t verify that the user behind the prompt is allowed to access the tool the model wants to call. Agent frameworks orchestrate multi-step workflows. They don’t enforce per-user budget caps or redact PII before it leaves your perimeter.
This isn’t a criticism of those tools — they’re good at what they do. The problem is expecting them to also be a governance layer. A load balancer isn’t a firewall. An LLM router isn’t an identity provider.
Compliance requires separation
HIPAA, SOC 2, GDPR, and PCI DSS all require demonstrable access controls, audit trails, and data protection measures. Auditors want to see a governance layer that is architecturally distinct from the execution layer — not governance logic buried inside the same code that processes requests.
When your compliance evidence comes from a dedicated control plane with structured logs, policy definitions, and identity verification, the audit conversation is straightforward. When it comes from scattered if statements in application code, you’re explaining why your governance is trustworthy despite having no separation of concerns.
How they work together
The best architecture runs both planes, each doing what it does best.
Here’s the request flow:
- User sends a prompt through ChatGPT, Claude, Cursor, or a custom agent
- The LLM decides it needs to call a tool — your API, your database, an internal service
- The data plane routes the tool call toward your backend
- The control plane intercepts — verifies the user’s identity, checks policies, scans for PII, enforces rate limits
- If approved, the request proceeds to your backend with verified identity headers
- The data plane returns the result to the model
- The control plane logs the entire transaction with full attribution
The data plane handles steps 1-3 and 6. The control plane handles steps 4-5 and 7. Neither does the other’s job.
User → LLM → Data Plane → Control Plane → Backend
↓
Identity ✓
Policy ✓
PII scan ✓
Budget ✓
Audit ✓
This separation means you can swap out your LLM gateway without touching governance. You can change agent frameworks without rewriting policy logic. You can add a new model provider and governance follows automatically — because it’s enforced at the trust boundary, not inside the execution layer.
Where to start
If you’re building AI agent infrastructure today, start by identifying which parts of your stack are data plane and which are control plane. Most teams find that they have a sophisticated data plane (model routing, agent orchestration, tool execution) and almost no control plane (shared API keys, no per-user audit trails, no PII detection).
The fix isn’t rebuilding your data plane. It’s adding the missing governance layer alongside it.
An Agentic Control Plane gives you that layer — identity, policy, content safety, usage limits, and audit — enforced at the gateway, independent of your data plane choices.
Get started → · Reference architecture → · Pricing →