An LLM Gateway Governs What the Model Says. Your Risk Is What the Agent Does.
LangChain published a good post this week: Building Governed Agents. If you run agents in production, read it. The framing is solid, the cost-visibility-first adoption path matches what we see, and it contains the best sentence anyone has written about this category:
“The greatest risk is often not what the model says, but what the agent can do.”
I agree with that sentence completely. I want to take it more seriously than a model-call gateway can.
Three kinds of agent action
Sort everything an agent does into three buckets:
- Model calls. The agent sends context to a provider, tokens come back. Cost lives here. So do prompt-injection inputs and PII in transit.
- Local actions. The harness executes something on the machine it runs on: a shell command, a file write, a git push, a browser click. No provider is involved. The bytes never leave the box.
- External effects. The agent hits another system — an API, an MCP server, another agent.
An LLM gateway — LangSmith’s, ours, anyone’s — sits on the wire between the agent and the model provider. That position is genuinely useful for bucket 1: spend limits, routing, redaction, token accounting. We run a proxy there ourselves and a third of our product depends on it.
But look at bucket 2 through that lens. When a coding agent runs rm -rf or force-pushes over your main branch, that action executes locally, inside the harness. It never transits a model-call gateway. The gateway sees the conversation about the action — the model’s tool-call output, maybe an echo of the result — after the harness has already run it. You cannot deny an action from a vantage point it doesn’t pass through.
And bucket 2 is not an edge case. Coding agents are the highest-volume production agents in most companies right now, and almost everything they do that can hurt you — deletes, pushes, config edits, package installs — is a local action.
Governing the doing
The enforcement point for local actions is the harness itself. Most harnesses expose hooks that fire before a tool call executes. Put a policy decision there — identity, tier, deterministic rules, deny/ask/allow — and you’re governing the action at the only moment it can actually be stopped. The same hook works across harnesses: Claude Code, Codex, Hermes, whatever your team actually runs. That matters because real orgs are multi-harness whether they planned to be or not.
Two properties fall out of enforcing at the action layer:
- The floor survives full-auto. A harness’s own yolo mode can’t switch off a server-side policy check. We surveyed eleven harnesses — almost nothing they ship survives their own permissive mode. A hook that phones a control plane does.
- Approvals become precise. You’re not approving “the agent may use tools.” You’re approving this delete, this push, with the arguments in front of you — and every approval can become a standing rule.
This week, while my coding agent was configuring governance for another agent, it tried to clean up a helper script with rm. The control plane held the delete and sent me an approval card. The agent setting up the cage got caged. That’s the property I want from this layer: it doesn’t know or care that the agent works for me.
Both layers, one plane
None of this argues against model-call gateways. It argues for completing the picture. The model side needs governing (spend, redaction, routing) and the action side needs governing (permissions, floors, approvals), and the audit trail is only trustworthy if both land in one place — the same identity, the same policy, the same log. Govern only the model wire and your cost story is solid while your rm -rf story is a hope. Govern only the hooks and you’re blind to spend.
I think of the control plane as owning the runtime call path — all of it, model calls and tool calls, across whichever harnesses the team runs. The gateway is one enforcement point on that plane. The hook is the other. The risk LangChain named lives mostly on the second one, and that’s the half most of the new governance offerings haven’t built.
Their post moves the category forward, and the category needs it. The sentence at the top deserves an architecture that can reach it.