An open, NIST-mapped benchmark for AI agent governance. 48 scenarios across 8 categories test what every governance layer must enforce: identity propagation, per-user policy, delegation provenance, scope inheritance, rate-limit cascade, audit completeness, fail-mode discipline, cross-tenant isolation.
We benchmarked CrewAI, LangChain/LangGraph, Claude Code, OpenAI Agents SDK, Anthropic Agent SDK, and Cursor — both natively and with ACP. Every result is reproducible against your own deployment.
Read in order, or skip to the framework or topic that matters most.
First post lands soon. Check back here or follow the blog.
Every AI governance vendor has a feature matrix. Whether the per-user policy actually enforces correctly when the user spawns ten subagents in parallel — that's a test, not a feature. A buyer who trusts feature matrices is buying marketing; a buyer who wants guarantees needs a benchmark.
AgentGovBench takes the same scenarios that procurement teams actually need (mapped to NIST AI RMF 1.0) and runs them against any governance layer you can write a runner for. The scenarios don't know what ACP is. ACP is one runner among several — vanilla (no governance), audit_only (logging without enforcement), acp (full enforcement), and now per-framework runners for every popular AI client and SDK.
The series unpacks each runner’s score, the structural reasons each framework scores where it does, and the specific failure modes worth knowing about — whether or not you use ACP.
Clone the repo, point at your ACP instance, run the benchmark. Same runner, same scenarios, same numbers. If you see different results, that’s either version drift or a gap we haven’t found.