Your Agent's Bottleneck Is Almost Never the Model

David Crowe · July 10, 2026 · 3 min read

engineering cost reliability research

The Agent Optimization Manual · Introduction

Most conversations about making agents better are conversations about models. Which one reasons best, which is cheapest per token, which tier you need for “reliability.” We’ve been running a small research program — metering a few hundred real tool-calling agent runs, per call, and scoring what they actually delivered — and the recurring finding keeps landing somewhere else:

Your agent’s cost and reliability are mostly decided by the harness around the model, not the model itself. And those decisions are invisible unless you measure every call.

That’s not a slogan; it’s what the data kept saying, in ways that surprised us. Three examples, each its own post in this series, each with metered numbers you can check:

A cheap model that looked unreliable wasn’t. Gemini Flash delivered a result less than half the time and looked like a false economy. The traces showed the model was doing all the work and our loop was quitting on a single blank round. One small loop change took it to 100% delivery at the same quality as a model 14× more expensive per successful result. The reliability lived in the loop. → No. 3
One sentence in a system prompt more than doubled a bill. A live “you’ve spent $X” budget line — a guardrail — was silently disabling prompt caching for entire runs, and doing the most damage to the most expensive agents. Freezing that one line cut cost per run by more than half. The waste lived in the prompt construction. → No. 2
28% of runs delivered nothing, silently. Agents did the research and then ended on an empty text turn the loop mistook for “done” — no error, full cost, no result. Delivering through a tool call instead made the failure both rarer and visible. The reliability lived in how the loop defined “finished.” → No. 1

Notice the shape. In every case the model was fine. The money and the failures came from loop logic, prompt layout, and termination conditions — harness engineering — and in every case the fix was cheaper and higher-leverage than reaching for a bigger model.

Why this is hard to see

If harness decisions dominate agent economics, why isn’t everyone optimizing them? Because you can’t optimize what you can’t measure, and most agent stacks report the wrong thing. They tell you total tokens and whether the run finished. They don’t tell you:

cost per successful outcome (cost per run flatters expensive-but-reliable models and hides cheap-but-recoverable ones),
the cache read/write split (a blended token count hides a prompt that stopped caching),
whether “finished” meant “delivered” (an empty run and a successful one can look identical),
the per-call trace that tells you which round went wrong and why.

Every finding in this series came out of that gap — the space between “the run ended” and “the run delivered a good result for a defensible price.” You need per-call metered cost, a real delivery outcome, and the trace, all wired to the same place. That instrumentation is the whole reason we can do this research, and it’s the same per-action cost accounting that a control plane over your agents gives you for free. We’re not the subject of these posts. We’re the microscope.

What this series is

Measured, reproducible field notes on making agents cheaper and more reliable — the harness engineering, not the model shopping. Every number is metered at the gateway, not estimated. Where a finding is provider-specific, we say so; where it’s a general principle, we show why. The point isn’t any single trick. It’s the habit: before you pay 10× for a bigger model, measure whether your harness is the thing that’s actually costing you.

Start with No. 3 — Don’t Give Up on Cheap Models; it’s the one that changed how we’d architect an agent.

The Agent Optimization Manual — a running series. New drops when we have something measured worth publishing; subscribe below.

What is an Agentic Control Plane? →

Share: Twitter LinkedIn