Session X-Ray: Debugging a Single Agent Run, Call by Call
In the companion to this post, we looked at an agent as a distribution — every run laid over every other, so you can see the spread, the tail, and the path it usually takes. That’s the view you use to decide which runs are worth opening.
This is about opening one.
A single run came back at 4.0¢ and took 11 seconds. Another identical request cost 1.2¢ and took 4. You don’t redesign the agent off an average — you open the expensive run and find the step that did it. That’s the Session X-ray: one execution — every call your harness made, in the order it happened — with the cost and latency of each, and the inputs and outputs of any step you want to inspect.
The whole run, in order
The first thing the Session X-ray gives you is the waterfall — every model call and tool call the run made, top to bottom, with how long each took and what it cost:
llm.synthesize — is both the longest bar and the biggest charge. The three searches are nearly free. You found 80% of the problem in one row.From the outside, the run is a black box that took 11 seconds. Inside, 64% of that time and over half the cost is one synthesis call re-reading everything the three searches returned. The searches — the “tool calls,” the external stuff you’d have suspected — are rounding error.
Three questions, three sorts
Once the whole run is in front of you, debugging is three questions, each a column you sort by.
Cost. Sort by it. The top row is almost always a model call inside the loop, not a tool. Here it’s llm.synthesize at 2.1¢ — the step that ingests the most context. That’s the loop tax made concrete: the bill isn’t the work, it’s the re-reading before the work.
Latency. Sort by it. Sometimes it’s the same step; often it isn’t. A slow run can be one model call that stalled, or a tool waiting on a flaky upstream. The waterfall tells you which, instead of you guessing.
Failures and blocks. A run that errored has a step where it errored. A run that got a tool denied by policy has the deny, in place, with the reason. You don’t reconstruct the failure from logs scattered across three services — you see it where it happened, in the sequence that led to it.
Then you read the step
Finding the expensive step is half of it. The other half is understanding why — and for that you open the call itself. The Session X-ray lets you inspect the redacted input and output of any step: what llm.synthesize was actually handed, and what it produced.
Maybe synthesis is being handed all three full search payloads when it only needed the top result from each — so you trim what feeds it. Maybe it’s running on a frontier model when the task is extraction — so you route that one step down a tier. Maybe the same context is re-sent on every loop turn unchanged — so you cache the stable prefix. You can’t make any of those calls from a cost total. You make them from the call’s actual inputs.
Why “per call” is the unit
Every one of those moves is a change to one call, justified by one call’s data. Not “the agent is expensive” — llm.synthesize on this run was handed 9,000 tokens it didn’t need. Not “it’s slow sometimes” — this step stalled for 7 seconds on attempt two. The Session X-ray works because it never abstracts away the atom. A run is its calls, in order, each with a cost, a latency, an input, an output, and an outcome.
The Agent X-ray tells you which run to open. The Session X-ray tells you what to change inside it.
It’s free for individuals — one install, no code changes, and every call is captured. See your first run in about thirty seconds →