Agent Cost Drift in the Wild: A Model Alias Zeroed Our Cache
Our ops agent runs a tool loop on Gemini Flash, governed and metered through our own proxy. Yesterday the metering showed something that shouldn’t be possible: 216 calls, about 13M input tokens, 0% implicit cache. A tool loop re-reads its whole history every turn — it’s the most cacheable workload there is. Cache rates on loops like this normally sit at 70–90%. Zero isn’t a bad rate; zero is a signal.
We went looking for the bug in our request shape, and found two real ones. Per-call-unique tool-call ids were leaking into replayed history, making every request byte-unique — that genuinely kills prefix caching, and we fixed it. We were also on Google’s deprecated SDK, so we migrated to @google/genai and moved functionResponse to the current API contract. Both fixes were correct. Neither moved the number. Still 0%.
So we stopped guessing and isolated the variable. One probe: a byte-identical 8,810-token prefix, sent repeatedly, back to back, same API key, same SDK — only the model name changes. Verbatim output:
gemini-flash-latest #1: cached=0/8810
gemini-flash-latest #2: cached=0/8810
gemini-flash-latest #3: cached=0/8810
gemini-flash-latest #4: cached=0/8810
gemini-flash-latest #5: cached=0/8810
gemini-flash-latest #6: cached=0/8810
gemini-2.5-flash #1: cached=0/8810
gemini-2.5-flash #2: cached=0/8810
gemini-2.5-flash #3: cached=8167/8810
Same key, same prefix, same everything. gemini-2.5-flash cache-hits 93% of the prefix by the third repeat. gemini-flash-latest never reports a cached token — because the response metadata shows it resolving to gemini-3.6-flash, a model generation that (as of this writing) doesn’t report implicit cache hits at all.
That’s the whole story: the alias moved. We pointed an agent at -latest and someone else’s resolution table changed which model our money went to. No commit, no config change, no deploy on our side. Cached input is billed at roughly a tenth of full price, so on a loop where ~90% of tokens should be cache reads, losing the cache multiplies effective input cost roughly 4–6x. Per-request token counters show nothing wrong — the token counts are identical. The only place this is visible is the one field almost nobody watches: cachedContentTokenCount, against what it should be for the workload.
This is what agent cost drift looks like
We’ve been calling this class of failure agent cost drift: the cost profile of an agent changing while its code and its outputs stay the same. It’s the cost twin of model drift, and it has the same defining property — nothing errors. The agent worked perfectly the entire time. Every run completed. The only symptom was money.
Three things this incident generalizes to:
A model alias is a config change someone else makes for you. -latest is convenient exactly until the provider’s resolution moves it somewhere with different cache behavior, different pricing, or a different quality profile. If your agent’s economics depend on caching — every loop-shaped agent’s do — pin the model. We pinned gemini-2.5-flash and the fix was one line.
Cache rate is an alerting metric, not a dashboard metric. We caught this because the metering is in the call path and we looked. The honest version of this story is that a cache rate of zero should have paged us, not waited for someone to read a table. A loop-shaped agent whose cache rate drops below its baseline is the loop tax coming back with interest — it’s the alert we’d want as a customer, so it’s the one we’re building.
Drift hides under green checkmarks. Everything that normally signals health — completed runs, correct outputs, stable token counts — was green for all 216 calls. Cost regressions don’t throw. If the only meter you have is the invoice, you find drift weeks late, priced at full rate.
One footnote we’re keeping for honesty’s sake: the run that surfaced all this ended when the agent hit its own daily spend cap — the same control plane that metered the drift also stopped the bleeding, and the agent doing the investigating wasn’t allowed to raise its own limit to keep going. That’s the point of the plane. The cap held, the probe took pennies, and the alias never gets to move our money again.