You have 1 article left to read this month before you need to register a free LeadDev.com account.
Key takeaways:
- Budget caps don’t work, and most engineers never hit them anyway.
- A one-hour cache (not five minutes) can cut AI-coding costs by 10-15%.
- Show engineers their spend and measure cost per unit of shipped work.
AI coding is amazing, but costs have really surprised all the engineering leaders. At Larridin, we spoke to engineering leaders from Uber, Coinbase, Fanatics, and many more, compiling a playbook to save on token costs. Our customers are seeing at least 20-40% in cost savings within days and weeks.
To save on costs, the most common response is a budget cap. It is the wrong one. Coinbase found that 91% of its engineers never hit their caps. The caps annoyed the many and interrupted the few, and the few were often the most productive.
If a budget cap is not the answer, what is it? In this article we explore mechanisms for achieving results.
Your inbox, upgraded.
Receive weekly engineering insights to level up your leadership approach.
Understand and optimize prompt cache
Ask most engineering leaders where an AI-coding dollar goes and they will say output tokens: the code the model writes. They are wrong. Take one task apart and the picture looks like this.
Coding agents remember nothing between requests. Every turn resends the full conversation. Prompt caching stores the unchanged prefix and re-reads it at a discount: on most Anthropic models, 10% of the input price.
That discount is what makes long sessions affordable, and the cache-read rate is the most important number on the price sheet. When Anthropic cut Fable 5.1’s rate from $1.00 to $0.25 per million tokens, the same task fell 42%. No change to the model or the work.
The cache expires
Anthropic’s default lifetime is five minutes; each hit resets the clock. A ten-minute coffee break kills it. So does a code review or a continuous integration (CI) pipeline run. The next turn re-reads everything at full price and rewrites it at a premium. One measured rebuild of a 61,000-token context cost $1.14. Uber lists expired-cache resumes among its top waste patterns.
The fix is a one-hour cache, not a cache warmer
Writes cost 2x instead of 1.25x; reads cost the same. It pays off in any session with a few gaps over five minutes, which is most of them. A warmer on a five-minute cache fires 12 times an hour, roughly the cost of one full rewrite; it only wins if you’d otherwise expire more than once an hour, and the one-hour cache beats it anyway. Uber runs one hour for interactive sessions, five minutes for subagents.
What can you change?
The default depends on billing. Claude Code requests one hour on a subscription but five minutes on an API key, Bedrock, or Vertex, which is how enterprises run it. promptCacheTtl: 1h and subagentPromptCacheTtl: 5m fix this, and managed settings push them org-wide. Larridin estimates these two settings cut an enterprise Claude bill 10 to 15%.
Ordinary actions break the cache
The cache is keyed to the model and effort level, switching either mid-task discards it. Tool definitions sit at the front of the prompt, a tool server connecting or disconnecting invalidates everything behind it. Compaction resets the conversation cache by design and costs most when the cache has already expired.
Measure it
Every response reports cache reads and cache writes. Reads should dwarf writes. If writes stay high, the prefix keeps changing. Coinbase raised its hit rate from 5 to 60% and credits that, with routing, for halving its bill.
Put a gateway in front of everything
More like this
All the changes in this article can be easily operationalized with an AI Gateway/ Router. This is one infra piece that will make everything much easier to manage.
For example, cache policy cannot depend on every engineer remembering a setting. A gateway sits between tools and providers, sees every request, and enforces the lifetime on requests that arrive without it. It also tracks spend per person and tool, and routes. Databricks reports routing each task to the cheapest capable model cuts average cost more than 30% at near-equal quality.
Change the defaults. Keep the choice
Simply changing defaults can make a big difference in saving costs. We worked with a customer to start all tasks on GLM 5.3 instead of Opus, and immediately resulted in 20% cost savings.
We did not take away engineers’ ability to choose the model. All we did was simply change the defaults. You can do this at Gateway layer (another reason why this is important).
Show engineers what they spend
Engineers fix numbers they can see. Uber put a live cost counter in the terminal, alerts at 50, 80, and 100% of expected spend, and a dashboard that flags 16 waste patterns with a dollar cost and a fix for each. Databricks escalates gradually: warning, approval gate, cheaper model. Suspension is rare and starts a conversation.
While cost data is available, just showing that cost data in Claude Code and Codex made engineers aware of the cost, and we found that engineers started making smarter choices when they could see the cost.
High spenders are often the highest producers. Make waste visible, not usage frightening.
Measure the output
Steps one through four cut cost. None of them asks what the tokens produced. You should have decent telemetry on what engineering velocity and quality is mapped to AI token costs look like.
The number that matters is cost per unit of durable output: code that ships, passes review, and does not come back as incidents or rework. Larridin’s benchmark across more than 100 teams puts median AI-coding spend at $213 per engineer per week with a 10x spread.
Spend alone predicts nothing. AI-native engineers kept converting dollars to output at $1,300 a week. Partial adopters saw returns halve past $600. Low AI engineers stayed flat across a 20x spend range. Same tools, same prices. The difference is skill.
Without this measurement, a high spender is either your best engineer or your most expensive mistake, and you cannot tell which. With it, you know where the budget converts to shipped work and where the constraint is fluency, which no setting fixes.
Promoted Partner Content
