London

June 28–29, 2027

New York

September 15–16, 2026

Berlin

November 9–10, 2026

Where did all your tokens go? Measuring coding agent impact with OpenTelemetry

Coding agents like Claude Code are reshaping how your team ships software, but how are you evaluating them? This talk is about using OpenTelemetry to measure their real impact on DORA metrics.

Speakers: Annie Freeman

Register or log in to access this video

Create an account to access our free engineering leadership content, free online events and to receive our weekly email newsletter. We will also keep you up to date with LeadDev events.

Register with google

We have linked your account and just need a few more details to complete your registration:

Terms and conditions

 

 

Enter your email address to reset your password.

 

A link has been emailed to you - check your inbox.



Don't have an account? Click here to register
November 17, 2026
NYC 27 Pre sale ticket block image

Coding agents like Claude Code are reshaping how your team ships software, but how are you evaluating them? This talk is about using OpenTelemetry to measure their real impact on DORA metrics.

I tried out viral Claude Code ‘hacks’ and measured how effective they really are.

Agentic coding tools like Claude Code have inserted themselves into the critical path of software development. But we were still evaluating our vibe coding the same way – on vibes. This doesn’t scale across an organisation.

We realised there’s a hidden layer: your development workflow is now observable. You can measure how software gets built, not just what gets deployed.

How? These tools aren’t just generating code, they’re emitting OpenTelemetry data like token usage, latency, acceptance rates, and pull request counts. But most teams aren’t capturing it, so they’re flying blind while these tools shape delivery performance and DORA metrics. By observing your coding agents, you can understand their real impact: where they accelerate delivery, where they introduce risk, and how they affect lead time, change failure rate, and MTTR.

In this talk, I’ll show how we started treating coding agents as production dependencies, not assistants, and share what we learned, what didn’t work, and how we’re getting the most value from this data.

The teams that get the most value from AI won’t be the ones using it the most, they’ll be the ones who can see and optimise what it’s doing.

Takeaways:

  • What data leaders should measure
  • How leaders can use this data to improve delivery performance
  • How to answer questions like: Is AI actually making us faster? Where are we introducing risk? Which tools are worth standardising on?