3 agentic AI questions engineering leaders must answer

Estimated reading time: 12 minutes

Key takeaways:

• Track developer wellbeing as agent use grows, since 41% of developers in one survey show signs of being hooked on AI coding.
• Enforce agent accountability in the pipeline: 87% of organizations claim AI governance policies and fewer than a quarter have implemented controls.
• Count the cost of verifying agent output, which takes 60% of agentic AI spending according to McKinsey.

---

In What’s next for DORA?, we focused on the legacy of the annual survey and the hole it leaves behind as the tech industry loses a critically independent benchmark in this time of great change. Since there was no 2026 survey, there’s no way of knowing what would’ve been on it. Except we can darn well assume that agentic AI would’ve been a big focus.

The 2025 State of AI-assisted Software Development found that 61% of developers had never interacted with an AI agent. That would likely be more than flipped this year, with 97% of IT leaders at least exploring agentic strategies. 

However, just because everybody’s doing it, that doesn’t mean anybody knows how to measure it. 

“Today usage is close to universal, and rising volume mostly points to where the system strains – in review, in recovery, in how well teams still understand their code,” Eirini Kalliamvakou, distinguished scientist at DX, tells LeadDev. “The outcomes we want haven’t changed. We do need to rebuild our understanding of the work that leads to them, and adjust the measures that describe it.”

Until now, the tech industry’s leading and lagging metrics were for the deterministic actions of predictable systems and the less deterministic and traceable actions of human developers. Now, we need to create new baselines to reflect non-deterministic AI agents.

“When it comes to evaluating your own performance or improving it, the internal data is extremely important. Understanding your own baselines, setting goals, and determining strategy based on where you are,” Laura Tacho, senior principal technologist at Amazon Web Services (AWS), tells LeadDev. 

It’s unclear how the DORA team will evolve beyond the annual survey, or how its work will change with agentic AI. As Tacho says, DORA has already helped companies build a “habit and practice of collecting data frequently, looking at how performance changes over time, and getting better at continuous improvement.”

For this follow-up piece, we ask developer experience researchers for the metrics and measures engineering organizations need to make a habit of in order to understand the impact of agentic AI as we enter 2027. 

How do your developers feel about your AI?

We’re talking more and more about AI, but our interviewees contend that the most important impact of AI to measure is on your human developers. 

“For three years, we saw essentially either a positive or neutral impact on employee wellbeing as you increased AI adoption,” Nathen Harvey tells LeadDev. Until recently, he was the DORA lead, and now has joined the AWS builder research team. “As agentic AI takes off and we’re doing more with agents, I am concerned that the wellbeing of individuals is going to dip, primarily because it can be so addictive working with agents, trying to do too much all at the same time.” 

You end up with a ton of context switching, Harvey says. “You also end up with maybe some higher expectations from senior management on your work output. All of this combined can lead to less job satisfaction, higher levels of burnout, and overall lower wellbeing for individuals in our teams.”

The recent Atlassian State of Developer Experience report found that 87% of developers feel AI is helping them write code faster, yet 75% face the same or greater workflow friction. 

Telemetry from a longitudinal analysis of developer logs by JetBrains found that, while AI users produce significantly more code, they also delete more. It also found that context switching is on the rise as developers switch among tools to prompt, verify, and review non-deterministic code and agentic workflows.

For senior backend developer Fernando Alvarez, this results in a 20-minute thinking gap with loops of analysis, planning, implementation, and verification “on a task you already understand.” Every switch has a price, and that cost depends on how different your tasks are, he writes. The more work you have in progress, the longer each item takes to move through your workflow. The usual trick is to limit work in progress, but that’s especially hard in the face of limitless AI.

Annie Vella ran a longitudinal study on the impact of AI coding assistants on software engineering and found that, while 84% of participants reported faster overall developer experience, for 27%, it actually worsened as both flow state and cognitive load eroded. 

Way back at Monki Gras 2018, in a talk on self-sustainability and burnout, open source community consultant Jessica Rose reminded the audience that the tech industry is the only one outside of drug dealing that calls customers “users.” Now more than ever that paradigm seems to have flipped onto the tech creators. 

Coddy Tech ran a survey of developers that found that 41% are “hooked on AI coding,” displaying at least three of four behaviors:

• Losing track of time using AI tools.
• Struggling to stop once they start.
• Coding after hours when they are meant to stop.
• Telling themselves “just one more” round.

Again echoing drug addiction, 80% of developers surveyed also say their AI usage has felt more like dependence than an advantage. 

“Activity and volume metrics are useful for finding the next bottleneck, and a poor way to prove return. Putting high stakes on them means betting on the wrong numbers,” says Kalliamvakou, feeding into the dopamine hit of AI.

Agentic success, she argues, comes down to a measured mix of predictions, problems, and personal stories.

Developer sentiment toward quality practices still matters, echoes Tacho. These lagging metrics move more slowly than keeping an eye on token spend which you might track daily. Some of these sentiment signals include: 

• Adoption – how many developers, what kinds of agents, and what stages of the software delivery lifecycle (SDLC) are they being used in?
• Percentage of pull requests used or created by an agent or agents.
• Cost by code review versus proactive security scanning. 

At the pace of AI, Tacho recommends that all teams keep a close eye on rework rate, code review, wait time, and size of pull requests.

“How developers think about the quality, maintainability, security, their confidence in the code, and how those sentiments are changing, along with adoption, gives a lot of insight into where more attention might be needed,” which helps teams spot emerging bottlenecks sooner, she adds.

Who owns what the agent builds?

If AI is performing autonomous actions, are developers still in charge? Clarity and enforcement around responsibility is one of the clearest signals of AI success, but often one of the hardest to measure. Add to this, while 87% of organizations claim to have AI governance policies, fewer than a quarter actually have implemented controls.

Kalliamvakou advocates for establishing rules around what agents may do without a person, when a human needs to approve, and who is accountable. However, those rules are meaningless without enforcement. 

“Is our policy enforced or only written down?” asks Charlotte Fleming, researcher at Octopus Deploy, pointing to a need for pipeline-as-governance. “A policy document states what you intend to happen. A deployment gate actually makes it happen. When agents produce change faster than people can review it, compliance has to run continuously in the pipeline, not as an audit after the fact.”

In the absence of policy-as-code, you at least need to create the psychological safety and clarity of who is accountable for outcomes – and don’t say the agents, says Dan Belcher, cofounder of Mabl. “Which decisions and work are we ceding to agents and how are we ensuring that these stay aligned with our objectives and intent? Where is the human in the loop? Where are the guardrails? How can we sample, audit, evaluate, and improve the process?”

In the age of agentic-driven software, it’s all about these accountability questions for Kevin Storer, now principal engineer at Storer Enterprise Strategy and former qualitative research lead at DORA.

“A lot of executives are asking their employees to verify the outputs of any kind of LLM-driven work, because they will be responsible for the veracity of that output. Yet, when you start talking about agents and multi-agent architectures and these deeper layers of abstraction, it becomes literally impossible to audit the trail of thinking. So, if something goes wrong, who’s responsible?”

Storer says it comes down to answering: if something goes wrong with an agent and it’s hard to audit, how do we know who’s really at fault for that mistake? “Can we really say employee A is at fault for something that happens that an agent has done incorrectly under their watch?”

If accountability for agents isn’t the simplest metric, what else can you measure?

“For me, the more important question is: how do we make sure that employees even feel comfortable executing the frequent, top-down mandates around using LLMs extensively when it is not clear to them how they will be held accountable?” Storer asks. “The downstream impact of that – from just a psychological and emotional perspective – is that workers often don’t feel confident actually executing on the LLM mandates that so many large companies are imposing upon them right now.”

A whole new metrics playbook needs to be written, he argues.

How is AI impacting your continuous delivery?

The 2025 DORA report found that AI is an amplifier of everything in your organization. There’s no reason to think that would change with AI agents, especially if they are trained on your internal processes, systems, and workflows. 

While you can still ask previous years’ survey questions, it’s time for companies to develop their own baselines, including unveiling patterns around successfully integrating agents into their software delivery lifecycle. 

“I think DORA’s biggest legacy is giving engineering leaders a common way to talk about delivery health. The core DORA metrics are still the best long-term read on whether a delivery system is healthy, even as that system is changing,” Quotient CEO Lizzie Matusov tells LeadDev.

Most of the change right now is happening in the inner loop, in how features are scoped, written, and reviewed before they enter the deployment pipeline, she adds. 

“Teams need leading indicators closer to the work: the speed and quality of what’s getting built, and the friction developers and their agents run into,” Matusov explains. With agent-assisted software delivery, “measuring that change comes down to how a unit of work moves through that pipeline, through the lens of the system, the humans, and the agents they collaborate with. Spending time iterating and measuring this system through system, human, and agent input is where the majority of friction is going to be uncovered, which ultimately impacts the health of the overall system.”

When code has little cost, speed of creation is no longer an interesting metric.

“Teams should measure agentic AI by whether reliable software reaches customers faster, at lower cost, including human review and rework,” James Jarvis, CEO of ClearRoute, tells LeadDev. “Writing code is only 20% of the software development lifecycle. Accelerating that while ignoring the 80% – testing, security, and approvals – just builds a bigger queue for customers.”

While the bottleneck was never on the dev side, ops is feeling more pressure.

“Shipping more code is no longer the hard part, so throughput alone tells you very little,” warns Alex Circei, CEO and cofounder of Waydev. “Teams should measure AI adoption, impact, and ROI separately: how much of the work agents are doing, whether that work survives in production without extra rework or a higher change failure rate, and what each shipped change really costs once you add tokens and the human time spent reviewing and verifying agent output.”

A June 2026 McKinsey report found that 60% of total agentic AI spending went toward iterative response refinement such as checking, correcting, and improving outputs before delivery,  rather than the initial prompt call.

“Verification overhead never shows up in deployment frequency,” Circei says. “It shows up later, in the incident channel.”

The 2025 DORA report cited platform engineering as an enabler of AI speed, quality, and control. However, at the rate of agentic AI, you also need to consider if your platform is ready to absorb AI-generated change – and to onboard agents. Platform engineering has always been defined as opt-in for developer users. Will agents use this same paved road and will it be optional for them too?

“If the platform is optional, or is disjointed, getting built piece by piece, it has only the boundaries someone thought to add to it,” Fleming warns.

“Teams with ephemeral environments, code coverage, and cost control were more likely to report that AI had a positive effect on delivery speed and stability,” she continues.

According to the Future of Platform Engineering Report, which she authored, more than three-quarters of teams report an extremely positive effect agents have had on ephemeral environments. “This probably says more about the platform as a whole than about any single feature: broad, automated platforms can absorb the extra volume, and this completely fits DORA’s AI amplification research.”

Don’t forget about the cost of maintenance too. The pendulum is swinging back toward build-your-own, as teams rely on agents and AI in general for more modernization and language migration. Teams that are doing that also lean on AI to maintain what it creates, Fleming says.

“Agents don’t remove maintenance, upgrades, or governance. They add to it,” so she reminds teams to clarify who owns not just the code but the long-term products.

Several studies have come out recently that suggest individual engineers are getting faster but overall teams are not.

Faros AI’s Q3 2026 telemetry review of 22,000 developers across 4,000 teams found that most companies are stuck in a speed trap, with code churn (lines deleted vs. added) surging 861% with high AI adoption. In addition, bugs per developer increased 54% and incidents per pull request tripled. This causes what the Faros team calls a “senior engineer tax” where they have to audit and untangle AI-generated work. 

While writing code was never the bottleneck, now the cognitive load of verification bears down on almost all teams.

“Everything about the process is open for review, for re-imagination, and I think it’s such an exciting time, for better and for worse,” Harvey says. “It’s also this exciting time where we collectively, as an industry, have an opportunity to learn together again,” if organizations continue to share what they are learning.