You have 1 article left to read this month before you need to register a free LeadDev.com account.
Key takeaways:
- Approvals existed, but nobody could trace what actually changed or why, exposing the real gap in AI governance: control, not capability.
- Rollback paths for multi-step AI workflows need designing before deployment, not improvised mid-incident.
- Governance only works when it’s built into the workflow, not filed away in a policy document nobody reads under pressure.
McKinsey’s most recent survey found that 88% of organizations now use AI in at least one business function. Only 6% qualify as high performers capturing real value from it, and 50% have already experienced at least one negative consequence. That gap between adoption and outcome is not a capability problem. It is a control problem.
I have seen this directly, working with developer tooling companies and the engineering teams using their products. The patterns that emerge are consistent, and none of them are the result of teams doing unreasonable things.
Unclear ownership over AI-generated changes
At a devtool startup, I was working with an engineering team that had started using AI-assisted code suggestions in their review workflow. The decision made sense – velocity went up, review cycles shortened, and the team was shipping more with the same headcount.
Then a change went out that introduced a logic error that accumulated in production over several days before anyone noticed. When we tried to figure out what had happened, we ran into a problem that had nothing to do with effort or intention.
The pull request (PR) had approvals. The review had happened. However, when I asked who had understood the change before it merged, nobody could answer. When I asked what the AI had suggested versus what the developer had written, nobody could answer that either.
We spent two days on something that should have taken 20 minutes. At the end of it, we still could not say with confidence what had changed or why it had been accepted.
The approval existed. The understanding did not. This is not a people problem. It is a structural one. Review processes were designed for a world where a developer wrote the code being reviewed. When AI generates the change, the same process runs, but the context a reviewer needs to evaluate it is different – and the process does not account for that yet.
More like this
Missing audit trails in AI-assisted incident response
Some months later, a client team used an AI-assisted tool to recommend configuration changes during a live incident. The on-call engineer approved them quickly. That is exactly what the tool was designed for, and it worked – the incident resolved.
Three days later, a downstream service started producing errors. We ran a postmortem. I asked for the decision log from the incident. There wasn’t one. I asked what the AI had recommended and why. Nobody knew. The engineer remembered approving “something about timeouts.” The tool had no structured record of what it had suggested or what inputs had generated the recommendation. The closest thing we had to an audit trail was a Slack thread from 2 am.
The postmortem concluded with “likely related to incident changes.” That is not a finding anyone was satisfied with, but it was the honest answer given what was available.
Nobody had made a bad decision. The tool had done what it was supposed to do. The gap was that nobody had designed a record-keeping layer around it, because when the tool was adopted, logging decisions had not seemed like an urgent priority. Shipping had.
Teams do not notice the absence of decision records until they need them, either during a postmortem, a compliance review, or when a second problem is caused by the response to the first.
Rollback paths that cause secondary incidents
A team I worked with had an AI agent handling parts of their CI/CD pipeline – configuration updates, pipeline triggers, and environment changes, executed in sequence. It worked well. Then a deployment failed midway through and the team decided to roll back.
The agent had already completed three steps before the failure. Step one had updated a configuration that another service was now reading. Step two had triggered a pipeline that had partially run. Step three had failed, and that was the visible problem, but steps one and two had already produced effects downstream.
Source: Himanshu Saxena
Rolling back step three was straightforward. Rolling back steps one and two, with other services already depending on those changes, took four hours and caused a second incident. A service unrelated to the original deployment went down because we reversed a configuration it had started using.
When I asked when rollback boundaries for this agent had been defined, the answer was that they had not been. This was not because the team was careless, but because defining rollback paths for multi-step agent workflows is a genuinely new problem. The team had thought carefully about the forward path. Nobody had mapped the undo yet, because until that moment, there had been no reason to.
The cost of ungoverned AI is showing up in budgets
These gaps have consequences beyond individual incidents. They are starting to show up as financial ones too, and faster than most leadership teams anticipated.
Earlier this year, Uber burned through its entire 2026 AI budget in four months after encouraging engineers to adopt agentic coding tools at scale. When the company’s COO officer was asked whether that spend had produced more value for customers, the answer was that the connection was not visible yet.
According to the Wall Street Journal, some large enterprises hit their full annual AI budget within three months, driven by agentic tools that consume significantly more compute per task than earlier AI products.
Source: Himanshu Saxena
I have seen a version of this on a smaller scale. Teams that give engineers broad access to agentic tools without usage boundaries find that spend accelerates faster than output does. Token consumption becomes a budget problem before it becomes a governance conversation. That is not a failure of intention – it is what happens when genuinely useful tools get adopted before the management layer around them does.
Token consumption becomes a budget problem before it becomes a governance conversation.
Why policy documents do not solve AI governance
After the rollback incident, the organization’s response was to schedule a governance review. That is a reasonable instinct. The problem is that governance reviews and policy documents are designed for environments where humans make decisions at human speed.
AI systems act faster than review cycles were built to handle. A policy document that nobody reads during a live incident does not function as governance at that moment. It is a record that governance was discussed.
The shift that addresses this is treating governance as part of the engineering system rather than something that runs alongside it. That means putting controls inside version control, CI/CD pipelines, infrastructure automation, and deployment gates – where decisions actually happen – rather than in documents that exist outside those systems. This is the same principle engineering teams already apply to testing and security: if it is not in the workflow, it does not run when it matters.
New York • September 15 & 16, 2026
The pace of change keeps accelerating. See what other leaders are doing about it, at LDX3 New York.
What changes when you treat this as an engineering problem?
The starting point is understanding where AI is actually influencing decisions – not where it has been approved in principle, but where it is acting today. Code review, incident response, infrastructure changes, deployment pipelines. For each area, three questions are worth asking:
- Who is accountable if this produces a wrong outcome? No answer → accountability gap.
- Can the decision be reconstructed after the fact? No answer → traceability gap.
- Can the action be reversed without causing something else to break? No answer → rollback gap.
These questions are not meant to slow adoption down. They are meant to identify where gaps will surface eventually, and to close them before they do. The teams I have seen navigate this well did not slow down adoption. They changed what they built around it.
The first change is definitional. Most teams have an implicit sense of where AI can act freely and where a human needs to be involved. Making that explicit – and encoding it into the workflow rather than relying on engineers to remember it – is what turns that sense into a control. The boundary does not have to be perfect on day one. It has to exist somewhere other than in someone’s head.
The second change is default logging. Decision records that depend on someone remembering to create them do not exist when they are needed most, which is under pressure. The teams that avoided the postmortem problem I described were the ones where the tool produced a structured record automatically, not the ones where someone had written a process document about why records were important.
The third is designing the undo before deploying the agent. For any multi-step workflow an agent owns, the question of what happens when step three fails and steps one and two have already run needs an answer before deployment, not during the incident. That answer does not need to be elegant. It needs to exist.
Deployments and incident response are the right places to start because the consequences of gaps there are most visible – and most likely to reach leadership quickly.
Teams that handle this well will not necessarily be the ones with the most capable AI tools. They will be the ones that recognize AI actions need the same engineering standards applied to any other change in their systems – defined ownership, logged decisions, and a tested path to reverse course… and start building those early.
AI governance is typically framed as a policy question. In practice it is an engineering question, and one that most teams are working through in real time. The organizations that treat it that way will find it easier to scale AI use confidently, rather than pulling back after the budget runs out or the second incident lands.
