AI won't make your team 2x faster

Estimated reading time: 6 minutes

Software engineers have always been searching for ways to simplify a system or speed up delivery.

The temptation to find silver bullets is driven by a tendency to break complex problems down into their constituent parts.

The latest silver bullet is generative AI, which promises significant productivity gains when harnessed effectively. However, most engineering leaders are only seeing minor productivity improvements of somewhere between 0-10% so far, according to the latest LeadDev Engineering Leadership report. In practice, most developers have to work around systemic bottlenecks, slow approval workflows, and knowledge silos.

Instead of thinking about engineering organizations as codebases to be tweaked, we need to think of them as interconnected systems. By mapping the relationships between teams, processes, and information flows, leaders can identify the constraints that prevent AI from delivering on its promised productivity multipliers.

Engineering is a sociotechnical system

Understanding systems requires more than zooming out from the problem at hand, and systems thinking isn’t simply a case of thinking bigger.

According to Donella Meadows, author of Thinking in Systems, “A system isn’t just any old collection of things. A system is an interconnected set of elements that is coherently organized in a way that achieves something.” At its most fundamental, a system “must consist of three kinds of things: elements, interconnections, and a function or purpose.” 

Sand on the beach is not a system; there are many elements, but they are not interconnected, nor do they serve a function or purpose. Your digestive organs are a system; change any elements or how they connect, and the system would function differently. 

Most engineering leaders tend to overlook the system as a whole and focus on only one part. Once you break the elements down, it’s easier to see how they fit together and how you could arrange them differently. This is a useful tool, but it only feels like systems thinking. Engineering is technical and social, and a sociotechnical system is not reducible to code. 

When you look closer, you see many different kinds of interconnections, including ones that connect within the engineering team (including people, processes, tools, and metrics), and ones that connect outside the engineering team (to teams such as product, finance, CS, sales, and, even outside that, users).

It’s here where we start to see the cracks in treating generative AI as a silver bullet. When a system requires elements, interconnections, and purpose to work in concert, adding a new element (even if it’s AI) isn’t enough to radically improve productivity. If teams want to transform, they need to transform both the elements and the connections between them.  

How to build a system that makes AI work

The potential for generative AI to accelerate software delivery remains vast. But rushed adoption, like mandates without a defined AI strategy, threatens that potential. For changes of this magnitude, the tortoise always defeats the hare. 

Take an outcomes-based approach

Speed is just one element of a high-performing engineering organization. Quickly building a product riddled with bugs or a product that goes against user requirements is not a victory.

First, avoid getting stuck in technical debates. Cursor or Copilot? Is Claude still better for code, or does this next version of ChatGPT put it ahead? These questions lose the forest for the trees. 

Instead of getting too granular on specific tools, focus on desired outcomes along the value stream and iterate backward from there. 

Change leadership expert Niko Canner explains that systemic change takes time and intention. “A strategy for systems change needs to frame how to pay the right kind of attention to the long arc, and the right kind of attention to the work that can be done now, in today’s conditions and with today’s resources.”

Along the way, take special care when deciding how to measure what you truly value. PR volume, for example, might be less important to measure with AI adoption as capacity becomes less of a bottleneck. But large organizations not accustomed to high velocity may struggle with quality and security as they try to go faster than ever before – these metrics suddenly become more important. 

Sense and respond

Pressure from above is motivating many engineering and company leaders to mandate extensive usage of AI tools as fast and as deeply as possible. 

Former Glitch CEO, Anil Dash, points out how unusual this is: “Did your boss ever have to send you a memo demanding that you use a smartphone? Was there a performance review requiring you to use Slack?…When they started using spreadsheets and email and the web…they absolutely didn’t have to drive adoption by making people fill out paperwork about how they were definitely using the cool new technology.”

Instead of trusting developers with their tools, many engineering leaders are taking a command-and-control approach, imposing changes from the top down. 

Instead, introduce changes and watch how the system responds. Find the teams that would benefit most from AI and test it with them before expanding. Incremental learning will contribute to organizational transformation much more effectively and deeply than ordering change. 

Layer fast and slow systems

The rush to adopt AI poses the potential for a system shock: while some parts stand to benefit other elements might suffer. According to environmentalist Stewart Brand, you can parse systems into different layers, each of which interacts with the other but which have “different change-rates and different scales of size.”

In an engineering context, the product team might be focused on increasing the release pace, while the security team slows things down to ensure security. This tension is a feature, not a flaw. 

If every team is forced to adopt AI on the same schedule and to the same depth, this balance can be thrown off kilter. In Uplevel’s research last year, we found that after implementing GitHub Copilot, developers saw a 41% increase in bugs within pull requests. In a sufficiently complex system, you can’t radically change speed without tradeoffs. 

Perform work in parallel

Systems are stable, but that doesn’t mean they’re still. As Alex Danco, Product Director at Shopify, explains, “If you try to change one variable, you can apply as much effort as you like, but the minute you let go, the system will just snap right back to its original configuration.”

Changing systems requires a sustained, parallel effort to change all three components. This “means you need to find all of the different people who you’ll need on your team, and somehow get them all listening to you at once, all probing and pushing on the system to change, for an extended period of time,” he writes.

To sustain such an effort, organizations need lots of people working together for a long period of time, and they all need to know what they’re working toward. 

The Uplevel Method emphasizes examining how an organization’s ways of working can become deeply ingrained, which allows for sociotechnical problems to reveal sociotechnical solutions. Low-level metrics and high-level edicts aren’t enough; successful change starts with leaders, but it also requires buy-in, co-authorship, and participation from developers so that all parties are invested in the outcome. 

Engineering leaders as stewards, not commanders

In 1992, developer Jack W. Reeves wrote, “Designing software is an exercise in managing complexity. The complexity exists within the software design itself, within the software organization of the company, and within the industry as a whole.”

Managing complexity requires systems thinking, and the role of the engineering leader is to navigate that complexity, not declare that they’re “AI-first” and shortcut transformation.

Engineering leadership is just as much about engineering successful organizations as it is about engineering code. The leaders who realize AI’s potential best will understand and shape its role in the systems that underpin their teams.

Get articles like this in your inbox

Choose your LeadDev newsletters to subscribe to.