Why 70% of engineers avoid measuring lines of code

Estimated reading time: 4 minutes

It’s been one of the hottest-debated topics in tech circles over the past few years: can you effectively measure and optimize software engineering team performance? 

When the management consultants at McKinsey weighed in last year, it triggered a lot of blowback. Why does measuring engineering team performance get such a strong reaction?

The consensus is that focusing on measuring outputs over outcomes ignores half of the software development lifecycle, excluding important factors like organization and developer experience. While there have been some strong efforts to close that loop through the SPACE framework, there is still no single accepted way to measure the efficacy of engineering teams.

Last year, we surveyed 600 engineering leaders to see if a clear consensus was emerging. The 2023 report showed resistance to measuring individual performance within an engineering team, especially for progression or promotion conversations. Further, it highlighted the broad range of approaches being applied to different teams.

Now, one year on, are we seeing some trends emerging? In October, LeadDev surveyed nearly 1,000 engineering leaders to find out how most organizations measure their team performance and their biggest challenges in doing so. Here’s what we found.

The metrics you love, and love to hate

Cycle time was ranked as the most useful engineering productivity metric for the second year running, followed by lead time, deploy frequency, and change failure rate. This shows a clear preference for measuring the time it takes for tasks to be completed to tighten feedback loops.

Conversely, 70% of respondents actively avoid measuring lines of code, flying in the face of Elon Musk’s school of engineering management. Another 47% avoid measuring pull requests, 44% don’t like to measure the number of tickets closed, and 42% steer clear of story points. Ability to be gamed, lack of context, and a focus on outputs over outcomes were all cited as reasons to be suspicious of these metrics.

Read the whole engineering team performance report here

The rise of DORA and SPACE

While the DevOps Research and Assessment (DORA) and SPACE frameworks are gaining traction, acceptance isn’t widespread. As such, engineering leaders are left bearing the cognitive load of quantifying their team’s output against wider company goals.

This year, we saw a growing awareness of these frameworks, and satisfaction levels rose in tandem. 41% of respondents find DORA metrics to be either a very effective or effective framework for measuring team performance, up from 34% last year. While fewer respondents are using SPACE, 34% saw it as a very effective or effective framework, up from 23% last year. Just 4% of respondents saw either framework as ineffective.

A rapidly changing vendor landscape

According to our survey, GitLab usage for measuring team performance dropped from 66% in 2023, to 20% this year, with a slew of best-of-breed tools filling the gap. LinearB rose from 3% to 15%, Pluralsight Flow from 3% to 13%, and Jellyfish jumped from 4% to 9%.

Still, the most popular tools for measuring team performance are the simplest: JIRA metrics (52%) and the humble homegrown dashboard (29%).

The goal-setting void

Before you can measure performance, you need some targets to shoot for. While most respondents use quarterly OKRs to keep their teams on track throughout the year, wider organizational strategy and goals aren’t as well understood as bosses think.

60% of respondents at the CTO or equivalent seniority level say they have “a complete understanding of strategic business goals.” This drops to 29% for managers of managers, and 21% for engineering managers, suggesting the message isn’t filtering down as well as senior leaders think it is.

Plugging gaps with learning

While hiring may be bouncing back, leaders likely have fewer opportunities to hire their way out of trouble, making training and internal upskilling more important than before. In fact, 70% of respondents said that learning and development for engineers is a current priority.

Technical expertise is the most common development area focused on, with 78% looking to plug technical knowledge gaps by upskilling their teams. Developing leadership and strategy

skills was a focus area for 41% of respondents, similar to communication and interpersonal skills, at 39%

Get into the detail

There’s this and much more to be found in the 2024 LeadDev Team Performance Report. Download it here.