You have 1 article left to read this month before you need to register a free LeadDev.com account.
Don’t miss out
Get weekly engineering leadership content in your inbox.
Estimated reading time: 13 minutes
The developer productivity metric has become popular at Meta, by measuring the time it takes to create code changes.
Software engineers at Meta are increasingly turning to diff authoring time (DAT) to identify developer bottlenecks and inform decisions on tooling investments. The parent company of Facebook, Instagram, and WhatsApp uses DAT to signal broad productivity trends, as well as measure any changes to its internal developer experience.
Of course, not all diffs are created equal, which might be why this metric hasn’t been widely used at other companies yet.
What is diff authoring control (DAT)?
At Meta, a diff is a pull request and DAT focuses on the inner development loop – the writing, building, testing, and debugging of code. The tech giant emphasizes diffs should be kept small and reviewable, which, in part, accounts for its average DAT of 50 minutes across 87% of available diffs.
As explained in an episode of the Engineering at Meta podcast, two things need to be validated to measure DAT:
- Coverage – Are they covering all the diffs they can?
- Duration – Are they capturing as much of the time as possible for that particular diff?
A popular way to measure developer productivity is the number of diffs per engineer. However, like lines of code, these metrics can be easily gamified. Since a diff can be any human-made change, measuring diffs of any kind is challenging.
For years, Meta developer metrics were largely anecdotal or instinctual. Now, DAT pulls data directly from the integrated developer environment (IDE) and any tool related to coding a diff. This includes version control and instrumentation at the operating system level, tracking the actions taken before the commit within a certain tab.
DAT “helps you target improvements to make the process more efficient, and, ultimately, speed up,” Sarita Mohanty, a Meta data scientist in charge of identifying pain points, bottlenecks, and inefficiencies in the diff authoring journey, said on the podcast. Also, it helps Meta engineers make more informed decisions on which tools to invest in, through A/B testing the impact of changes on development speed.
This is the fifth version of DAT at Meta, which sees the data science teams setting thresholds and factoring in edge cases. The next iteration will focus on how they can measure DAT the same way across the company.
“You don’t want the definition of your metric to change, but you also want the metric to evolve,” Moritz Beller, software engineering researcher at Meta, said on the podcast.
More like this
How Meta runs experiments with DAT
Measuring the impact of a change before company-wide rollouts is another top DAT use case at Meta.
Meta data scientist Henri Verroken spoke of an experiment the Hack programming language team ran during a follow-up podcast episode. Hack is Meta’s open source extension of PHP, with a more rigorous type system. Until recently, developers would get to the point of committing a diff before it ran through type checks – only flagging any signals outside the inner loop, breaking flow state.
The Hack team A/B tested if triggering type mocking (when available) earlier in the development cycle would speed up or hinder development. After just three weeks, this change triggered a remarkable 14% decrease in average diff authoring time.
Of course, Meta has a massive codebase with several diffs a day being created by tens of thousands of engineers, which gives exceptional control over its experiments, as well as speed to results.
“You don’t need to run the experiment for very long to find statistically significant differences,” Beller said.
The festival of engineering leadership
London • June 16 & 17, 2025
Should you measure the individual developer?
“It’s not 100% accurate for all developers,” Mohanty admitted, which is one reason why DAT isn’t measured at the individual level at Meta.
The main argument against this, though, is psychological safety. Meta has already laid off about 23% of its workforce over the last three years, with an expectation to soon cut another 5% of its “lowest performers”. Measuring individual productivity could decrease happiness, which could decrease productivity.
Outside of these DevEx experiments, DAT is used as a top-line metric at the organizational level. It can be seen at a lower level, but, if a cohort is less than a certain number, there are safeguards in place to make it impossible for managers to de-anonymize developers.
That’s a stark difference from BlueOptima, an enterprise benchmarking tool that aggregates software development data, including hours of developer coding effort. It’s very similar to DAT, but their Team Lead Dashboard provides insight on how to improve at the individual level. However, engineering leadership cannot access this level of granularity.
“We see [it] as a positive way to be able to reinforce an understanding to those team leaders and those managers of: How can they identify where the problems are within their team? How can they go and help them?” Adam Minter, account director at BlueOptima, explained.
“You don’t hire bad developers, right? They go through rigorous processes to become a developer in your organization,” he continued. “If something’s happening, you need to be able to identify that and be able to go and help them. We’re trying to give that power back to developers.”
No matter what and who you’re measuring, remember that engineering is both a science, and an innately creative endeavor. This requires a mix of quantitative and qualitative metrics to measure the developer experience.
“Developer delight is not just looking at the data and [saying] ‘we know best what developers want’,” reminded Verroken. “Listen to developers to make sure that they have a delightful experience.”
