Register or log in to access this video
Drawing from recent research of 250m+ lines of code: three risks of AI-authored code, with an idea or policy to mitigate each.
As AI continues to reduce the proportion of code guided by human experience, what is lost? This talk examines industry-scale code line data (including repos from Google, Microsoft, Meta, and 1,000+ others) to enumerate three main areas where AI-authored code bends toward incubating tech debt.
While the benefits of AI accumulate in the daylight of well-funded industry reports, the maintainability challenges of AI fester in dark corners. To illuminate how AI-authored code differs from its human counterpart, we take five years of “code operation” trends, and combine it with newly-released AI usage APIs.
The story emerging from the data suggests that historic “best practices” (e.g., DRY code) have loosened. The magnitude of this change appears unprecedented. Effective policies can mitigate the downside risk. Engineering leaders can sharpen their intuition for how to build smart policies by leveraging recently available data.
Key takeaways
- The scale of code quality changes are measurable and unprecedented
- There is learnable consistency in how AI can poison code quality
- Even though we often read about profound increases in code velocity, the incentives of the companies building the LLMs remain undocumented and likely imperfect
- Higher velocity carries new costs: better to understand them than to spike adoption without regard for long-term detriment
