You have 1 article left to read this month before you need to register a free LeadDev.com account.
Estimated reading time: 6 minutes
Key takeaways:
- Software engineering knowledge decay is structural. Fast cycles and churn strip context, even from teams that try to document.
- Most code work is upkeep: only 15-20% is new, the rest is modernization and maintenance.
- AI’s real value is in capturing the reasoning behind code, not just the code itself.
In software engineering, knowledge decay has some long-lasting effects. Most of us have experienced a colleague who unexpectedly departs their job, leaving the rest of the team scratching their heads to fill in the gaps. Perhaps their day-to-day work was never jotted down. Maybe their files and accounts are missing. Perhaps their contacts were stored on a personal device.
This is institutional knowledge decay: when technical knowledge lives in someone’s head and vanishes when they leave. In software engineering, that loss can create a black hole, especially for teams supporting complex architectures with intertwined dependencies and legacy languages.
“It happens all the time,” says Neel Sundaresan, general manager of automation and AI for IBM Software, creator of IBM Bob, and a former founding engineer of Microsoft Copilot. “Almost nobody ever plans for having to leave, but everybody does.”
For decades, this reality has stunted software operations and maintenance, where personnel churn is common. Under tight delivery timelines, documentation and architectural context are often the first things to go.
Sundaresan believes AI-assisted development could finally nip this trend in the bud by preserving reasoning alongside the codebase.
More like this
Why software knowledge traditionally decays
When you revisit old software projects, especially in large enterprises, they often have very little documentation, and the people originally assigned to them left long ago, Sundaresan says. On top of this, today’s rapid continuous integration and delivery (CI/CD) cycles lessen the perceived need to document regularly, since code changes iteratively with each commit.
IBM is confronting this firsthand as it modernizes legacy codebases for the AI era. “When you’re trying to take monolithic systems and containerize them or create microservices, you need to understand the architecture decisions that were made,” he says. Yet, some Java and COBOL workloads supporting IBM products are 30 years old, lacking both context and language-specific experts to maintain them.
Developers of yesteryear may have written highly efficient code and documented it for incremental changes. However, that documentation is rarely sufficient to guide modernization or migration into entirely different languages, leaving future teams without the firsthand knowledge needed to direct those efforts.
“You’d assume that software was built well then,” says Sundaresan, but even with teams of tech writers, that wasn’t always the case. “It’s almost a curse of software,” he adds, noting that documentation writers, project managers, and developers long operated as separate functions, exacerbating the disconnect.
The downsides of knowledge decay
“The biggest challenge is documentation being out of sync with the API that’s available,” says Sundaresan. “Docs being out of sync is actually a worse problem today since we release much faster.”
While working at Microsoft, he found some Azure command-line interfaces (CLIs) had more than 1,500 commands. CLIs and APIs change with new versions, and new parameters are often added without updating the corresponding documentation. To his point, a 2024 APIContext study found that 75% of production APIs don’t match their specification files. This mismatch can break integrations and applications.
Knowledge decay also creates substantial rework. “The challenge is everywhere,” says Sundaresan.
“Back when I worked at eBay, engineers would get moved around all the time,” he says. Much of the work relied on consultants, who would move on to the next job once a project was complete. This led to substantial duplication, as later developers would struggle to understand the original code and decide to redo it or change large portions of it.
He estimates that only 15% to 20% of code is newly written, while 60% to 65% involves modernization, migration, and maintenance. Whether it’s out of sync documentation or opaque code bases, that one missing link can result in significant wasted effort. “You can get 99% right, but then the 1% that you get wrong can appear as a huge time sink.”
How AI can fix knowledge retention
AI can now make old projects far easier to understand. One approach is using it to document legacy code. IBM engineers, for example, are feeding old Java and COBOL projects into AI to improve understanding and generate documentation and architectural diagrams.
“The person that created it might have a completely different capability compared to who’s looking at it today, who might be a college graduate that knows Python,” Sundaresan explains. Since AI systems have a large memory and have seen legacy patterns before, they can therefore help accelerate that knowledge retention for new hires, he adds.
It’s about more than just improving reasoning for old projects. It’s about weaving documentation and other structured artifacts into developers’ workflows so they can inform future development and modernization. That’s where AI agents can really help.
New York • September 15 & 16, 2026
Expectations are rising. See how other leaders are handling it at LDX3 New York.
How AI-assisted artifacts retain knowledge
For AI-coding agents to retain software engineering project knowledge, the context used during development becomes the new knowledge layer. “Your artifact is not code anymore,” says Sundaresan. “Your artifact is the prompts I write, the rules I make, the modes I create, the tools I use.”
In practice, that could mean including README files, documentation, and reference architectures in each repository, then feeding them back to agents. For instance, Bob, IBM’s AI assistant for the software delivery lifecycle, can create templates that preserve prompts, context, tools, and architectural choices, including whether agents use parallel or sequential tool calling and whether subagents are involved.
Of course, feeding agents this information with every prompt could consume significant context. “128K used to be a great context window,” says Sundaresan. “Now it is almost useless.” That said, he doesn’t see the context window issue as a hindrance, given context limits are increasing and architectures for accessing context are continually being optimized too.
Still, studies indicate AI coding is increasing duplication while reducing reuse and legacy refactoring. Avoiding maintainability woes will require teams to pair today’s AI-supercharged pipelines with deliberate knowledge-retention strategies. “It is a whole new skill to use AI to achieve what you want,” says Sundaresan.
As an analogy, he compares it to sports: “say people once ran 100 meters in, I don’t know, 15 seconds. Now, they can run in 10 seconds. To achieve that five-second gain, you need new kinds of shoes, new running style, new this, new that, all of that. Engineering jobs are becoming that. You need to have these kinds of skills.”
The exact protocols, standards, and strategies for accessing knowledge will keep changing alongside agents and their underlying large language models (LLMs). For Sundaresan, that makes documentation even more important. “Fast forward six months and all will change, but at least we document the context in which it was written.”
The past is the future
In 1984, computer scientist Donald Knuth introduced the idea of literate programming, an abstracted form of software development that combined underlying code with human-readable explanation. The goal was to create code that could be compiled by a machine but would also be understood by people.
Flash forward to today, and people are again talking to computers in natural language, only now directly via agents. The past appears to be repeating itself, with agents bringing us closer to this original intent, says Sundaresan. “In my mind, everybody’s going to be a software engineer,” he says. “They’re just going to write in natural human languages.”
So, while AI might seem futuristic, for him, it’s bringing us closer to the fundamental goal of IT in business. In the process, it could improve knowledge retention while evolving what it means to contribute to a software project, and who is involved in its ongoing development.
