A recent thread on Hacker News highlighted the danger to team dynamics and effective knowledge sharing created by a prevalence of tacit knowledge in software development, and how it differs from tribal knowledge.

Tribal knowledge is collective wisdom within a team, informally shared through interactions and experiences, encompassing unique methods and practices not officially recorded. Tacit knowledge is individual understanding shaped by personal experiences and perspectives, characterized by intangible insights and skills that are challenging to articulate or formalize.

Both types of knowledge are critical in organizational contexts. However, when this knowledge isn’t properly shared among team members, it leads to the accumulation of technical debt due to a lack of shared understanding about best development practices. The absence of a common knowledge base can make it challenging to identify and follow the most efficient path, further contributing to tech debt.

While technical debt cannot be eradicated, it can be slowed down by encouraging a culture of continuous learning and open communication, where knowledge is actively shared, documented, and integrated into the workflow.

How tribal and tacit knowledge creates tech debt

The team lore of tribal knowledge

Say a software development team has been working on a product for several years. Over time, they’ve developed a unique set of workarounds, practices, and solutions to handle specific challenges in their project. These aren’t documented, but are well-understood within the team.

A new member joins and starts contributing to the codebase. However, due to the lack of docs, this new developer needs to be made aware of the team’s practices and workarounds before they can be an effective member of the team. In the meantime, they introduce changes that conflict with the established but undocumented practices. This leads to issues like system instability, code inconsistencies, and increased complexity, contributing to technical debt.

The individually acquired insights of tacit knowledge

In another organization, an experienced developer has been working on a critical component of a product. Over the years, they have developed a deep understanding of the code’s intricacies and have fine-tuned the performance through subtle optimizations. This expertise includes understanding how different parts of the codebase interact, optimizing algorithms for specific use cases, and navigating complex dependencies, all based on personal experience and intuition.

The developer eventually retires, and a replacement is brought in. Despite having access to the code and basic documentation, the new developer lacks the deep, intuitive understanding of the system the former developer had. Technical debt then increases because the new developer has no good documentation to work from, doesn’t fully understand the nuances of the code and comes up with less efficient solutions that will have to be fixed later down the line once they, or others, gain this complete understanding.

In both of these examples, an overreliance on informal, non-documented knowledge (tribal in the first case, tacit in the second) creates challenges in maintaining and evolving the software system efficiently. Non-optimal solutions are implemented that will need to be revisited and revised. This results in a cycle of inefficiency, where the lack of comprehensive, well-documented knowledge leads to suboptimal decisions and implementations, necessitating future updates – the very definition of technical debt.

Knowledge transfer strategies for informal knowledge

Better documentation is just one piece of the puzzle. The problem with tribal and tacit knowledge is that some things can’t be written down, and even when they can, teasing out the information in the best way to convey the knowledge necessary to the rest of the team requires a deliberate process. 

The three components of a knowledge transfer strategy should be:

1. Direct knowledge sharing, where the team shares knowledge within itself through specific knowledge transfer sessions.
2. Documentation, both formal and informal, uses not just text but visual, video, and audio components as necessary.
3. Culture building, so knowledge transfer becomes ingrained in the company's values.

Direct knowledge sharing

This is a formal, experience-based approach to transferring knowledge within a team. You want to allow all team members to share their experience and knowledge synchronously through:

• Interactive learning sessions. Regularly scheduled meetings or workshops where team members share experiences, solutions, and challenges. These sessions can be show and tell, ask me anything, or roundtable discussions to facilitate the exchange of tribal and tacit knowledge.
• Mentorship and pair programming. Pairing less experienced team members with veterans in a mentor-mentee relationship. In software development, this can be effectively implemented through pair programming, where tacit knowledge can be transferred in a practical, hands-on manner. This one-on-one interaction is crucial for guiding complex, nuanced aspects of projects.
• Storytelling and case studies. Encouraging sharing stories and real-life examples that embody the team's experiences and lessons learned. This can be a powerful tool for conveying tacit knowledge.
• Exit interviews and transition sessions. When experienced team members leave, conduct thorough exit interviews and transition sessions where they can pass on crucial knowledge to their successors or the team.

As an engineering manager, your role here is to be a facilitator. Create channels for open communication within your team and encourage these meetings and discussions. This open dialogue helps to transfer both explicit and tacit knowledge effectively.

More specifically, you want to establish the mentorship programs and pair-programming sessions, and when team members leave or new ones join, manage the transition effectively. Ensure knowledge is passed through comprehensive handovers, documentation, and mentoring from existing team members.

Documentation

Documentation should systematically capture and store institutional knowledge using:

• Internal documentation. While recognizing that not everything can be documented, capturing as much knowledge as possible is still important. This includes decision logs, design rationales, and troubleshooting guides that are maintained and updated regularly.
• Knowledge repositories. Utilize digital tools to create a central repository where team members can access and contribute their knowledge. This can include wikis, shared drives, or internal forums.
• Other tools. Explore and trial new tools to facilitate knowledge sharing, such as online whiteboards or code visualization tools.

Without effective ownership, documentation can suffer from a “tragedy of the commons,” where no single team or individual owns documentation, no one takes responsibility for it and it falls stagnant.

To push back against this issue, you must designate specific individuals or teams responsible for maintaining and updating documentation. This clear assignment of responsibility ensures accountability. To share the load, you can also make updating documentation a part of the regular workflow rather than an afterthought and then schedule periodic reviews of the documentation to keep it current and relevant.

Culture building

Creating an organizational culture that prioritizes knowledge sharing, learning, and collaboration across different teams and departments isn’t easy. You can start by: 

• Building cross-functional teams. Encourage collaboration between different departments or units within the organization to broaden the scope of knowledge sharing. These can be joint projects, rotations, or even simple meetups to discuss what different teams are working on. This helps limit knowledge silos and transfer tribal knowledge across different areas of the organization.
• Encouraging a culture of continuous learning. Promote an environment where seeking and sharing knowledge is valued and rewarded. This can be achieved through recognition programs, providing learning resources, and encouraging innovation and experimentation. 
• Establishing feedback mechanisms. Implement regular feedback sessions where team members can discuss what knowledge is missing, what needs to be updated, and how the knowledge transfer processes can be improved.

As a leader, model the behavior you wish to see in your team. Encourage continuous learning and professional development. This can be through workshops, online courses, or attending conferences. By valuing learning, you cultivate a culture where knowledge acquisition and sharing is the norm.

Final thoughts

A reliance on undocumented tribal and tacit knowledge leads to increased technical debt. Inefficient solutions, borne from a lack of shared understanding, require future revisions. By effectively transferring tribal and tacit knowledge within the organization, you can reduce the accrual of technical debt. This requires a strategy emphasizing direct knowledge sharing, comprehensive documentation, and cultivating a culture of learning and collaboration. 

These measures will not just help increase current performance, but also secure the team’s future capabilities, thereby managing future technical debt more effectively.