You have 1 article left to read this month before you need to register a free LeadDev.com account.
Estimated reading time: 5 minutes
While dashboards and metrics are great for monitoring system health and expected behavior, customer feedback can complement metrics and help you unearth those unknown unknowns.
Sitting down for a podcast interview, Jeff Bezos told Lex Fridman: “When the data and the anecdotes disagree, the anecdotes are usually right.” Bezos cites an example from early Amazon days: data showed that customers waited less than 60 seconds for support, even though customer feedback reported longer wait times. During a weekly business review meeting, Bezos went ahead and made a customer service call, and the wait turned out to be more than 10 minutes.
Customers’ anecdotal feedback (customer feedback) is personal, story-based feedback from a customer about their experience with a product or service. In practice, I have seen it highlight a wide variety of issues like unknown customer friction, unintended system behaviors, and developer churn. Note that “customers” here aren’t just the end customers; they could also be businesses or intra-company clients.
While customer feedback is integral to your product health, it should not replace metrics altogether. In my team, we’ve learned to view and use them as canaries, informing us where the metrics themselves may not be correct or complete.
Recognize what your metrics may be missing
Metrics and dashboards are often a good way of keeping tabs on your systems to monitor intended behaviors. But they aren’t the only way. Often, these methods miss unknown unknowns – uncertainties that you may not be aware of or cannot anticipate. For example, an unknown architectural gap in a new product may cause widespread customer churn at launch, even though the involved team may not even be monitoring it, since it was an unexpected scenario.
As our systems grow in volume and complexity, unknown unknowns increase in frequency, becoming harder to detect (due to volume), and impacting more customers. If you lead a high-volume or complex product, establish a reliable feedback loop for your metrics, such as customer contact data, user surveys, or client feedback. This anecdotal data will help you validate that: (a) you are monitoring the right set of metrics, and (b) the insights derived from those metrics represent the actual customer experience.
Your inbox, upgraded.
Receive weekly engineering insights to level up your leadership approach.
Build a repeatable feedback loop
To gain a more holistic temperature check on your systems, you need to find repeatable mechanisms for regularly reviewing customer feedback alongside your regular metrics.
A good starting point could be to bring insights or metrics derived from customer feedback into your regular retros and decision-making processes. For example, in your weekly business reviews, you can audit the number of customer contacts reporting a specific class of issue (like customer call wait times) and identify changes in its trend over time. In the event of a deviation, teams should investigate the cause and share insights with leadership. In some cases, the trend changes may be related to seasonality, while in other cases, it might signal a need for mitigative action, decision, or a deeper dive. Year-over-Year (YoY) or Quarter-over-Quarter (QoQ) comparisons are good markers of seasonality that the teams can use to differentiate expected behaviors from anomalies.
Next, consider building structured and periodic customer experience debriefs. These are recurring sessions where cross-functional teams (like engineering, product, operations, support, etc.) review qualitative feedback and customer anecdotes together. It allows teams to review verbatim customer feedback to derive insights that their metrics may not be measuring.
Depending on the type of customers you serve, you could also set up regular Call Listening sessions and debriefings with the team. Call listening allows team members to evaluate customer conversations in real-time or with anonymized recordings after a call concludes. Alternatively, you could send out periodic surveys to customers and review the collated insights. The kind of insights I have seen emerge from such sessions is surprising. In one instance, it helped my team identify a feature managing customer authorizations that was causing unintended friction. Our metrics were not catching it, but a customer survey conducted by a peer organization highlighted it, enabling us to resolve the churn and build monitoring around it.
Outside of these processes, build forums where engineers, managers, and support teams can raise operational concerns that are not getting appropriate attention. For instance, a developer on my team highlighted a component that slowed down every launch, impacting our clients. Once managers were aware of this detail, a quick refactor fixed it, saving two weeks of development time per feature launch.
More like this
Use AI to amplify the feedback loop
Recent advancements in GenAI and machine learning (ML) offer powerful ways to force-multiply the impact of customer feedback, further strengthening teams’ metrics. While AI adoption is in its nascent stages, teams should find opportunities to train AI agents that help with clustering qualitative inputs like support tickets (or freeform fields). Such AI Agents automatically highlight trend changes faster and signal teams to dive deeper or mitigate previously unknown customer issues. For instance, applying AI to a sample set of customer anecdotes on a daily/weekly basis can generate insights that direct the team to look deeper.
Embed the mindset in your team
Engineers and managers should feel empowered to highlight customer concerns, even when they only see them anecdotally. To do this successfully and sustainably, leaders should strive to promote a culture that enables teams to share early warning signs and adjust their metrics when customer feedback reveals gaps.
To help embed this philosophy, teams and leaders should openly recognize individuals who surface customer feedback, leading to positive improvements. On our team, we’ve used small notes of appreciation (“kudos”) on a company-wide tool, with occasional gift cards and leadership appreciation for frequent recipients. These tools should help reinforce behavior and build a long-term, purposeful mindset.
Deadline: January 4, 2026
Call for Proposals for London 2026 is open!
Final thoughts
By ensuring structured attention to customer and team anecdotes (along with their existing key performance indicator metrics), leaders can build a tighter feedback loop. It helps surface unknowns and grounds leadership decisions in data as well as lived experience.
If these mechanisms don’t work for your teams, I recommend looking for options that help balance quantitative data (metrics) with some other qualitative insight available to you and your teams. This helps in reducing surprises and strengthening long-term quality and delivery.
