Making ‘Big Changes’ Successfully

Big changes always require a lot of time and effort, so how can we make sure they succeed?

I joined a team many years ago as a consultant working on an internal application with a tight deadline. Our strong team spirit and an ambitious goal focused our efforts, and we were on track to deliver. But I was curious to find out more about the teams around us. I started conversations in our communal kitchen to discover other projects and initiatives. One tech lead, let’s call them Alex, shared how their team was working on a complicated migration project – moving from a relational MySQL DB to a NoSQL DB (specifically Mongo DB). This sounded like a ‘big change’.

At this time, not many companies used NoSQL DBs. Alex had found a great use case in their company. Their team was responsible for a core service built in a standard Java stack (Spring, Hibernate and MySQL). With growing demand from customers, this core service started having frequent performance issues in production. Alex said that a NoSQL DB would resolve the performance issue, removing the overhead of destructuring and restructuring data to and from a relational DB; a perfect use case for Mongo DB.

Since this core service had several years of history, Alex’s team had a lot of work ahead of them. Management had agreed to let six of their eight developers work on the ‘big change’ of migrating, with the other two providing ad hoc support and maintenance. Their team had already been working on this migration for at least six months when I first spoke to Alex.

Fast forward a couple of months, and we had delivered our project ahead of our deadline. Alex invited our team, amongst others, for a showcase of their migration project and use of Mongo DB. Given the early days of NoSQL DBs, I sensed the excitement in the room. We were all curious to understand what it was like to work with a NoSQL DB. Alex led us through their code, highlighting how much simpler it had become. Alex also walked us through other everyday operations, such as running queries and debugging.

At the end of the demo, Alex invited questions. I raised my hand and asked, ‘I understand you picked a NoSQL DB to fix the performance issue. Did it fix the issue? And if so, how much has performance improved?’ Alex appeared annoyed by my question. With hands on hips, Alex responded, ‘Of course it fixed the performance issue! Performance is one of NoSQL’s benefits,’ and moved on to a different question.

I wasn’t satisfied with Alex’s answer. But I also recognized asking further questions in this forum wouldn’t have a positive impact. What reason would Alex have to listen to me? After all, I was seen as an external consultant working on a different project with no real reason to provide input on their work.

I left this client a few months later; with our business sponsor happy, we’d delivered well beyond his expectations. Despite leaving the client, I kept in touch with some of the developers. A year later, I inquired about the NoSQL migration project. I found out that Alex’s team spent another six months completing their migration work. In total, that was more than a year’s worth of effort! When they went live, they discovered a big surprise. The migrated code did not resolve the performance issues; somehow, it had made the performance issue worse! After several failed attempts to find and address the cause, the team rolled back their work. Management declared the migration project a failure and disbanded the team.

The ‘Story of Alex’ and a failed attempt at a ‘big change’ in technology happens far too often. Yet, every successful technology company will face a ‘big change’ at some point. ‘Big changes’ are often less about ‘bad decisions’ than ‘good enough decisions’ that let the business continue to thrive. But due to the business and technology landscape changing more rapidly than software systems, good enough decisions start to look like poor decisions in hindsight. 

For your business and product to remain competitive, you now face a ‘big change’. Your ‘big change’ might look like the long journey from on-premise to cloud computing, turning a monolith into a set of services, or rebuilding an application in a newer technology or framework. If you find yourself on a ‘big change’ project, don’t approach it like Alex did: relying on a lot of faith. Use the approaches below to make a ‘big change’ more likely to succeed.

Quantify the problem and success criteria 

A ‘big change’ means a lot of time and effort invested. From a business perspective, it’s a double hit because there is a lost opportunity cost on top of the time invested: a team could have been working on something more valuable. Considerable time invested should have a proportionally large return. Build a model to quantify the return on investment (in terms of time or money). Measure where you are today and define what success looks like to know when you’ve reached your goal.

Start with a tracer bullet 

I first came across the tracer bullet (also known as a ‘pathfinder’) pattern in the original Pragmatic Programmer book. A tracer bullet is the most straightforward end-to-end task working in a production environment. This work allows you to gather real-world feedback about critical decisions and uncover any unknown risks. For example, if you’re automating a complicated manual deployment process, your tracer bullet might be a static HTML page running through your automated deployment pipeline.

Work in small, end-to-end increments

Use end-to-end increments to show progress and build trust. If you pick your end-to-end increments well, those increments should deliver immediate value. Delivering value earlier than later always builds trust. Stakeholders will experience benefits rather than hearing the promises of benefits. When stakeholders realize benefits, it also relieves pressure around ‘When are you done?’ You’ll find more support to continue with your ‘big change’.

Prioritize increments by risk and value

A ‘big change’ invariably includes many risks. Work on increments that offer high value. But also prioritize increments providing the best learning opportunities. Discovering a fatal flow in your approach after two or three weeks will be better than finding it six or eight months later. As an example, you might choose a low-risk but valuable piece of work as your tracer bullet. Then you might choose a high-risk, moderately valuable piece of work shortly after.

Use ratcheting to prevent regressions

Ratchets allow for movement in one direction only, such as when you use a socket wrench for tightening nuts. Use this concept to make sure you’re always incrementally moving forward. For example, I worked on a project migrating away from Hibernate (an ORM) to jOOQ (SQL in code). We used a tool called Checkstyle to prevent developers from using Hibernate in new code. We used an exception list to allow for the existing code. Our team agreed to never add to this exception list, preventing the situation from getting worse. Additionally, each team member would remove at least one exception from the list each week. 

Start what you finish

One of the reasons that ‘big change’ is hard is that the journey is long. If you initiate a ‘big change’, you did so to solve a business problem. Make sure you deliver your promise, and be sure to see the end of what you start.

Making ‘big change’ is never certain, nor easy, but sometimes circumstances demand it. Avoid following in Alex’s footsteps and use the tips above to make your ‘big changes’ more likely to succeed.