One of the main reasons why software development feels more challenging today than a few decades ago is the overwhelming amount of choice across all layers of the tech stack.

Consider all the options available in programming languages, frameworks, databases, tools, platforms, cloud services, and more. The latest CNCF report and the MAD (ML, AI & Data) Landscape both outlined the dizzying number of software options available on the market. Not to mention the various architectural styles, system design patterns, agile methodology approaches, etc. The choices are overwhelming.

Having alternatives gives teams more ways to solve specific project requirements and achieve business goals. However, while this may seem a good thing, in practice, it can be daunting, leading to indecision, procrastination, suboptimal selections, or even complete inaction.

The paradox of choice

In his TED Talk, "The paradox of choice", Barry Schwartz illustrates why an abundance of options can lead to decision paralysis, where individuals become unsure of which road to take. The impact of choice paralysis creates a mental strain on the individual and compromises their decision-making ability. This prevents engineering managers from effectively implementing the company’s technology vision and guiding their teams to build sustainable, adaptable architecture. 

We often associate freedom with the ability to choose, underpinning the belief Schwartz articulates as, “More choice equals more freedom, which equals more happiness.” However, this ideology also has a darker side, marked by an increase in anxiety, indecision, and perpetual dissatisfaction.

The two primary negative effects of excessive choice are:

1. Choice paralysis: With a plethora of options available, the decision-making process can become so overwhelming that it becomes difficult to make any choice at all.
2. Dissatisfaction with decisions: Even after making a choice, we often look back and compare our decisions against the options we didn't select. Moreover, the sheer volume of available choices escalates our expectations for a "perfect" fit, leading inevitably to disappointment when these lofty expectations are not met – even if the outcome is objectively good.

Anatomy of a “hard choice”

When you’re presented with a choice between A and B, and A is the better option, the decision is straightforward – you choose A. When A and B have equal merits, the choice remains simple. If they are equally good, you may go ahead with either. However, when A is better than B in some ways, but B is better than A in others, you’re faced with a hard choice.

In an easy decision, one option is evidently superior, or they are so similar that the choice becomes inconsequential. In a hard choice, you must perform a trade-off analysis, weighing the pros and cons of each option, knowing that neither is absolutely superior to the other. As philosopher Ruth Chang would describe it, the alternatives are "on a par": the choice can significantly impact the outcome, yet one option isn’t universally better than the other.

This scenario is common for senior engineers, who have to chart a course for the system architecture, knowing that there is no single "correct" way to design a system. All choices are valid as long as they meet the product and technical requirements and the team is aware of the trade-offs and potential technical debt.

Randy Shoup, VP of engineering at eBay and WeWork, captures this sentiment perfectly in his 2022 GOTO presentation: “There is no perfect architecture for all scale, for all phases of evolution, for all problem domains.”

The layers of decision-making

Understanding human decision-making requires considering a multitude of factors, including the physiology of the human brain, psychological influences (such as cultural biases, emotional triggers, past experiences, internal and external expectations, and peer pressure), and external environmental factors.

Just to illustrate the complexity of all the factors that influence our choices, consider these studies:

• An average human can hold 7 plus or minus 2 items in their short-term memory. According to Miller’s Law, on average, we’re good at comparing one variable at a time with a maximum of seven choices. For instance, choosing between seven apples based on price alone is manageable. However, our performance decreases as the number of variables – such as color, weight, shape, taste, etc. – increases, complicating our ability to perform complex trade-offs and make optimal decisions.
• Comparisons change how we evaluate things. In his TED talk, Dan Ariely outlines how our choices are influenced by the arrangement of options. For instance, Ariely notes in the same talk that when selecting wine, a consumer is likely to choose the middle-priced option from a set of three respectively priced at $8, $25, and $33. Introducing a higher-priced fourth option (e.g. $57) can make the previously highest price of 33$ seem more reasonable, thus shifting consumer preference. 
• We place a disproportionately high value on things we create ourselves. This phenomenon, known as the “IKEA effect”, and explored in a 2011 study, shows that people are willing to pay up to 63% more for furniture they assemble themselves compared to pre-assembled items. However, marketers knew well before this study that the more effort someone puts into something, the more they will value it. For example, back in the 50s, it was this principle that led General Mills, the US-based food company, to revamp their struggling "just-add-water" Betty Crocker cake mix. By changing it to a "just-add-an-egg" mix, the simple addition of a real egg made the baking process feel more rewarding, and sales soared. 
• The time of the day affects your choices. This is called the “Hungry judge effect” and it was named after a 2011 study that revealed that judges are more likely to grant parole in the morning than in the afternoon, due to decision fatigue. Although this particular study was later revisited due to an overestimation of the impact of decision fatigue, other papers support the idea that our decision-making capacity diminishes throughout the day, leading to more impulsive and less confident decisions.
• When our prefrontal cortex is busy, our emotional brain takes charge. The 1999 “Heart and Mind in Conflict” study found that when our prefrontal cortex is overloaded we are more prone to make suboptimal decisions, often favoring emotion-driven decisions (choosing cake over fruit) due to reduced impulse control.
• Incorrect incentives might have counterproductive outcomes. The cobra effect exemplifies how misguided incentives can backfire. This anecdotal case originally saw the British government in colonial India offering a bounty for every dead cobra, inadvertently encouraging the breeding of snakes for profit. In practice, we often pick an option that seems beneficial right now, even if it might hinder our long-term goal. 

Best practices for complex choices

There are countless strategies, philosophies, and books dedicated to enhancing decision-making skills. After all, this skill is crucial for software development and other disciplines such as finance, research, and data analysis.

Here are some best practices that I have personally found effective:

1. Check your assumptions before you decide

Before making a decision, review and challenge the constraints and requirements that will guide your choice. Ensure all stakeholders agree on what is essential versus desirable. Understand the boundaries (for example: financial, regulatory, SLAs, etc.) within which you need to operate. This helps prevent misalignment and the need for backtracking later.

A notable example of the importance of details is the loss of NASA's $125-million Mars Climate Orbiter in 1999 due to a metric conversion error, illustrating how critical accurate information is in decision-making.

2. Employ the explore-exploit trade-off

The explore-exploit trade-off involves balancing the choice between trying something new and sticking with what is known. It can be particularly helpful, for example, when balancing the choice between trying a new architectural style and sticking with a known quantity (e.g. modular monolith).

If facing a tight deadline or a quick release cycle, it’s advisable to "exploit" by leveraging existing knowledge and choosing the familiar option. Conversely, if the time frame allows, "exploring" (i.e., experimenting with a new architectural style) may yield valuable insights that improve future decisions.

3. Ask for support from your community

Decision-making does not have to be a solitary task. As psycho-economist Sheena Iyengar points out in her talk The art of choosing, involving your team or community in your decisions can reduce the burden of making a choice. Relying on the judgment of trusted and respected colleagues does not diminish your competence; it underscores the importance of collaboration and recognizes individual limitations – after all, software architecture is a team sport and everyone benefits when a software system is designed collaboratively.

4. Don’t waste more time choosing than doing

The 2003 overhaul of the Linux scheduler, an operating system that manages the execution of processes and threads, illustrates this point beautifully. The 2003 revision saw the O(n) scheduler exchanged for the O(1). 

The O(n) scheduler ranked each task individually, often spending more time on task prioritization than on task execution. Instead, the O(1) scheduler grouped tasks into a limited number of priority buckets. This resulted in increased performance by being less precise but executing in a constant time.

This example highlights that no choice is perfect. Surprisingly, it can sometimes be more effective to approximate a decision and complete a task, moving on to the next one, rather than spending excessive time researching and prioritizing options.

Final thoughts

Navigating the vast array of choices in software development is no trivial task.

Engineering managers should remember that no choice is perfect, and importantly, very few choices are irreversible. Opting for a suboptimal solution, or accumulating architectural technical debt, simply signals the need for continuous evolution and improvement in your system’s architecture. This ongoing process necessitates adapting and making more informed decisions as your system and its requirements evolve.

Ultimately, the goal is to find a balance that empowers engineering teams to make meaningful decisions without becoming overwhelmed. By implementing strategic decision-making processes, we can provide the necessary framework to reduce choice overload, thus enhancing productivity and satisfaction in software development environments.