Challenges in this debate can range from cost and business value, to team morale, ego, and bureaucracy. With so many factors, how have engineering leaders been navigating the decision, and what lessons have they learned for the next time the question rears its head? 

As part of the ‘Weighing up the pros and cons of build vs. buy’ series, LeadDev brought together a group of engineering leaders from across the globe to share their experiences of build vs. buy: their decisions, their regrets, and the considerations that need to be made.

To open the session, Dawn Parzych, Manager of Developer Marketing at LaunchDarkly, gave a presentation on the ten questions engineering leaders should ask before deciding to build or buy. These questions fell under the umbrellas of cost, business decisions, and growth and change; they included:

• What are the known costs and unknown costs?
• What is the cost to purchase?
• What is the cost of having two or more systems?
• Is building this part of your core business value?
• What are the motivations behind building?
• Who are the users?
• What happens if you change course?
• How easy is it to switch to a different solution?
• Who will maintain and enhance the product?
• What am I missing?

After Dawn’s presentation, the group embarked on a discussion around what they had built in the past, and some interesting points on feature flagging tools were made.

What have you built in the past?

The group was asked to note on a virtual whiteboard what they had built in the past, with the most common answers being feature flagging tools, content management systems, and logging systems. 

The discussion quickly steered towards building feature flagging tools and the concept of ‘it being an if statement, but it getting out of hand quickly.’ One attendee expanded on this saying how you start with a build as a proof of concept, knowing it can be thrown away, but soon drift into the discourse of, ‘Well what if we make it just a little bit better?’ Eventually, it ends up at a point of ‘Okay, we’ve built a feature flagging system – but a slightly rubbish one. If I had my time over I probably wouldn’t have built it this way.’

Another attendee continued to mention why builds can be chosen over buying at first: external dependency is scary. It can be daunting for developers to say ‘We’re going to have to integrate something into our codebase’ when the team could just write it in as they know the systems and the language, and they won’t have to create an API surface for it. Additionally, when needing to develop something quick, attendees discussed how using a third-party vendor means a lot of initial bureaucracy with legal teams and sign-off.

However, attendees were quick to move onto discussing the obstacles that appear later on down the line of a build. Feature requests start appearing, more conversations are needed to manage it, and, eventually, tech debt accumulates. One attendee described how you put the flag in, you launch the feature, but nobody goes back to take that out; you end up with bloated code and lots of flags that aren’t needed.

As a proof of concept becomes a built product, attendees discussed how this can lead to ego getting involved, and the idea of buying a replacement can cause friction as people tend to guard their territory. One attendee said that you can melt resistance by ensuring that whenever you’re talking to an engineer who built the product, to assume that everyone involved did a great job with what they had to deal with at the time, and to find positives in what they built.

This part of the session finished with attendees answering if anyone who had built a feature flagging system had then turned around and bought something instead. The resounding answer was: ‘I’ve wanted to’. Obtaining sign-off from folks who ask, ‘Why do we need to buy it if you’ve already got it?’ and providing justification can be difficult; the path from build to buy is not an easy one.

At this point in the session, the group split into four breakout rooms to each discuss one of the following questions:

• What’s most important when considering buy vs. build?
• What is something you bought and ended up building? 
• What have you regretted building in the past? 
• What are you currently considering buying or building?

They then returned to the main group to share their discussions and hear the experiences of others.

What’s most important when considering buy vs. build?

The main topics covered by the breakout group included costs, scalability, go-to-market, and the option of open source. When this was opened to the rest of the group, considerations included: What are the skills of your current teams? Have you even got the ability to build? How well can you onboard a third party? How long will the bureaucracy take? What do we need to think about in terms of governance, security, and data privacy?

What is something you bought and ended up building?

A member of Group 2 shared their account of a bought product, feeling as though they had to ‘put it in the corner and tell it to behave while the engineering team gets on and actually solves the problems.’ This attendee described how they bought for speed: to get the minimum viable product out even though it was not quite right. It may have got the team to market, but it soon caused a slow down in terms of business-as-usual activity, so the team had to look at how they could safely replace the platform they had bought by iterating on top. The bought product may seem to solve everything in the beginning, but once the team moves into the actual product engineering, and working with the product, it can soon be realized that the problem can’t be solved with the tool unless ‘major surgery’ is performed on it.

The attendee likened this analysis process of a buying decision to waterfall: figuring everything out upfront and then if there are emerging requirements being discovered on the fly, realizing the tool is not going to support the newfound capabilities needed. It was mentioned that buy-then-build is usually seen in larger, older companies and that startups usually begin with building internal bespoke custom tooling. Another attendee chimed in here to say they had the opposite experience, that at a startup they started with a buy because they needed to get it off the ground quickly. Later on, when they had a robust enough engineering org and enough growth to justify a bespoke solution, they built. This attendee continued to say that it’s a sign of a healthy engineering org when you change your minds on build vs. buy as circumstances and context change: just because it’s true now doesn’t mean it has to be true later.

When buying, the motivation of the team should also be considered. One attendee discussed how if you force teams to adopt something they weren’t involved in the decision of, or they don’t understand it, you’re making their jobs harder. ‘This is how the business has to work because this is how the tool is designed to work.’ Ultimately, the team is still going to have to work with that tool and if they’re spending more energy working around it than with it, they will see little benefit in having the tool around.

What have you regretted building in the past?

One attendee viewed this question from an interesting angle: it’s not always your own decisions you regret. They spoke of situations where they have shown up and the tool was already there (e.g. from an acquisition). It’s easy to say ‘Why did they choose this when clearly there are better ways?’ But without knowing the circumstances in which a decision was made, it is not someone’s regret to harbor; they were not there.

To answer this question, another attendee said ‘feature flagging’. They shared their current situation of now considering an open source or paid feature management tool due to their two in-house systems now buckling under the pressure of the data. With this next move also comes the questions of ‘Would a new tool integrate with the older stack? How attached are the engineers to the in-house systems?’ This attendee recognized how their team is currently wrestling with the decisions that were made eight years ago when the internal systems were built, and how now the organization is at a different level of maturity.

The regret of building an authentication system also arose, with an attendee speaking about how in the beginning the API was simple, but it grew into a social authentication where different options needed to be integrated due to fraud control and security. The extension cost of adding things to it became higher than integrating a third-party solution.

The notion of fraud control was familiar to another attendee. They spoke of knowing somebody who built payment processing for a startup; they used some off-the-shelf tools but didn’t use a sophisticated vendor and therefore had to build some things of their own to move quickly. Soon, they had to deal with and fight fraud. The question was then asked: how sophisticated do things need to be when thinking about the adjacencies to the tool that you may run into once you’re in production?

What are you currently considering buying or building?

One attendee was looking to buy tooling, e.g. login software, cross-browser testing. The reasoning behind this is that there is a lot of choice on the market, and so it would be inefficient and difficult to build a similar tool with all the features the team would want.

On the contrary, another attendee was looking to build because the tooling they were looking for is not available to buy. This person works in pharmaceuticals, and this is a common occurrence in the industry where an incredible level of customization is needed. However, the attendee also recognized that there are some tools they wouldn’t even attempt to build themselves, such as Snyk or another container security scanning type tool.

To round out the discussion, it was asked: with so much availability of tooling on the market, why then do teams want to build their own? ‘Frustration!’ The attendee who called this out explained that every tool is going to have its own issues, and when more engineers feel like the bought tool is creating more issues than it’s solving, ‘Why don’t we just build our own?’ feels like an easy question. However, the reality is that the build will need to provide at least the same level as the tool being replaced, and there is no guarantee that the built product will be any less frustrating.

Closing thoughts

The debate of build vs. buy is perennial for a reason; the weight of circumstance and context is heavy in this decision-making process, and no one can decide for an engineering leader whether build or buy is the correct move for their organization.

The best engineering leaders can do is to learn the considerations they need to balance, and accept that the decision they make is right ‘for the current climate’. Healthy acceptance enables organizations to adapt and change their tooling decisions as they themselves adapt and change. Through this empathy for themselves and their teams, engineering leaders can ensure their confidence in the path they choose to take, and act accordingly when this path isn’t quite right anymore.