Designing for money and happiness in large-scale open source

With the release of OpenTelemetry Specification v1.0 – a project that uses a standardized data format to capture distributed traces and metrics from your application – I’ve been reflecting on the path we blazed to get here, and the road ahead.

We’ve been at it for well over a year – three years if we count OpenTracing – and I’ve learned a lot about how to manage a large project such as OpenTelemetry. I’d like to talk about one aspect of the large-scale project design that we got right this time, due to effort and focus in the early days of the project.

That topic is money. Big money. Money and mental health.

Well-defined scope and goals are critical for mental health

Some background is in order.

I would politely call the last round of large-scale open source development I participated in, the ‘Container Wars’. I’m coining this term myself, but if you were there, you probably know what I’m talking about. While the technology was exciting and interesting, and ostensibly there was a lot of collaboration taking place, it often felt emotionally draining. I learned a lot about what I want, and what I don’t want, out of a large project that involves many organizations and ‘corporate interests’.

I don’t want a project that pretends to be a utilitarian utopia. That isn’t because I have no interest in creating an enjoyable project filled with respectful people working towards a common goal. In fact, I am interested in precisely that. But you can’t get there without trust, and you can’t have trust without honesty.

For the communities involved in the Container Wars, there was an honest desire within most participants to make something happy and vibrant. But there was a problem. The problem – the only problem – was money.

This problem would often get papered over. After all, everything in open source is free, right? So gauche to mention it, heaven forbid design around it. The natural reflex was to sweep it under the rug.

But the reality was that money was on the line. Money was being spent – lots of it – and even more of it was going to be made. Maybe. But no one knew where, or how. Perhaps by attaching your name to this component? Well, that component better stay in the project. Maybe by building a proprietary layer around this other component? Well, that better stay out of the project.

This lack of clarity led to a lack of honesty, which led to a lack of trust. Proposed changes became a mix of valuable input and weird corporate agendas. Discussions that should have been technical became political. And as engineers, we found that really, really stressful. It wasn’t what we signed up for. Open source stopped being fun at this point.

If you, dear reader, also served in the Container Wars, would you agree?

Anyways. Never again.

Boundaries make the world go round-aries

If you find yourself in an amazing situation, creating a large-scale and important open source project, this is my advice. What I’m about to write might sound like I wear a monocle and a top hat, but this could more aptly be named A Hairshirt Engineer’s Guide to Surviving Capitalism.

Be honest about the money

Be crystal clear about the alignment of interests, and be open about your own interests. If you can’t discuss this subject without looking bad, or if you can’t discuss this subject because it is literally incomprehensible, redesign the project. Redesign it until you feel like you can post this information, right there on the front page of your website, and be proud of it.

Don’t presume altruism

Altruism is not the same as alignment. When money is involved, you want alignment. Think through all the ways that someone might run away with the ball, and find a structure that makes those actions unattractive. Don’t presume your own altruism, either. You should have an interest too, and if you’re getting paid then that interest will involve money. And there is no need to be shy about that. In fact, you should be the opposite of shy. A key part of making this design work is having a clear and attractive story about where and how the money will be made, and how the project will benefit those who build and use it. You need this story; if your project is large enough that money must be involved, you must have a compelling explanation for how that function is expected to work. You will not be able to align your collaborators without it.

Lastly, don’t wait and hope

Trust me, success can be an incredible torsion force. If the points above are not thought through, early and clearly, they may rip the project apart. Or they may rip you apart trying to hold it together. I’ve seen several examples of this recently, and it makes my heart break. Just like testing, you will not be able to easily graft these features onto your project in the aftermath. They must be part of the design from the beginning.

Holy leaky abstraction batman, that all sounds dark! But I’m not being dark. This is your one life! You want to enjoy it, so do this work first and then relax. That feeling of trust is so important. It lightens every load. Put that load on the structure of the project, not the individuals who inhabit it. It is so relaxing to trust that everyone is on the same page, working towards the same goals.

I believe that interest alignment by design is the key to positive mental health in open source communities. You can make the happiness and satisfaction of your community a design requirement. You can make it happen for you, your colleagues, all of us. You can make it happen for me when I join your project. It’s worth the effort!