How to measure and improve success in your engineering team

Moving the needle begins with setting the right goals.

As an engineering leader, senior developer, or founder of a new startup, I want you to take a minute and think – really think – about a project you are currently working on. 

Now, let me ask you four questions about that project:

1. What will you consider a huge success?
2. What will you consider a massive failure?
3. How do you know that you are currently on the right path towards success?
4. And my last question is, once finished, how can you tell if you nail what you wanted to achieve?

As engineering managers, we run many projects – including:

• Moving from one frontend framework to another;
• Replacing our database with a new and shiny one;
• Creating the killer feature that will give us the edge over our competitors.

But while you are working on a project, how do you know it will actually make a difference? Why are you doing it? And how do you make sure that it won’t feel like wasted time and energy once finished?

Results-driven culture, impact-driven culture, data-driven development, etc. have the following three fundamental rules you must follow:

1. Have a well-defined Key Performance Indicator (KPI);
2. Set a goal on that selected KPI;
3. Allow your team autonomy and give them ownership of the plan and the execution for reaching that goal.

Sounds simple, right? And I bet you already have Objectives and Key Results (OKRs), KPIs, and goal-setting processes of some sort. But although these rules are all easy to achieve in a superficial way, they’re much harder to actually get right. For it to work, you need to nail all three of them. So, how can that be done?

Selecting the right KPI

What is a KPI? They are a set of metrics that help you measure and track what you want to achieve. 

However, there is one thing to remember about KPIs that is sometimes missed: remember the ‘K’ in KPI. It stands for ‘Key’. If you have multiple KPIs, then you have many things as your top priority. And if everything is a priority, then nothing is.

For a project, select only one KPI and aim to improve this metric and nothing else.

From my experience, successful KPIs follow the following six rules:

Rule 1 – A KPI should be measurable. Don’t settle for boolean ‘yes or no’, or something amorphous that you can’t measure. Let’s say that you want to get into fitness. Let me ask you, what does ‘getting into fitness’ mean to you? Is it running a marathon? Is it cycling for three hours without stopping? Is it the ability to lift 50 kilograms? If you can’t measure it over time, or it only has two states of ‘yes or no’, you will not be able to track your progress. This means you cannot ensure that what you are doing is actually pointing you in the right direction.

When thinking about goals and KPIs, the first thing most people do is think about the goal they want to achieve. Really, you need to do the opposite. Think, instead, about what you want to change and how you can measure that. Great! You’ve found your KPI. Now you can attach a goal to it (which we will explore later in the article).

Let’s go back to the ‘get into fitness’ goal. Without thinking about what ‘getting into fitness’ means to you and how you can measure it, your goal is almost like a wish. There’s nothing wrong with this, but to be proactive in reaching your goals, think about your KPI first. Set a goal second.

Rule 2 – The meaning of the metric’s direction on your KPI should be obvious. There should be a clear understanding that your KPI’s direction is either good or bad. It shouldn’t be vague.

Rule 3 – Your KPI should reflect the value you are trying to bring and not the value’s outcome. Earnings are the outcome of a great product. Fewer pager duty alerts are the outcome of improved quality in design, code, and testing. Team velocity is the outcome of fewer dependencies and a smoother development experience. It is easy to set the KPI on the value’s outcome, but try to avoid this because the outcome can be achieved in different ways, and it can hide the thing you really want to change.

Rule 4 – The metric on your KPI should frequently change, at best daily, but no more than weekly. This is so you can track what you’re doing, and react fast to adjust your plans when things aren’t going according to plan.

Rule 5 – Your KPI’s metric should be something that you and your team members have the power to move. If your KPI depends on someone else, it will hurt your team’s ownership and accountability. Replace it with a KPI that has a metric you have direct influence over.

Rule 6 – The KPI should lead to action and not staleness. For example, the best way to reduce the number of bad deployments is to not deploy at all. If you choose a defensive KPI, it will lead to fear and staleness, and will not drive your team into action.

Now, before you can move on to set a goal on your selected KPI, consider the following:

• Check the current status: measure where you are now. Sounds obvious, right? But not everyone does it. Once you know where you really are, everything becomes much easier for you to see where you want to be, and in turn, this improves your goal-setting accuracy.
• Visualize the metric status as much as possible: one dashboard is worth a thousand words. A hundred ‘What is our current status?’ questions and several boring alignment meetings that waste time for you and the entire team. So put it everywhere. Send an automatic daily email or Slack message to everyone in your team and anyone else who cares and shares your KPI’s current metric status. If you are doing dailies, open the dashboard there as well. The more people that see it, refer to it, and take it into consideration, means the more alignment, commitment, and focus you will get. 
• Don’t be afraid to change it if needed. Yes, you have given it a lot of thought, and you are sure you have found the right KPI – but things tend to change. People change, businesses change, and focus shifts. The KPI should serve you, not the other way around. Following the wrong KPI is very disappointing; it’s better to, instead, replace the KPI from time to time and be more agile, than to work hard to move the metric on a KPI no one cares about.

The subtle art of setting goals

You have now selected the best KPI for your team that is measurable, and it encapsulates the value you are trying to achieve. So why is setting goals even necessary? Why is defining our KPI not enough?

The answer is: a KPI isn’t time-based, but a goal is. Time makes everything real and drives engineers to action.

Examples

• The KPI is weekly active users; 
• The goal is to double our current status by the end of the year.
• The KPI is the number of completed story points each sprint;
• The goal is to increase it by 30% in the next quarter.
• The KPI is the percentage of our code coverage in new features;
• The goal is to have 85% of code coverage for all new features in the next five deployments.

Of course, the goal you select is unique to your organization, your business domain, and to what you consider a success. However, there are some guidelines that can help you.

Select a realistic goal

Yes, we all want to be great and achieve as much as we can as fast as possible. However, setting unrealistic goals can devastate your team and your company culture. If the goals you are setting just can’t be achieved, two things will happen.

1. Firstly, your team will be highly frustrated and will feel a sense of failure. An unrealistic goal will demoralize everyone and make them feel that their hard work is not enough.
2. In the long-term, the impact of this will be even worse. People will stop caring. Because you recruited smart people, they will realize that no goal you set can be achieved due to unrealistic expectations. Your team will lose faith and completely ignore your goals. 

Another pattern that has short-term benefits but in the long-term will backfire, is to think,  ‘I’ll set a goal which is very high, but I won’t let the team know that if we just reach 50% of it, that would be great.’ This behavior might work for the short-term to achieve the current goal you are working on, but it hurts your team members and it will have the same negative implication on your team’s culture.

Easy-to-achieve goals are as bad as unrealistic ones 

You set manageable goals, your team achieves them every time before the deadline, and everybody is happy, right? What is the problem with that? Well, easy-to-achieve goals have two significant downsides. 

1. You need a paradigm shift and not more of the same. Let me explain this with an example. If you are looking to increase your team velocity and set the goal to increase it by 10%, it will merely lead to your team working 10% harder. This is not what you are after.

   Moving the needle isn’t about just working harder. If you really want to make something extraordinary, your goal should be scary at first glance. If you or your team are looking at a goal and immediately know how to achieve it, or your first reaction is ‘OK, we just need to work harder’, then it probably is not challenging enough. Aim for a paradigm shift that will move the needle and not simply result in an increase in hard work.
    

2. No sense of doing something great. If your team manages to achieve their goal every time, then everything becomes routine and nothing is exciting. When aiming high, it means that sometimes you fail. But this will be worth it once you actually accomplish a challenging goal. Think about your team’s sense of accomplishment and pride, and how awesome you and your team members will feel. Don’t rob this from them.

   Now ask yourself – which company you would like to work at? A company that pushes you for out-of-the-box thinking and, at times, makes you feel like you are doing something outstanding? Or a company where everything is always fine?

Stretch goals are great

Setting stretch goals allows you to challenge the team beyond your original and more-realistic goal. Stretch goals also have another vital benefit: the understanding that the better the results, the better the outcome. Most of the time, a goal is just a milestone for a specific time frame, and after reaching that goal, or the time frame passing, your team will continue in the same direction and you’ll set a new goal for the same KPI.

However, remember that a stretch goal is not the original goal. It doesn’t replace it. If your team has achieved the original goal, celebrate, and be as happy as if the stretch goal didn’t exist in the first place.

Don’t expect it to rise linearly

In most cases, once you ask your team members to do something extraordinary and reach a challenging goal, they may need to take steps back in order to move forward. In engineering terms, they may break things to ensure they run faster in the future. Due to this, it is entirely normal that progress on a graph will not be linear, and it might take some time before you can see the metrics improve. It may even decrease slightly at the start. But hopefully, after a short time, steady growth will be able to be seen. 

One size doesn’t fit all

Suppose you are running three different projects or running four development teams – each with its own KPI. Don’t think that setting the same goal for each project will reduce friction. Some believe that setting the same goal, for example, a 50% increase in every team’s current status, is fair and removes bias. This is not true. Doing so will only lead to frustration and confusion. The difference between projects is too vast for you to set one goal for every team. The team’s members, the challenges they will face, and the team’s experience in solving challenging problems, will all affect the selected goal; so really, no one size fits all. Yes, it means more work for you, but this way, you will have a better chance to achieve what you are after.

Your part as the leader

You have selected a KPI and set a goal for your team to reach in a specific time frame. Now, what should you do to support this methodology and embrace this results-driven culture?

Connect the team

The first thing you need to do as a leader is to connect your team to the KPI and the goal once they’ve been set. Explain to your team members once, then twice (and even more if needed) why the KPI you selected is important for the company and the team, and why reaching the goal you set will actually be significant. Done right, you will see the excitement and engagement of the team, and this will make your job easier when it comes to motivating them and gearing them into action.

Give ownership

Giving ownership to the team and taking a step back might be the hardest thing leaders have to do, no matter how many years of experience they have in leadership. If you have recruited right and have confidence in your team’s skills, it is time to take a step back and let them face the challenges ahead. Don’t think you know best for every decision. Instead, use your experience to ask the right questions, and help steer them in the right direction.

Support your team

Taking a step back does not mean fade out. Ask for progress. Have your team present progress periodically and guide them. You can start with doing this once a week and stretch it as you gain confidence in them. Set proper milestones along the way and check in with your team to make sure they are on track. Mentor your engineers in 1:1s and team meetings, and always offer them support. 

Demand accountability

Ensure that the team understands that once you empower them with the ownership of how to get the job done and reach the goal, they are solely accountable for it. Don’t expect your team, especially if this is a new team, to understand this immediately. Tell them that they need to take into account all the things that might go wrong in the planning phase. Ensure they understand that excuses like ‘We would have reached our goal, but the DevOps team didn’t deploy my Cassandra table’ or ‘We were ready, but my dog ate the unit tests’ will not be accepted. It is on them if it fails or if it succeeds.

Measure by the results

Measure by the results and not by the effort invested; this is so much easier said than done. It took me a long time to achieve it, and I am still failing at it from time to time. The results set you where you want to be and what your team is trying to achieve. And this is what you should give the highest value while evaluating your team members’ performance. I’m not saying that other criteria doesn’t matter, but results lead to an impact that leads to success. With a KPI and goals in place, measuring your team members becomes a little bit easier.

Give credit to the team

Some people will find this one obvious. However, I am sure that at least once in your career, you worked so hard on something that then your boss got all the credit for. I bet you can still remember how much that sucked and how demoralized you felt. Don’t be that boss. Don’t take credit for your team’s work. Finding the right KPI and setting the goal are the first steps in the right direction to your project’s success. But the team members that are actually doing the hard work are the real heroes: they deserve the credit for it.

Celebrate success!

I can’t stress this one enough: don’t forget to celebrate a reached goal. Once we achieve a goal, a new one will be knocking on our door and we will rush to start working on it. Our work is never done and there is always too much to do in a small amount of time. However, it’s so important to stop and celebrate. If this is a project that you are working on by yourself, go and do something you always wanted to do and make this day a memorable one. If your team is working on it together, celebrating success will give everyone a sense of accomplishment, set the bar for future projects, and provide real meaning for all your team’s hard work. Make stupid T-shirts, hats, and stickers. Go and have a drink together. Sing karaoke. Whatever! It will have a hugely positive impact on your team’s culture.

Conclusion

Moving the needle is not easy. Making great things is not easy. Selecting the right, measurable KPI that captures the value you are trying to bring; setting a challenging yet realistic goal; understanding the leader’s part in this unique culture – none of this is easy. However, when done right (and hopefully this article helps you a bit), these factors can reshape the way you think about the success of your projects and how to measure them along the way. You can make a significant impact on your organization by focusing your team and ensuring they are on the right path. Help you and your team achieve what really matters.

Good luck!