Three ways to remove bias from technical interviews

Want to know how to make better hiring decisions with less bias? Here are three steps to build a fairer process.

Have you invested a lot of work into sourcing candidates from diverse backgrounds for your team, but the candidates getting offers haven’t seemed to reflect the diversity in your recruiting pipeline?

If this sounds familiar, you’re not alone. It’s a common frustration for engineering hiring managers. If you don’t evaluate candidates fairly, you won’t end up with the results you want – no matter how much effort you put into sourcing. Every stage of the hiring process must prioritize equitable practices, and interviewing is often where unconscious bias has the most opportunity to sneak in.

Over the years as an engineer and a manager, I’ve conducted hundreds of technical interviews, and coached many engineers on how to conduct interviews. Based on that experience, I’ve learned that structured interviews, job relevance, and progressive questions are key measures for mitigating bias in technical hiring.

In this article, I’m going to describe the different types of bias, and offer three ways to remove bias from technical interviews.

Types of bias that can occur during technical interviews

No one likes to think that they are biased, and yet we all have unconscious bias that can affect how candidates are evaluated during an interview. Here are some examples:

• Affinity bias: When you have something in common with a candidate, it’s harder to be an objective interviewer. From a diversity perspective, this creates a vicious cycle where candidates who are similar to the people who already work at your company are more likely to receive high marks in an interview.
• Halo effect: Sometimes, one positive aspect of the candidate can cast a glow on the rest of their performance. It could be anything from their bright smile to their description of their volunteer work at a coding school. Without realizing it, you might overlook the fact that a candidate struggled through the technical interview questions.
• Horn effect: The horn effect works in a similar way to the halo effect. It is when one negative aspect of a candidate overshadows other more meaningful attributes. For instance, if a candidate stumbles early in the interview, an interviewer may continue to see them underqualified despite a strong performance in later questions.
• Confirmation bias: This happens when you look for information about the candidate that supports what you already believe to be true. For example, suppose you are interviewing a candidate and you worry that their résumé shows a lack of system design experience; you might unintentionally ask them harder system architecture questions than you asked of other candidates.

Before you say, ‘that’s not me,’ keep in mind that we also all have a bias blind spot, and studies have shown that the majority of people believe themselves to be less biased than others.

Three ways to remove bias from technical interviews

There is no way to remove bias from technical interviews entirely, without removing humans from the process. And human interaction is still important in technical interviews. You and the candidate might end up working together for a long time, so you need a chance to get to know each other!

Fortunately, there are ways to mitigate the bias pitfalls described above. Encourage your engineering interviewers to practice consistent, repeatable methods for developing and conducting a technical interview. Their unique perspectives will still be a key part of the process – they’ll just have a method for being more objective when it comes to evaluating whether the candidate is qualified for the job.

Here are three practices your team can implement to reduce bias in technical interviews:

1. Run structured interviews

Structured interviews use a consistent set of questions and evaluation criteria across candidates. Research has shown that structured interviews allow employers to better predict how candidates will perform on the job. And, an objective and consistent interview process reduces bias in hiring.

To create a structured interview, start by deciding what skills the team needs to measure, and then develop questions that all interviewers will use to measure those skills. This ensures that interviewers are not improvising, which can result in unintentionally giving some candidates harder or easier questions. Apply this structure to the whole interview: make sure that the order of the questions remains consistent, and train interviewers to ask each question the same way and give the same hints or prompts to every candidate.

2. Ask job-relevant questions

To avoid unintentional bias, it’s important to keep questions focused on real, on-the-job skills. This will ensure that candidates aren’t hired based on how much free time they had to study algorithm brainteasers on forums like LeetCode.

Another trap to avoid is asking interview questions that assume particular cultural knowledge, which might not be common knowledge for everyone. For example, I’ve seen a hiring team using a question centered around stats for a baseball team. Sounds harmless, right? Unfortunately, many non-American candidates struggled to understand the question because baseball wasn’t part of their cultural awareness.

Avoid these pitfalls by keeping your questions as close to your real work as possible. Try basing questions around simplified versions of actual problems your team has encountered in their own work.

3. Ask progressive questions

The order in which questions are asked during a structured interview is important. Ideally, all questions should be around one scenario, with multiple stages​​ that progress in complexity from easy to hard. This sets the candidate up to experience a small win right away, boosting their confidence and allowing them to demonstrate their true skill set in more challenging questions later on. These questions can be used across all candidates, regardless of their experience or skill levels, since you can use the same scenario and adjust based on how far they get.

Progressive questions that build on one problem are also inherently more realistic. In real life, code is constantly evolving. Engineers must take their initial solution and continue modifying and adapting it as business needs change. So, asking them to do the same during an interview simulates what they’ll be expected to do on the job. Asking this kind of realistic question helps keep an interview focused on evaluating a candidate’s job-relevant skills—rather than biased and unreliable proxies for skill.

Reflections

Technical interviews may never be totally free of bias, but there are best practices you can implement to reduce and mitigate human bias. These include: learning and recognizing the types of unconscious bias that impact hiring; implementing structured interviews; asking job-relevant questions; and using a progressive question format. By taking these steps to build a fairer process, you’ll position your engineering org to make better hiring decisions with less bias.