How do you identify great engineers when hiring?

‘Should I hire the engineer in front of me?’

This decision is high-stakes and costly. It can make or break teams. The clock is ticking, and you only have 60 minutes. How do you make the best use of this time to identify a great engineer?

What does a great engineer mean to you?

The first step is to identify what you’re looking for in the engineer. Depending on the stage of your company, the current skillset of your team, and the product roadmap, the definition of a great engineer varies. Your company’s values and its engineering growth framework are good reference points. Another advantage of leaning on the engineering growth framework is that you could calibrate the candidate against your current engineers’ skillset and compensation.

At Rasa, where I work, we evaluate engineers based on their execution, product sense, teamwork, influence, and technical skills. The questions I prepare gather information in these competency areas to increase our success rate in identifying the best talent. I’ll leave out ‘technical skills’ from this article, as that deserves a separate article on its own.

Preparing for the interview

Set your expectation

I start by writing down the expectations I have for the role in each competency area. Then I come up with the corresponding questions that can test the particular expectation.

Stay away from hypothetical and abstract questions

I tend to avoid hypothetical or abstract questions for an engineering role. For example: ‘Name your greatest strengths and weaknesses’, ‘If your colleagues were to use three adjectives to describe you, what would they be?’, and ‘What would your last manager say about you?’ This class of question is irrelevant to the role and how the engineer will approach their responsibilities. It can even introduce biases to the interview. These questions favor personalities and cultures that are more comfortable with presenting themselves.

To gauge the engineer’s strengths, asking questions that target the competency areas I’m looking for is more effective. And instead of asking for the engineer’s weakness directly, I find it more helpful to understand the engineer’s self-awareness and willingness to improve. I will ask, ‘What was a piece of feedback that was hard to take but helped you improve?’, or ‘What is the most challenging part of being in a senior engineering role for you?’

Review the candidate’s resume

As well as my prepared set of competency-based questions, I also come up with questions on the candidate’s experience and career path: how they made decisions to join or leave a company, the kind of experiences they highlight in their resume, and their takeaways, are valuable insights for me to assess a fit.

The secret sauce

Is your candidate the best fit for the role or are they just good at interviewing? As the interviewer, how can you tell? The secret sauce is in your follow-up questions. To get to know a candidate in-depth, you need to lead the interview into a conversation that matters. Good interviewers dig deep and their questions look a lot like coaching prompts. In 1:1s, managers make use of coaching techniques to understand their reports and help them reflect on their work. The same can apply to an interview. You know you have asked the right questions when your interviewees have to pause and think about the response! Let’s look at an example of how you can make use of your coaching skills to nail an interview.

• Active listening. Be fully present in the interview. Build a rapport with your candidate.
• Echoing. Summarize your understanding and align with the candidate. ‘I hear that handling conflict is hard in your team.’
• Follow up with open-ended questions starting with ‘What’. In coaching, open-ended questions starting with ‘What’ help folks open up the most, followed by ‘How’. ‘What is challenging for you about handling team conflicts?’, and ‘How did you navigate the team dynamics when your peers got into a conflict?’. At the bottom of the ranking are ‘Why’ questions. ‘Why did you do this when you were in the conflict?’ ‘Why’ questions invoke the unpleasant vibe of an interrogation and trigger defensiveness.
• Be curious. Develop a genuine curiosity for the candidate. Dig deep into what is being said and what is not being said: ‘You mentioned that you weren’t happy with how you handled the conflict. What will you do differently next time?”

While you’re picking an engineer to join your team, your interviewee is also assessing whether they want to work with you and your team. The coaching method helps you bond and create a rapport with your candidate. When the time comes for a candidate to decide between their offers, the relationship you have built with the engineer may turn the tide!

An example preparation document for a role

Let’s look at an example set of expectations and questions for a senior engineer for a product/feature team.

Execution

Expectations:

• Stage projects into well-defined milestones as minimal viable products to get early user feedback.
• Ask the right questions and move forward projects with uncertainties within the context of their team. 
• Coordinate development and monitor dependencies in the product and other teams.

Example questions:

• How do you structure a project working in a team?
• Tell me the last time you had to make a call in your project without complete information.
• When you were working on project X, who were your stakeholders?

Product sense

Expectations:

• An expert on how users use our products with a deep understanding of their pain points.
• Continuously propose solutions backed up by these insights and technical expertise.
• Work with stakeholders to set priorities and scope across multiple competing projects on a team level.

Example questions:

• Tell me about the product of your current company. Who are the users? What pain point is your team solving for them?
• You worked on this project X. How did you evaluate the project’s success?
• You were pushing for initiative Y. What were the tradeoffs for spending time on Y instead of the other projects in the pipeline?

Teamwork

Expectations:

• Ensure expectations within their team are aligned between team members and external stakeholders.
• Lead discussions in the team; is able to create consensus and buy-in.
• Help create an engaging, inclusive, and equitable environment in the team.

Example questions:

• Please share with me an example: how do you adapt your communication with different stakeholders?
• What is the hardest thing for you working in a team?
• Tell me one time you had to handle a strong disagreement.
• What kind of environment enables you to do your best work?

Influence

Expectations:

• Challenge yourself and your team to be exceptional. Share knowledge and give direct feedback to your colleagues.
• Participate in hiring.
• Have a strong appetite for learning.

Example questions:

• Tell me a time you got tough feedback from someone.
• Tell me about an engineer you have mentored and your approach.
• When you interview engineers, what do you look for?
• What is the best outcome you expect from the role?
• What is a skillset you’re currently working on to improve?

Conclusion

Building a strong team is an engineering leader’s core responsibility. To identify the great engineer for your team, first set your expectations on the ideal candidate. The next step is to come up with specific questions to assess the fit. Staying away from hypothetical and abstract questions, reviewing the engineer’s resume, and practicing coaching techniques have been helpful for me to get to know the candidate in depth. And don’t forget: the interview is also the window of opportunity for the candidate to assess you and your team! If done right, the interview experience will be the best pitch for you to attract the greatest talent.