Patching the hiring pipeline in engineering

How can you recognize and stop the leaks in your hiring process?

In 2019, I joined the New York Pivotal office as an engineering manager. At the time, I had the opportunity to join the most diverse team I’ve ever been a part of. The team was over 50% women, nearly half the team had been born outside the U.S., and for some, English was not their first language. Our lead engineer was a brilliant woman, Natalie Arellano, who was soon followed by another, Yael Harel.

On that team was one of my first reports – an intern named Sophie. At the end of her internship, Sophie shared with me that this was the first time she saw herself represented in the engineering industry – through myself, Natalie, Yael, and other women on the floor. She finally felt confident pursuing a career in software engineering.

The diversity of our team that year was incredibly unusual and I was grateful to be a part of it. Research regularly shows improvement for individuals, teams, and companies that embrace diversity. Yet, under-represented minorities (URMs) in tech are still rare. Many technology companies have diversity initiatives citing increases in innovation and yet we still see inadequate representation. In an industry with so much promise, where does change need to come from in order for folks to join or return?

While there are surely more qualified people to speak on this subject than myself, I want to offer up my experience of how I collaborated with an amazing group of colleagues to try and improve the diversity of our teams. That work started where everything does: the hiring pipeline. The hiring pipeline starts when a candidate hears about a company and ends with that candidate signing their offer letter. Like any pipeline though, there can be leaks. Here are some of the leaks we found and patches we tried during my time at Pivotal.

The hiring pipeline

In early 2016, my peers and I were frustrated by the disparity in gender and racial representation in our engineering organization. I was the first female engineer to lead a project in our office, which was now over five years old. It was clear something was wrong and we had to get involved. We started by asking questions.

I learned that our hiring pipeline was standard for the industry: a generic software engineering role was posted to the usual job boards, and HR attended elite university career fairs to meet candidates in person. Candidates applied, went through a phone screen, performed a one-hour remote technical screening, and finally, a full-day in-person pairing interview. If they passed the last stage, they received an offer. Now, in my opinion, a standard approach produces standard results, and this standard pipeline was full of leaks.

Given that background, we were able to break down the recruitment process into its individual stages. At each stage, we found areas where we could better leverage our resources to find and acquire candidates. Next, with a 2×2 matrix on a whiteboard, we sorted our ‘patches’ by impact and effort. This process resulted in a prioritized list that we could try.

In those early days, there were not enough people to exercise every patch that we thought of. We understood that our goal was big. It would take significant time and resources, and we all still had our day-to-day responsibilities. We self-assigned what we could manage and touched base regularly to share updates or request help. We found opportunities to iterate on our process, and over time, others got involved so more leaks could be found and patches tried.

Finding and patching leaks

Had URMs even heard of us at all?

Our company wasn’t famous. We weren’t in FAANG and it was unlikely that URMs would have heard of us let alone sought out our job boards. That meant they needed to hear about us at fairs, through job boards, or word of mouth. Even more so, by restricting ourselves to career fairs at universities, we were missing out on candidates transitioning from other industries, graduating from technical bootcamps, or returning to the industry after some time off. At the career fairs we did attend, we were competing with huge engineering companies with deep pockets. The odds of attracting candidates were low.

So, we changed up our strategy. We leveraged job boards that focus on minorities. With a limited recruiting budget, we stopped posting to LinkedIn and StackOverflow, and redirected job posts to PowerToFly, Lane, and Handshake. Next, we attended career fairs with diverse student bodies. We reached out directly to graduates of local tech bootcamps tailoring our message to their personal projects and work experience, and made our case for why they were a great fit.

In the next few years, we tried a lot of different ideas. The New York office organized a co-op program led by Matt Horan with universities that had diverse student bodies (this is how we got Sophie!). The Los Angeles office hosted meetups for Write Speak Code and participated in a city-wide job fair attended by over 10,000 people. Our booth at the fair had a pairing station to show how we pair programmed every day. We also went straight to historically black colleges and universities to pitch ourselves. On one of these trips, our VP of Engineering, Mike Dalessio, and I traveled to North Carolina A&T State where we held a panel featuring alumnus that worked at Pivotal, and held a workshop to demonstrate our technical interview process. We did a live trial run with myself posing as a candidate. Thankfully, I passed and kept my job!

What was our job description saying or, rather, not saying to candidates?

The existing job role descriptions made no mention of paternity benefits, salary range, caregiver assistance, remote work options, or tuition reimbursement. The requirements section listed a university degree, when, in reality, we already had incredible engineers without one.

We decided to rewrite the entire description shaped by contributions from the diverse employees of Pivotal. We introduced a new nice-to-have section where the university degree was relocated to. We prioritized collaboration and inquisitiveness in our requirements. To find gendered and non-inclusive language, we applied Kat Matfield’s Gender Decoder. There were many more modifications we would have liked to have made, but it was progress.

How were we equipping our interviewers?

Well, we weren’t. There was no interview training and the interview had little structure. This was perhaps the most worrisome leak. Interviewers are critical to the pipeline. They need to know exactly which skills are necessary for the role and how to observe them. They need to create opportunities for a candidate to demonstrate those skills. Most importantly, they need to provide the same opportunities and apply the same evaluations to all candidates.

Edie Beer and Chhavi Kankaria took the first steps in creating formal interview training for the NY office. They produced a guide that detailed the role and responsibilities of an interviewer, how to prepare for the interview, what questions to ask and which exercises, as well as general principles for how to conduct themselves in order to try and ensure a fair interview.

Then there’s the catch-22 – was our own interview panel diverse?

While technical skill is important to objectively assess, there are important interpersonal skills that may be experienced by different individuals with a single candidate. Creating a holistic picture with different perspectives is critical.

As our teams grew and became more diverse, so did our panels. Eventually, we hit a new issue where we had to understand if/how we were burdening our minority employees with ‘glue work’. This should be a conscious concern for any diversity and inclusion (D&I) effort. Pivotal leadership started to address this by including D&I work in reviews, deeming it an essential part of making the company successful.

Finally, how was a candidate evaluated during the technical interview?

After the interview, the interviewers independently completed an evaluation rubric. This evaluation directly influenced how a manager determined what engineering title and salary a candidate would receive in their offer. The subjective framing of questions in the rubric exposed all kinds of bias. Biased feedback could cause managers to under-level minorities leaving them to decline offers or worse, enter the company already experiencing wage inequality.

We worked with interviewers to have them focus on assessing an individual’s ability and aptitude. We wanted to emphasize their ability to learn new skills, like test-driven development, while also taking into account the skills they already had. Chhavi and Edie then rewrote the rubric to reframe questions, remove opportunities for highly subjective conclusions, and introduce new criteria. The rubric facilitated the writing of more objective feedback by requiring specific examples to substantiate an evaluation.

There’s so much left to do

In describing where we patched our hiring pipeline, I’ve only touched on a subset of everything that we identified and fixed. Many Pivots contributed to improving the diversity, equity, and inclusion at our company, and I wish I could mention them all here.

As for Sophie, today she is a full-time software engineer at the company that acquired Pivotal. She works on the Buildpacks team building software that my current company, Doximity, uses. I’m regularly in her Slack workspace pestering her with questions. Someday, we’ll work together again on the same team. Until then, we’re thinking about how to bring about change to our respective companies: to get other women to engage with us and not only stick around, but help scale our efforts to fuel even greater innovation.