You have 1 article left to read this month before you need to register a free LeadDev.com account.
Your inbox, upgraded.
Receive weekly engineering insights to level up your leadership approach.
Estimated reading time: 5 minutes
As the hype cycle begins to peak, new positions like “vibe coder” roles enter the market, and interview parameters start to change.
In recent months, companies have been reimagining their processes for hiring developers to meet the AI moment. Experience with tools like GitHub Copilot and Cursor is increasingly being included in developer job listings, and many companies are evaluating how candidates use these tools throughout the interview process.
Company leaders say this shift is vital to staying competitive in the new business landscape being driven by AI. Even more so than experience with AI-assisted coding, they’re looking for developers who are open to the direction AI is taking the industry.
“Increasingly I think it’s a red flag to have a working developer in 2025 in a small startup like ours refuse to engage with any AI tools in any way,” said Mike McQuaid, a former principal engineer at Github and now cofounder and chief technology product officer at WorkBrew, who said he shifted his interview process after he started using these tools himself.
Now hiring: AI experience required
In a recent job listing for a Senior Engineer II, Backend role, security firm Abnormal AI (formerly Abnormal Security) included a requirement that simply didn’t exist not long ago: “Familiarity with AI development tools such as Cursor, GitHub Copilot, or Claude.” The listing further explained how the company is “building a team of top-tier engineers who are excited to use generative AI tools like Cursor, GitHub Copilot, and Claude to redefine how software is built – faster, smarter, and more efficient.”
Shrivu Shankar, a staff machine learning engineer at Abnormal AI who leads the GenAI team, said they included this on the job description to signal the kind of company they are and the people they want to work with.
“We’ve generally seen a filtering of candidates,” said Shankar, explaining that candidates apply – or don’t – based on whether a job listing’s emphasis on AI aligns with how they want to work. “A lot of people who use these AI tools are very productive with them. They are looking for companies that give them a place to be even more impactful with them.”
Moonvalley, a company building generative AI video models with fully-licensed data, has similarly listed experience with AI coding tools as a requirement on job listings, but it’s gone even further. The company recently opened up a role for a “vibe coder,” referring to the practice of writing code using natural language only through AI tools, which has made waves in the software development world in recent months.
Cofounder and CEO Naeem Talukdar said the role was still an engineering position, and the title was a technique to grab the attention of engineers interested in such an AI-forward role.
“That position was keyed up to meet folks who were really leaning into AI engineering,” he said, showing the increased importance the mention of AI coding tools – or lack thereof – is playing in job descriptions across the industry.
More like this
Technical interviews just got easier (or harder?)
After experimenting with asking candidates to use AI coding tools in take-home coding assignments, Abnormal Security brought the effort into live coding interviews. The assignment itself isn’t all that different now that AI tools are incorporated, but the expectation is. The company’s technical interview always included a “stretch question” that they would use to challenge candidates. Now with AI, they’re expecting everyone to be able to get that stretch question.
This shift in expectation is one of the clearest changes underlining how companies are evaluating engineers in the AI era.
“Whereas a year ago, when we were running these interviews, we would have had a much simpler [technical] task… Now, we’ll have a more robust app that we want them to build,” said Talukdar, explaining that candidates should be able to build something more fleshed out in the same amount of time.
Jesse Collins, cofounder and CTO of AI mortgage underwriting startup Friday Harbor, said that while he leaves it up to the candidates to decide if they want to leverage AI during their technical interviews, if they do choose to use it, expectations around the calibre of work rise. He gives all candidates a multi-stage exercise and expects those using tools like GitHub Copilot and Cursor to go further.
“Without AI, we might expect them to make it to stage three. But with AI, we might expect them to make it to stage five [out of six total stages],” he said.
For developers interviewing at WorkBrew, it’s a similar story.
“I do not personally require anyone to use AI tools but, if they choose not to, they are accountable for their productivity level compared to those who do lean into them,” McQuaid said.
London • June 16 & 17, 2025
Speakers Gergely Orosz, Camille Fournier and Lara Hogan confirmed.
Evaluating candidates in the AI era
The usage of AI coding assistants in interviews adds yet another dimension to evaluate candidates on. First and foremost, company leaders want to ensure candidates aren’t over-relying on the tools without understanding the logic behind code suggestions. At Abnormal AI, for example, they ask candidates to go through all the different edge cases and share how they’d address them.
“You have the naive candidate who uses only AI and says ‘here’s what the AI told me. These are the edge cases.’ And then there’s the hybrid approach we look for, which is when you have the AI and interrogate what it’s saying to really iron out all of those edge cases until they’re satisfied and they’ve also reasoned through the problem,” said Shankar.
Overall, AI’s ability to produce code is causing a massive sea change to what developers actually do on the day-to-day, and thus how foundational coding skills are perceived in interviews. Company leaders are now paying less attention to algorithmic knowledge, and the LeetCode-style interview may not last much longer.
Hiring companies “care less about purely algorithmic solving, because a lot of that’s being done by the AI,” Shankar said. “Now, you want to see how they’re able to design something, but then also implement it, and then test it.”
While AI tools add new dimensions to evaluate candidates on, companies continue to look for reasoning skills, resourcefulness, and creativity. AI simply adds another avenue for these qualities to be expressed and explored.
“If they’re very resourceful with things – looking at Stack Overflow for answers, Reddit, reaching out to their peers – I see that as a good sign. And I see AI as another tool in that scenario,” said Collins.
While Friday Harbor leaves AI use in an interview up to candidates, Collins admits the firm will increasingly favor those who do.
This plays into a new paradigm where openness to using AI is perhaps the most important culture factor.
“It’s more indexing for, ‘are you an engineer that wants to be AI-enabled? Are you an engineer who’s excited about these tools, who’s open to using these tools and learning about how to improve? I think that’s the biggest piece, because everybody’s still figuring out the right workflows,” Talukdar said. “Nobody’s an expert yet.”
