Beat 2,000 Resumes With Voice with Daniel Chait, CEO & Ophir Samson, Head of Voice AI at Greenhouse

Maybe we'll take the, the challenges first and I think this happens on both sides. It's like the job seeker side and the recruiter side. But maybe let's do job seekers first. Opher, do you wanna talk us through what are the challenges for job seekers right now? 

Ophir Samson: Yeah, I echo what Daniel said.

It's a really difficult time to be on the job market right now. If you are applying for a job that you've seen a job post for, and you send in your application, you're likely competing with 1,000, 2,000 other applications, 60, 70% of which are bot applications or fake applications, 100% of which are AI-generated resumes.

And so if you're a good candidate who's applying for a role that you're genuinely interested in getting, you're gonna bring drowned out by just so much noise. And so your likelihood of being noticed by a recruiter is minuscule. It it's tiny. And so- that's one half of the problem that we're trying to solve here.

How do we give good candidates who are genuinely interested in a role that they're applying for the opportunity to be heard, to be understood, to stand out, and to dramatically improve their likelihoods of making it into that job than what the current state allows them to have? And we'll get more into the sort of how we do that and, how we're using voice later on but that is, the problem that we're trying to solve.

Richie Cotton: Absolutely. Certainly standing out in a field of a thousand other resumes, especially when you've got bots that are tailoring things exactly to, to the job description 'cause they don't need to be a real human or a r- tied to a real experience, then yeah, that's certainly a challenge.

But so h- having a thousand applications, that sounds like something that's good for recruiters, but I can imagine it's also a, a challenge. Daniel, do you wanna talk me through, like, where the problems come in? 

Daniel Chait: Yeah. It's interesting because you're right. Typically you would think, oh, the companies are out there looking for talent, like the more talent that they have access to, the better.

But it turns out, no. It turns out that kind of the mirror image of the problems that job seekers are facing are also being experienced by companies that are looking to make a hire. So what we're seeing across our customer base is typically they'll open up a job, and they'll be inundated with hundreds or thousands of applications that they have very-- that they have d- difficulty telling the difference between.

And so kinda like Ophir was saying, a lot of them are AI-generated auto applies, and everyone's resume and is kinda being polished into sameness by the same AI technology that we're all using in other areas of work. And so those unique candidates that are best for the job just aren't standing out really in any way from all the rest of the noise.

And so when you have a thousand applicants to pile through, where do you start? Like, how do you even tell the difference? And how do you even know of the people who applied, surely there are some who very much want this job, who've thought hard about it, who've targeted your company as a great place that they wanna be, and are like, "This is the job they're most interested in."

And then there's a bunch of other applications in that same pile that look very similar, that have never heard of your company, that didn't even really apply intentionally, that maybe a bot applied for them. And so the companies really are stuck not knowing where to start or how to even get through that whole pile of resumes either.

And so it is an unusual time in the sense that historically, you had either the job market would be strong or weak, and one side or the o- or the other would feel like they have the upper hand. But this is really the first time that I can remember when both sides were just dissatisfied and didn't see it getting better.

Richie Cotton: Yeah, it's a bad situation where both sides of this are struggling. So I'm interested that a lot of people don't know where to get started. Okay, so let's do, see if we can figure this out. Where do you start Ofir? 

Ophir Samson: If you're applying for a job that you know that you want and you're and you have the opportunity to be interviewed by our voice AI system, which we call Greenhouse Voice AI, we recommend starting there.

And the reason for that is your likelihood of... If you're a candidate your likelihood of success is dramatically higher if you take the effort to do that than if not. So we're seeing that candidates who do a voice AI interview are three times as likely to reach a final round than candidates who haven't done the voice AI interview.

So I think, a lot of candidates are justifiably skeptical of AI in the process, whether it is AI interviewers or other implementations of AI, just because there's a lot of AI slop out there. There's a lot of really bad tools. But when it's implemented correctly and it's deployed by a recruiter or a company that is really thoughtful about how to give transparency to candidates, then, and you are on the receiving end of that, then we really encourage you to do it because you will be you will be noticed in the way that you won't be noticed otherwise.

Yes, it's a bit more effort. Yes, you have to put in 10, 15, 20 minutes of your time and think through some answers. Who's not doing that? The 1,500 other bot applications who, are still competing for the attention of the recruiter. It's a way of getting to know you as a candidate in a far greater level of detail than the recruiter will be able to understand just based on your resume.

And that would be the starting point that we'd recommend. 

Daniel Chait: And maybe if I could just pick up on, on something that Ophir said, because, I think part of what's so frustrating about being a job seeker right now is there haven't been great things for you to do to give yourself a better chance.

Like, why are they sending out so many applications? Why are they signing up for these $19 a month auto app- auto apply AI tools? That's what, that, that's what they can do, and they just wanna get a job. And there hasn't traditionally been, until now, an effective way for them to put more effort in and give themselves a better chance at getting a good job.

Everybody understands, I think most job seekers are okay with the fact that good jobs are really competitive to get. You're not gonna be the only applicant. I think the challenge is when it's not-- when it goes beyond being competitive and you're just lost in a sea, kind of a resume black hole what do you even do to get out of it?

Like, how much-- how do you if you wanna try harder, like, how do you try harder? And I think, so what Ophir is talking about is, the voice AI is a great opportunity for this person who's applying for a job who really wants it to put in some effort, yes, but in ways that help get past the resume and really show who they are, what they're good at.

And I think we've all had the experience when writing a resume of gosh, I only have this limited space to put my whole life into. It has to be in these bullet points. It's not really saying what I wanna be able to say. And so I think the opportunity to speak and to tell your story and to answer some questions and to, by the way, ask your own questions d- really does give people a new and better way to show what they're all about.

Richie Cotton: Being able to tell your own story and explain why you care seems like a, a useful thing. I I'm curious a bit about the, the logistics, though. So whereabouts does this voice the AI voice interview fit into the hiring process? Is this at the point where you submit your application? Is it instead of a screening interview?

I presume you don't have a final round interview with AI. Yeah, Ophir, do you wanna tell us where it fits into the flow? 

Ophir Samson: Different companies implement it in different ways. So the, the majority of the use cases are that the company will receive a bunch of inbound ap- applicants, so they'll get 2,000 applications, and then the recruiter will choose a subset of those to invite to to a, a voice interview.

Now, the word interview is actually a... it's an industry term. It's a bit of a misnomer though, because It's not instead of an interview. So w- when I originally built the company that Greenhouse acquired, my my, my goal was to not replace the human interview, but to supplement the resume and to supplement the written application.

And so that it comes in a point of the pi- pipeline where the recruiter wants to know more about you beyond what you've just written, and is choosing who are the 30, 40 people that they want to spend their human time with. So it really is a way of like supplementing the resume, and is con- normally very early in the pipeline.

There's some recruiters and companies who use it a bit later on when they want to understand more about ca- candidates in between different stages of interviews. But it really is normally at the kind of front end of the application. 

Richie Cotton: Okay. Yeah, that, that makes sense, having it earlier on rather than later.

Okay. So I s- I feel like having being interviewed by AI, it's gonna be polarizing, so you're gonna be some people are very enthusiastic, others are gonna be very concerned about this, maybe feeling like it's just a sort of a cost saving measure on the HR department. Do you wanna talk me through which of these concerns are legitimate?

Daniel Chait: Yeah, look, I, it's people have the concerns that they have, and I'm not here to say that they're not legitimate. I totally understand. I'm extremely empathetic to these worries as I started this conversation by saying. So yeah, we've interviewed we've surveyed many thousands of job seekers and asked their opinions and collected lots of data about what are your attitudes and what are your fears and what are your worries about these technologies.

And generally they come under three kind of big buckets. The first thing people worry about is, the experience is gonna be really clunky and awkward. We've all had that experience where you're shouting representative, at the phone company, trying to get through to someone, and it just feels like these technologies historically have not felt great.

I think the second is that people wonder, Like what's actually gonna happen out of this conversation? Is this gonna be a machine that's gonna make a decision arbitrarily based on something I happen to say during the conversation, or is it gonna be fair? And what's really like the decision-making sort of framework behind all of that.

And then the third kind of concern that people have is like you said, like who's this really built for? Is this built for me as a job seeker or is this built in some kind of way to save money or dispatch with me or keep me away? And so what I can tell you is when we went to look for the right solution to take advantage of this amazing new technologies and innovations in AI and in voice technology, those three things were very much in our mind.

And when we connected with Ezra AI Labs and started to hear what Ophir had built, it was really apparent that they had built a technology that was first and foremost meant to give a very comfortable and naturalistic experience to the job seeker. That it wasn't gonna be that kind of stilted thing where you're interrupting each other constantly or some like creepy like robot cartoon talking to you or something like that.

That it was gonna be built on a solid defensible foundation of structured hiring with clear decision-making rubrics and evidence that's being presented to humans to make decisions. And that it's gonna be really candidate-centric, that it's gonna help the candidates ask their own questions, interview the company and about what they wanna know about, and really transparent about, "Hey, you can opt out of this if you want.

Here's what's gonna happen next." And what we found is that when you take care of those main concerns of candidates, it really does overcome most of their worries, and the feedback that we've collected is really excellent. The customers or the job seekers of our customers have rated the experience very highly.

And so I get the worries. We have those very front and center. But again, I feel like The choice is, do you wanna have your resume go into a black hole never to be seen again, or do you wanna have the opportunity to present who you are and ask your own questions? I think when people are confronted with that choice, they're very enthusiastic about it.

Richie Cotton: Okay, yeah. Certainly I think everyone's had a terrible experience with those phone systems where it's like, "Press seven to speak to this department maybe." Yeah, y- you certainly don't want that in a job interview. But on, on the other concern about the decision-making process what's the output from this AI interview?

Is it just y- th- there's a transcript and a summary to be given to a human to make a decision, or does the AI make a hiring decision by itself? 

Ophir Samson: Yeah, so hiring, So a human always makes a decision, so we're very clear that there is no, no AI will kinda make a decision about the fate of a candidate.

That is kinda core to our principles. Another part which is core to our principles is to eliminate bias and to do this in a highly structured way. So we have independent third-party bias auditors that evaluate us every single month and test our systems across tens of thousands of interviews to make sure that we are pro- we, we do, we are conducting the evaluations in a bias-free way across every single protected class.

We also, what... The, we see this very much as wave two of AI interviewing, with wave one being the clunky stuff that, that Daniel mentioned earlier. And a part of kinda wave two is Not just like taking the transcript of a interview and giving it to, an LLM and s- and asking like, "Is this a good candidate or not?"

And just like getting a score of 68% great candidate. That's not what we do. We... Every single candidate is evaluated relative to the specific rubrics that the recruiter set before the interview began. So the inter- the recruiter will set three, four, five, however many questions as they like, specific questions that they want every candidate to be asked, and every candidate will be asked those questions.

And along with every question, they also set rubrics for what determines a great, a good, an okay, and a poor answer to each one of those questions. And they also describe to Voice AI what they're looking for in an ideal candidate. So we have a ton of structured rubrics information that is specific to that recruiter, to that job, to that company, that culture, that team through which to evaluate the specific candidate that took an interview for that role.

So every evaluation is done relative to that structure, and that is a way that we guarantee consistency across, every single interview for any different role. And the other kind of benefit of that is, or the other implication of that is if you're setting up an interview for role A, that evaluation, that setup will look completely different than role B, and which will look different from role C.

It's a different company. So every, every one of our every use case is a snowflake all built on the foundation of highly structured, highly rigorous, kind of bias, bias-free AI that is not making a decision, but giving the recruiter the information they need to make a really thoughtful decision.

Richie Cotton: Okay. Yes. Certainly consistency's a really big problem with hiring. If you're hiring for lots of the same role over and over again, I'm thinking something like like account executives maybe, where it's a fairly similar role that you're gonna wanna hire lots of times. Is that the sweet spot for this kind of this, this sort of tool?

Or are there any particular roles where this voice interview works better or worse than others? 

Daniel Chait: Not necessarily. I think to, to Ophir's point about the kind of generations and the evolutions, I think there was a time not so long ago when companies thought I'll have AI speak to, a blue collar worker for like a, food service job," or something to that effect.

"But I would never have them talk to, an employee that's gonna come work in the office as, say, a salesperson." I think the technology has gotten really good now. The experiences are g- are really good to the point where that's not really the thinking anymore. And so it's really more about, the company wanting to make sure that they don't overlook the talent that's in, that's already in their pipeline.

People who, as we talked about, are applying to jobs and getting lost in the shuffle. They want to find those great talents. So our customers are using it to hire salespeople engineers, people all over the organization. And it's really more about wanting to take control of their hiring experience and bring some science to it versus it, it is about necessarily the type of candidate.

And if you think about what Ophir was saying a minute ago, just to build on that, like there are decades of science that have shown the most effective ways to make hiring decisions are also based in fairness. That idea of having a clear assessment rubric for this role that ties directly to the requirements of the role.

Doesn't have to do with your credentials. I don't really need to care about, did you go to a fancy school or things like that. I want to know, can you do the job? If you build out a rubric, meaning a, a, a clear set of identified questions that every candidate's gonna be asked in the same order, and that the answers are gonna be evaluated in the same way across every candidate, you're gonna get better decisions, you're gonna get fairer decisions.

And what I was most impressed with when we looked at what Ophir had built was before we even get to the point where we're talking to a candidate The technology interviews the hiring team and asks them to build that whole framework out. And so these are the things that our best customers struggle to do, because it takes a huge amount of time, and doing that consistently is something that you have to be really thoughtful in order to do.

And that now with AI, automatically every time you open up a job, the team can just build out a fair scorecard. Here's the questions that everyone's gonna be asked. Here's the way everyone's gonna be evaluated. And the AI can then go dutifully follow that plan with every candidate, giving everyone an equal chance.

And I think that's, that sort of behind the scenes setup is something that might not be as visible to the job seeker, but I think should give people a lot of comfort that when you do approach technology through the lens of fairness and good decision-making, that we actually can make things much more consistent and fair than when people have been doing it, which we've known for forever have created lots of hiring problems.

Richie Cotton: It seems like when you do a lot of planning up front to decide what you want before you start hiring, you're gonna have a better outcome. I think that's the same i- in many fields. If you plan things first, you're gonna get a better outcome, and so the-- I- I know this is related to the idea of structured hiring that you're a big fan of, so we'll perhaps we'll come back to this later.

But while we're on the subject of fairness I think there's gonna be some issues around fairness for voice AI, I'm sure. 'Cause I spent a lot of time trying to get people to come and speak on this podcast and in webinars and things like that. It's a very divisive thing. Some people love it, and other people struggle.

So perhaps if y- English is not your first language, you're talking different language, it makes it harder. If you've got a stutter or a lisp or some other variant speech, then it's gonna be tricky. How does the voice AI deal with non-standard speech? 

Ophir Samson: Yeah, absolutely. So it's something that we take very seriously, and it's core to our engineering from literally day one of writing code for this.

So firstly we support interviews across many languages. We have the world's most cutting edge transcription models and k- speech to text models that work extremely well across different dialects and accents, and we've tested it in highly, we've really stretched the limits in our testing of accents and dialects and across, thousands of interviews.

But- As good as the technology is, there's no technology that's perfect. The question then becomes how are you making sure that you're not negatively judging someone based on those characteristics? Even if protect for 99.9% of what you've said and in the technology upfront, you also wanna make sure that you're not negatively assessing someone if in that 0.1% of cases that fall through the cracks.

And to that end, we only judge the content of what someone says. So we do not judge any kind of communication style, any kind of fluencies or disfluencies none of that will impact the score that a candidate gets. Again, that was absolutely critical to the value set that I had on day one in building Ethra and the value set that we have now at Greenhouse.

Richie Cotton: Okay, so it's largely being scored on the transcription rather than the actual voice itself. 

Ophir Samson: Yeah. It's it's scored as a literal comparison between what the recruiter wanted to see, like how they define their rubrics of what makes a great answer, bad answer to each question, and what the candidate actually said.

So the content of that. Y- there's lots of thi- obviously recruiters care about speaking style and communication style and presentation and things like that. Those are the kinds of judgments that The human should make, right? So there's-- the-- we bifurcate the problem between what are the kind of repetitive tasks that the, that can be handled by an AI really well with cutting-edge technology.

Handling questions that come from candidates about the salary, the benefits do I need to be in the office every day of the week? So the Voice AI system can handle those questions and does across like hundreds of interviews for a role. That's not the-- that's not a good use of the recruiter's time.

A better use of the recruiter's time is judging things like communication, culture fit, things that are like really human, and those are things that we don't believe that a the AI can or should do, and so that's not something that we touch. 

Richie Cotton: Okay. All right. I like the idea it's focused on just assessing content against like the predefined criteria.

How does it work when you've got technical terms? I'm thinking the classic example is like w-with pharmacists or something, where there's a lot of drugs which sound very similar but have completely different effects, and if you prescribe the wrong one, you get the name slightly wrong, and then someone dies.

So I can see how like little changes in the, the words, if the transcription is not perfect for technical terms, then it can have big impacts on the scoring. How do you deal with that? I guess you're fair game. 

Ophir Samson: Yeah. So we've done a huge amount of testing across industries that have highly technical terms as well, and for-fortunately we're not prescribing drugs, so there's a different, different kind of set of standards.

But we have in our code, in our kind of engineering approach, we have multiple ways of transcribing and, extracting the meaning of what someone is saying so that we minimize the rate of error. There is no technology in the world that won't ever g- you know, misunderstand words, in the same way that humans won't either.

And that's one of the really important things for us at Voice AI, to not overpromise, do not overpromise the capabilities of AI, and that is why we want to keep the human in the loop, because a human recruiter for that healthcare company, that pharmaceutical company, like he or she will be the one that actually recognizes specific terms of their drugs just in case the transcription models don't work as, as planned.

So it's a multilevel approach of, excellent engineering. I feel like we have the best, or I can show we have, you know- the best transcription models out there, plus a le- level of of human verification to make sure that the kind of end-to-end product is trustable. 

Richie Cotton: Okay. Yeah, so if the human's making the final decision at the end, this rather than the AI, then you've got a bit of grace for, okay, if the, even if the transcription's not perfect, then you can correct for that.

Okay, nice. Suppose you're interested in this. How do you know whether it's working well? What sort of metrics for success are there, and how does using AI rather than the human affect those metrics? Daniel? 

Daniel Chait: Yeah. As a company, thinking about this investment of technology, what they're thinking about are a few things.

First of course, they're thinking about efficiency, and they wanna make sure that they can run their hiring processes, as efficiently as possible and get through all of the applications, in a timely way, for sure. But more importantly than that is what they really are looking for is access to top talent, is the, the customer wants to know when they open up a role, can they fix this problem that they have right now where there's not a, a reliable signal in their pipeline about who's good and who isn't?

There's not reliable amount of trust in the pipeline about who's genuinely interested in the job versus the ones that aren't. And even about misrepresentation, who in the pipeline might be misrepresenting themselves versus who are the, vast majority of legitimate, job seekers out there.

And the customers that we are working with, they're thinking about those things. They wanna know, they want some assurance that the candidates that they're gonna spend their own time with pass those tests. 

Richie Cotton: Okay, yeah. So certainly I can see how saving a bit of money upfront is useful, 'cause it helps you scale, find more candidates.

But really, like the, the big test is, are you hiring good people? So it's gonna be like measuring like the, first performance review scores or something after, after people get hired. I- is that about right? 

Daniel Chait: Yeah, qua- quality of hire is notoriously difficult and, I think we could spend the next week on this podcast, arguing about different methodologies.

I think every company and, every organization will think about that question a little bit differently. But however you think about your quality of your talent surely it starts with not overlooking most of it, not discarding most of the applicants but making sure that you give everyone a fair shot.

Richie Cotton: Okay, yeah. So having better top of funnel is gonna hopefully lead to better quality further down the line. 

Ophir Samson: I'd say also that other things that, Recruiters track is of the candidates who've been interviewed by voice AI or spoken to voice AI, what's the pass-through rates of those candidates to subsequent rounds?

Because if it is, fundamental, a core hypothesis here is that it can find the needles in your haystack. So if it's actually finding you needles in your haystack, then those needles should be progressing to subsequent parts of your pipeline at a higher rate than the rest of the haystack.

And it-- they are. There's progressing at three times hi- higher rates than candidates who haven't been interviewed by by the system. Also, they look at the-- a total amount of time spent by recruiter on a hire, and that is going down by about three and a half times. Recruiters are spending about three and a half times less per hire than they were before.

So there's a huge amount of time savings. And then last thing I mentioned, in addition to, everything that, that Daniel said, is that candidate experience, and Daniel touched on this earlier, but we're seeing currently just over ninety percent of candidates are rating the experience as excellent, and only around somewhere between six and ten percent of real candidates are dropping out.

So the opt-out rate... So every candidate is given the opportunity to opt out if they want. It's never forced on a candidate, but the opt-out rate is also very low. So recruiters track everything that Daniel mentioned, everything I said to be highly thoughtful about the implementation of this.

Richie Cotton: So you mentioned you, you want to compare performance of people who are using this versus not. Is it typical to do an A/B test then? Do you just put I don't know, twenty percent of your people through this to begin with and see how that works? 

Ophir Samson: Yeah. Ma-many of our customers do that.

They, they take a, an open job that they have. They get an inbound pipeline inbound of, say, 1,000, 2,000 applications. They split them into two cohorts, with voice AI and without voice AI. And they see that in the cohort that uses voice AI, they're spending far less time, as, as I mentioned, about three and a half times less time per hire than they were in the cohort that didn't have voice AI.

Richie Cotton: Okay. Nice. All right. While we're talking about methodology can we go back to this idea of structured hiring? Daniel, do you wanna talk with your what does this involve and what the steps are? 

Daniel Chait: Yeah. The basic idea of structured hiring is old, right? It's tried and tested science that basically says the way to run a hiring process is to define upfront the criteria that you're looking for.

In Greenhouse, we call that a scorecard. Here's a set of attributes that we wanna assess candidates on to know who to hire, and then to define for each of those attributes one or more different kind of interviews or assessment process you might go through in order to rate the candidates on those attributes.

So for example, for this job, we wanna know if you can, problem solve in a certain way. I wanna know if you have certain technical skills. I wanna know if you have certain behaviors that you're gonna exhibit, and I'm gonna design certain questions to get at that. And then the key is you put every candidate through the same exercise, through the same set of questions, and evaluate them on the same scorecard.

If you do that, that leads to much more reliable and much more valid data to help make decisions on. And the reason we do those things is because, it's really hard, it's really hard to make decisions about, about, about people in a job, in a hiring context. Our brains are hardwired for all of these kinds of biases.

And, people hear bias and they have certain ideas in their mind about what that does or doesn't mean, and unfortunately some of that's been politicized. But it's pretty simple if you think about the fact that our brains are very good at rapid pattern matching, right? These are like things that we've evolved over millions of years to be able to do.

We all live in the world, and we go out there and we have experiences and those experiences give us certain associations that may or may not lead to the best decisions but they happen really quickly and almost unconsciously. And so one fun example I always like to give is, if I ask you to picture a genius You know, most likely most people I talk to in, in, in America or Western, Western Europe will picture Albert Einstein.

Richie Cotton: Yeah. Definitely. 

Daniel Chait: And so it's like with this very simple thing, it's oh, I associated this white-haired guy with a buffy mustache and the concept of genius. Or if I 

Richie Cotton: ask you to look at- I love the association between mustache and genius. 

Daniel Chait: Exactly. You should be a huge fan of that.

Or if I ask you to look at an optical illusion, most people will see those optical illusions and their brains will be tricked, but it's not 'cause your eyes are broken. It's because your brain is finding these patterns almost against your will and helping you a- and leading you to make these decisions that are wrong about what's actually on the paper.

And so I give those examples to say it's the same when you go meet another person, and you immediately have a set of associations about what they're gonna be like to work with, whether or not you want to or not. And so if you don't have that basis of structure underneath it, what you're left with is, I don't know, whatever the boss thinks or, "Hey, you went to the same school I went to," or some of the things you were asking about Richie earlier.

Maybe the person has a speech impediment or maybe there's something about the, the way that they're dressed. There's a big debate right now on LinkedIn, if you're following it, around iced coffee, and is it okay to bring iced coffee to a job interview. And the attitudes people have about this are all over the map.

And the answer is is it a good thing or a bad thing to bring iced coffee shouldn't matter. It's obviously not a criteria for almost any job, and yet people have really strong feelings about whether bringing iced coffee is appropriate or not for a job interview. And so hopefully that tells us that no, if you don't put iced coffee as one of your criteria on your scorecard and you don't test specifically for that, you'll make better decisions.

And so that's really the heart of what we mean by structured hiring. 

Ophir Samson: And just to add one, one point. From a technical standpoint within the, the evaluations in the voice AI, it doesn't-- we don't consider the resume. So two candidate- one candidate who's been to, who, who's been to an amazing Ivy League university, and another candidate who hasn't been to that kind of educational experience, hasn't had that e-educational experience, if they're answering the questions well, equally well, they get the same score.

It doesn't consider where you've been to. So yeah we kinda manifest, like, all of the philosophy of the, the conceptual philosophical frameworks that that Daniel mentioned in-- literally into the code that we write. 

Daniel Chait: And I was actually talking to a recruiter in South America who was talking about this exact issue, that she was helping really talented technical talent in South America try to get jobs in the US.

And it was dif- one of the difficulties that she faced was that most of the recruiters and most of the teams hadn't heard of these universities in Argentina I don't know off the top of your head if you could tell me what the top technical school right now is in Peru. I certainly can't.

And so the point is, if you can answer the questions correctly and show what you're capable of, it shouldn't matter that I've never heard of the school you went to. And so that's really the heart of this idea of structured hiring that we're trying to bring in. 

Richie Cotton: Okay. Yeah, and certainly there's so many different cognitive biases we have and so much weird information around hiring, like the iced coffee example is one.

I heard there's also a thing where taller people are more likely to get promoted into managerial positions, and I say this as someone who used to have a boss who was 6'7" and was, like, an okay boss, but not amazing. I feel like there, there might have been some bias there. But yeah, if you're taking interviews on a video call, then you can't tell how tall people are, so that gets rid of that.

Daniel Chait: And that's why, at the start of the conversation when I said, "Look, I think there's a lot of risks and a lot of concerns about AI that we all need to be cognizant of," and we spend a lot of time on that, and we bring the receipts. We publish our audits monthly, as Ophir mentioned. But I think also there's an opportunity to do much, much better, and I think these kinds of things that we're talking about, these are mistakes that humans have been making for hundreds of years.

And so the stat- it's not like the status quo is great. It's not like getting a job has always been perfectly fair. There was a study in 2003 that showed a tremendous amount of bias just in reading resumes and deciding who to interview at all, right? That showed that if you sent these, these exper- these experimenters sent out a few thousand resumes that were identical other than the name at the top.

A- and half of the resumes they sent out, the name at the top was a stereotypically white-sounding name, and the other half was a stereotypically Black-sounding name. And they measured who got invited for an interview, and what they found was a 50 percentage point difference. 

Richie Cotton: That strikes that's a real bias and it's having a huge effect on recruiting.

And it's negative for the person who's trying to get hired and also for the company, 'cause it's not hiring the right people. 

Daniel Chait: You got it. And these-- so my point is that these have been massive issues in hiring for literally generations. And as Ophir mentioned, now we have the opportunity to teach a computer program, don't look at the name on the top of the resume.

It's not relevant to the job. Here's the rubric. Here's the things we care about. I think we can skip resume review altogether, define that 50% of bias completely out of existence, make the process work better for everyone, and help people get jobs. So I'm very optimistic that there's a lot of benefit to be gained, even as we need to be very cognizant of the risks and downsides.

Ophir Samson: And we're also not making the claim that AI is not bias. That that's a very broad claim, and we- we're not making-- Yeah we may-- We're not hiding the fact that we use models from various frontier labs. We are not making the claim that those LLMs have zero bias into it, because then you're in the kinda realm of how are they trained, does it...

that, that's not a case we're making. The case we're making as the, is that the software that you receive as a user of this is a combination of two things. One, that, the various frontier models that we're using and different LLMs that we're using. And two, is a thoughtful implementation of them that mitigates the bias, and it's on two, the second part, that's where we can do a lot of work to make sure that 50%, gap that Daniel mentioned based on racial differences are eliminated, as well as gender-based differences, location, all the protected characteristics.

We're very kinda careful about which claims that we're making. We're not trying to speak on behalf of, like, all of AI, because clearly there are, questions of kind of bias and m- and training data in those. 

Richie Cotton: Absolutely, yeah. It's not gonna completely solve all your problems, but at least it gives you a framework to think about them.

Actually, whi- while we're on the subject there are quite a few regulations now around AI usage and particularly with the EU AI Act HR use cases are considered high risk. If you want to adopt this voice AI interviews is that gonna expose you to some regulatory problems?

What do you need to deal with here? 

Daniel Chait: Yeah certainly, I think with good reason. The- these are things that affect people's lives and we need to be very careful about it. So absolutely what we've seen is our customers are having their legal teams and even their security and privacy teams involved in these decisions early on and often.

We have a ton of resources available that we've published about this exact question, helping customers think through these issues of what do they need to pay attention to, and how can they safely adopt AI technologies. The fact is that this framework talk-- whether it's the EU Act or other AI laws while they do impose regulatory requirements, they also give everyone more confidence that things are being done well.

And so we've been very encouraging of, legal oversight and regulatory review of these technologies and our customers that have gone through that. They're very happy to be able to say that they've bought these technologies, implemented them in accordance with, known laws and published standards.

And our own legal team has actually done a huge amount of work to make it easy to understand and clear what the requirements actually are. And of course, our product teams, led by Ophir, are very good at making sure that what we do is compliant with the law. 

Richie Cotton: Okay, yeah. 

Daniel Chait: I've As I should point out, as opposed to human interviews that are out there today saying who knows what to who knows who outside of control.

And again, so it's, it does come back sometimes to this question of as compared to what? And I think, we've-- I've built a career on the notion that most hiring has a lot of problems on it that we need to fix, and this is one of them. So I do think that companies who are thinking about these questions of fairness and of the implications of these technologies on job seekers I do think that they're starting to see the real benefit of bringing technology to the problem that used to be done by humans who have all kinds of issues.

Richie Cotton: Absolutely. Certainly when human interviews aren't transcribed, then, all sorts of things get said far too often. Or actually I have one more possible objection for this. There are a lot, there are a lot of possibilities for faking things now. Obviously you can have this a tool called Interview Hamoach, like an AI support thing for people being interviewed.

So it finds out what the interviewer is asking you, and then it gives you, like some text to read from. Or you can have a friend in the room, you can clone your voice and have someone else say things or something. There are all sorts of ways of of faking this. H-how do you deal with this?

Ophir Samson: Yeah I can take this one. We have built a lot of anti-cheat technology into the platform. Anti-cheat and also anti-fraud technology. I'll say k- well, one, one thing is yeah, we're mentioning the bot applications earlier. We're seeing some ro- with some roles that fifty to seventy percent of inbound applications are just not real people.

So just by virtue of putting an AI interview in the pipeline, you're already going to in one fell swoop kill off a lot of those bot applications. Then within the interview itself, we have anti-fraud, anti-cheat detection. We can it will give the recruiter a heads up and a flag with a confidence score and a justification of that confidence score of, whether we think a candidate is reading from a script, whether that script is, a ChatGPT or a, a Claude or one of the cheating tools that, that you mentioned.

And we, And so we detect that. Importantly for us is we don't let that affect the score of the candidate. This is in the realm of the types of things that re- still require human judgment. Because there's no cheat detection, no fraud detection, the lie detection system, even the FBI's, by their polygraphs, nothing is completely foolproof.

Our approach to this is, "Hey, we flagged. This is really suspicious. Here are the five reasons that we think it's really suspicious. Here are timestamps in the interview to look at for yourself, and you make the de- determination. We're still gonna give you the score to how well they did based on the content of what you're saying or of what the candidate said.

You make the final decision." 

Richie Cotton: Oh, yeah, so a- if they actually say the word M dash out loud, then they're probably reading from the prompt. Okay, no I like the li- again, it's it's giving information to the recruiters so the human makes the decision about do they think this is fraudulent behavior or not.

And I suppose if you're gonna cheat, then you're gonna get caught out pretty soon, either in a later interview stage or fairly s- soon after you get hired if you're just making stuff up. 

Ophir Samson: It's a massive problem. It's a huge problem. And it's becoming ex- extremely important for all of our customers to recognize that upfront before they get to the point of actually hiring the candidate and- so yeah, it's something that we spend a huge amount of time on.

Richie Cotton: Okay, nice. All right just to wrap up, I always want more people to learn from, so whose work are you most excited about now? 

Daniel Chait: Oh my goodness. Ours, of course. Look, I think there's a ton of really interesting stuff happening. I would point to the work that the recruiting team at Zapier is doing as really interesting, as well as I think Paradigm Strategies is doing some really interesting work around measuring culture and bias within the organization.

And then I'd be remiss if I didn't talk about the team at Anthropic. 

Richie Cotton: Sounds like they're both doing great work there. Excellent. Ofir who are you looking at? 

Ophir Samson: Yeah I just kinda echo what Dan said about Zapier. So they've published a few case st- For anyone who wants to kinda really take a deeper dive of into would this work for their organization, what are best practices for implementing this, how how do other kind of leading recruiters use it, then Z- Zapier have published some case studies showing showing that.

So that's some kind of good reading. Anyone who is interested in the more kind of technical side of voice then, and wants to go down an engineering direction, I recommend the reading of a company called Pipe Cat and Daily. But yeah, if anyone wants to see the the product, we'd be happy to show it to them as well.

Richie Cotton: Nice. All right, so we got two votes for what Zapier are doing. Sounds like, yeah, it's worth looking into their recruiting process. All right thank you so much for your time, Daniel and Ofir. It was great chatting with you. 

Daniel Chait: Thanks a bunch. It has been great. 

Ophir Samson: Thank you.