Keep the Humans in Mind
A new Stanford study takes the first real look at the AI tools companies use to screen job applicants — and the scale is huge: 3.4 million people, 4 million applications, 150 employers. The detail that ties it all together is that every application was screened by the same AI vendor.
Two things stood out. First, bias. By the federal standard for discrimination, 26% of Black applicants and 15% of Asian applicants were applying to jobs where the AI favored their group less than others. And here’s the strange part — if a company averages all its jobs together, that bias disappears. You only catch it when you look one job at a time.
The second finding is the one that sticks with you. Because a single vendor screens for so many companies, its decisions all lean the same way. So one rejection tends to become all rejections. Among people who applied to four jobs, one in ten got turned down by every single one. Older studies never showed this — it only happens when one AI takes over the market, and suddenly a single algorithm can shut you out of an entire industry. (The study is from Stanford HAI.)
This is a very frustrating problem, and I really feel for the people who are just coming out of school and have to face this head-on.
It seems really unfair to me.
I think it would be almost patronizing of me to say, “Just make your application as good as possible, build projects in your free time, and do well in school,” because it has gotten more complicated than that.
But those are still the only things that I know you and I have direct control over.
I wish I had better advice regarding this.
A basic filter, then humans
What I would urge companies that are hiring to do is involve humans more in the process. At the very least, if they are going to use third-party AI tools to filter applications, I think they should introduce some variation in the tools they use, rotate between them, or keep humans in the loop.
It seems to me like AI has been applied to a problem here where it may not have been needed.
And that might be part of a larger discussion.
At what point, and for what kinds of problems, is AI overkill?
Because in this case, from my perspective, a basic filter should probably be all that is required. I have not been a recruiter, so take my opinion with that understanding.
But there should be a hard set of requirements. A degree, in certain cases. Some level of experience. A certain number of years. A set of lines you must cross, a threshold you must meet, in order to fit the general profile they are looking for.
After that first filter, though, I think the decisions become largely subjective and require humans to make them.
That is the purpose of the interview.
A person working at a big, impressive company who is now looking for another job can be interpreted in two completely different ways. They could be a great candidate coming from a great company who performed extremely well. Or they could be someone who happened to work at a big company, underperformed there, slacked off, and may underperform at your company as well.
Likewise, someone at a much smaller company who performed exceptionally well could be a far better candidate than the person with the impressive company name on their résumé.
A superficial look at a résumé would never give you enough information to know which person you are dealing with.
That comes from the interview. That comes from talking to the person, understanding what they actually did, how they think, what they contributed, and how well they performed.
So this seems to be a case where a problem that may have needed a relatively simple solution — a basic filter — was given an extremely powerful solution, and we need to understand whether that has actually improved the process.
Cheaper is not the same as better
I imagine the purpose of introducing AI into hiring is obvious. Time savings. Labor savings. Resource savings. Ultimately, money savings.
But then the important questions become:
Is the time saved actually yielding more productivity or revenue for the company doing the hiring?
Are the employees being hired through these systems actually better than the employees who would have been hired without them?
And when we measure the reduced cost of recruiter hours, are we also taking into account the possibility that keeping more humans involved might lead to better candidates being hired?
I could not accurately guess the answer. I have no way of knowing whether the savings produced by these systems ultimately improve the bottom line once you account for all of those factors.
It is a complicated equation. And I think that would be an extremely interesting thing to study.
Research like the work being done at Stanford is important because we need to understand the effects AI is having across every area of society, not simply assume that because a process is cheaper or faster, it is automatically better.
Anecdotally, there is clearly a lot of pain emerging among people who are looking for jobs. If you look at forums where people are applying for work — something like the CS Majors subreddit in my case, or other computer science communities — you see people talking about applying to hundreds of positions and receiving almost no responses.
They are not getting jobs. They feel like they have no opportunity. Some of them spent years earning a degree and seemingly cannot even get the chance to use it.
I think that is a horrible thing.
Of course, there will always be some percentage of people who struggle to find work. That is unavoidable.
But the question is whether what we are seeing now is normal. Is this roughly the same percentage of people who would have struggled before? Or are the systems we are introducing contributing to unemployment or making it harder for qualified people to even get in front of another human being?
That is the part I think we need to understand.
We need a metric
I want to be optimistic about AI. AI is meant to make our lives better.
There is clearly an anti-AI sentiment right now, and I think some of that sentiment is earned. Some of it is probably overblown. And some of it, I think, is unearned.
I believe in the benefits AI can bring.
But if we are going to introduce this technology into every part of society, we need some way of deciding whether its introduction is actually successful.
We need a metric, or perhaps a handful of metrics, that we can use to evaluate these systems.
And that evaluation cannot only include cost savings.
Human experience has to be part of the equation. Economic outcomes have to be part of the equation. The quality of the people being hired should be part of the equation. The opportunities available to people trying to enter the workforce should be part of the equation.
I would have to do much more research into economics before I could tell you exactly what those metrics should be. I am reading more about economics now and trying to educate myself, so I do not want to pretend I know more than I do.
But anecdotally, there is real pain emerging among people looking for jobs and among people already in the workforce.
And I think that pain needs to be taken seriously.
Companies want profits. Of course they do. We all want to get paid. We all want companies to be productive. I completely understand that.
But as we introduce AI and all the benefits it can bring into society, we need to figure out how to introduce it in the best possible way.
We have to keep the humans in mind.
The goal does not have to be to make life perfect. It will never be perfect.
But the goal should be to make it better.
And I wish I had better advice for the people applying right now.
I feel for you.
The only things I know to tell you are to control what you can control: make your application as strong as possible, continue learning, build things, and keep trying.
Beyond that, my hope is that more people continue researching these systems and measuring what they are actually doing.
And I would urge companies, if they are going to use AI to filter applications, to at least keep humans involved and consider rotating the third-party systems they rely on so that one opaque system does not become the single gatekeeper deciding who gets an opportunity.
People deserve a fair chance.
And if AI is going to become part of almost every part of modern life, then we have a responsibility to make sure that the systems we build with it are actually improving human life — not simply making processes cheaper.