Will AI automated hiring work in 2019?


Artificial Intelligence is the hottest subject in business and some people say we will have AI hiring soon.  Some people have asked for my take on the matter.  So without further ado…will we see AI automated hiring in 2019?

Maybe.  But here is the rub; artificial intelligence will never be able to automate the hiring process.  That does not mean some companies wont try it.  And that certainly doesn’t mean there wont be a bunch of snake oil salesman pushing it.

There are issues with both the hiring process and limitations of modern machine learning that make it impossible.  Let’s explore why and then we can talk about some aspects it can improve upon.

Problem #1: Biased hiring data

The first major issue with utilizing past data to build your current model is that the past data is biased. I am sorry if that conflicts with your philosophical world view, but as a matter of math it is indisputable.

There have been several examples in the news recently where this has come to light. The most famous of course is Amazon scrapping their internal system.

Amazon is one of the most famous and effective data native companies on earth. They thought they could use their internal hiring data to create a system that would automatically look at resumes and show them the top candidates. The problem was the system would not stop discriminating against women.

How does something like that happen? In most cases it is because the behavior of the system is biased. The algorithms are hiring manager bias encoded in math.

Sometimes the bias is laughably bad. One firm audited their AI platform and found that the first name Jared and playing high school lacrosse were the top predictors of success.

The Jared / lacrosse example is funny because the firm realized it and pulled the plug. How many firms put in the necessary due diligence to realize these things? I have no data, but I think the answer is, “not many.”

Not funny anymore, is it?

This lacrosse player named Jared thinks it is a fantastic algorithm

Problem #2: Lack of data / feedback

The insurmountable reason why AI will never be able to automate hiring is the lack of feedback.

Building a machine learning classification model works like this:

Step 1: Take in a set of data (features) along with outcomes (labels)

Step 2: Split off a portion of that data to score the completed model…do not use this data in building the model(1).

Step 3: Build a model with the original data utilizing features to predict labels

Step 4: Score the model using the split off portion to see how well it predicted the labels

Step 5: Depending on how well it did either go back to the drawing board, make some tweaks, or accept it as a good model and put into production

That sounds easy and straightforward, why won’t it work for hiring?

A machine learning model like this will not work in the real world because it is impossible to get accurate labels

There is no way to know whether someone you did not hire would have been a good hire. You never get to see how they actually perform somewhere else.  That means you will not have any data on those candidates to judge whether the model evaluated them correctly.

There are techniques for modeling when you do not have good data for a portion of the data set. As you might imagine, this limits their accuracy and there is a limit to how much data can be missing.

In most cases, over 99% of people who submit resumes for a position are not hired. That means the model would not have feedback on 99% of its predictions. How well do you think that would work?

Now that I think about this, I could write a highly accurate model for screening candidates. For most job openings, at least 80% of the resumes submitted are totally unqualified. If the model just said no for everyone it would be at least 80% accurate!

That is a great example of the machine learning concept, “accurate and useful are not necessarily the same thing.”

What about modeling based only on people who were hired and did well?

Interesting idea…well, there are a couple of flaws. The first is, who says whether someone is a good hire? Performance rating systems are notoriously flawed. Few companies keep a good and consistent performance records.

The second flaw is that a model that only looks at what you have is probably is not all that useful. You don’t need a machine learning model to hire people who are similar to your successful employees. That is how things mostly work due to the human thought process, which lead to biased hiring data in the first place.

There are other problems with AI controlled hiring, but no need to go through them. Biased hiring data and the inability to train the model are both deal breakers. You do not need any reasons beyond them.


What about good stuff?  Artificial Intelligence can improve the recruiting system in some areas

You may have thought after reading the first section that The Analytics Dude has no faith in artificial intelligence. Not at all, I just get terrified when people treat it like a magic wand. Let’s dig into what AI recruiting can do to improve the experience.

The first thing to remember is that the while there are potential benefits, the vast majority are not achievable. Most of the purported benefits are marketing nonsense that will not live up to the hype. If someone really knew what they were doing with AI/ML they would not be working in HR.

Automate repetitive tasks

The biggest benefit is the same benefit that AI can deliver for almost any office worker. AI has the potential to automate low value and repetitive tasks.

I give HR and recruiting a lot of crap. There is a lot of stuff they do not do well. However, they ARE good at talking to candidates and hiring managers to craft job listings. A solution that allows them to spend more time talking adds value.

AI sourcing

AI bots can help source candidates. Web searching and scraping can sort through more than just the obvious on LinkedIn profiles. If done well, you can find candidates who have serious talent, not just the right positions on their resume.

Note: This is difficult and bears some of the same risks as using AI to screen resumes.

Chat bots

Chat bots can improve the candidate experience and save some time for recruiters. Chat bots can do something simple like confirming candidates are available and save recruiter time.

L’Oréal has an article on their site about using AI to help the recruiting process.

AI video analysis

Video interviews are common and important. AI can perform facial recognition and transcribe the discussion for use afterwards.

There is also something worrisome about some solutions on the market. There are vendors who claim their solutions can do things like predict personality and determine honesty.

A video AI that determines how well you would interact with the group, or whether or not you were telling the truth, is a weapon of math destruction. Its workings are opaque, you would not be able to contest the results, and it would be easily scalable to any video interview.

Companies of the world, I beg you, please do not use these. This is digital enabled phrenology.


Applying AI to real world problems is harder than most people on the internet or LinkedIn would have you believe. Quality predictions are difficult with a lack of access to the whole data picture. When you add in biased data it becomes impossible to create a full AI solution.

AI can do things to make the process smoother. Automating the repetitive workload of recruiters will free up time for higher value added activities. However, there is always risk in taking decisions away from humans. Be very mindful about what your models are actually predicting. Make sure that they are auditable, e.g., you can understand what factors are affecting the outcome. And finally make sure there is some method of recourse for those affected by the model.


(1) That is not strictly true of course, there are methods like K Folds validations and Random Forests which use all the data and take turns holding out various portions of the data