As we’ve reported, artificial intelligence tools like ChatGPT are changing the hiring process, making it easier for job seekers to tailor their resumes to each position and harder for recruiters to understand who a candidate really is.

Evaluations meant to predict how an applicant would perform have long been part of the recruiting process, especially for roles in administration, retail or customer service. 

But screening assessments are now common across a variety of jobs. More than a dozen platforms offer these in the forms of aptitude games, job simulations and personality tests — all before the applicant talks to a real person. And they’re getting more high-tech.

Josh Millet, founder and CEO of an assessment platform called Criteria Corp., walked me through one of the company’s five-minute pre-employment tests. It was divided into three games with 40 questions each.

“This game is called the Robot Inspector,” he said. “It’s going to show me a whole sequence of robots, and what we’re doing is answering whether we think they’re the same or whether they’re different.”

The game showed us a pair of cute cartoon robot dogs. They both had googly light bulb eyes and tails that looked like vintage joysticks. But one had a line of orange bolts on its legs.

“And we have only six seconds for each one,” Millet warned. “So feel free to yell out answers!”

The robots began coming rapid fire, with the differences getting harder to spot.

I should note that the fictitious job we applied for is not “robot inspector.” This game can be used to help assess applicants in engineering, accounting, nursing or tech support.

“This is measuring the very sort of narrow skill of attention to detail,” Millet said.

It’s one of dozens of different assessments Criteria offers employers who are looking for more than a resume and cover letter.

“How candidates are viewed is moving from being kind of a rear view, like ‘Hey, what’s on your resume and what have you done?’ versus like, ‘Show me what you can do,'” said Lindsey Zuloaga, the chief data scientist at another assessment platform, HireVue.

It uses AI to assess one-way video interviews of applicants, answering questions like, “How would you deal with a difficult customer or adapt to a last-minute change?”

“We train those algorithms on thousands and thousands of answers that have been rated by humans with a rubric — kind of what a high, medium, low answer looks like,” Zuloaga said.

But AI trained on historic hiring data can amplify historic biases, like women being less likely to be hired as engineers or men as nurses.

In 2021, HireVue discontinued facial analysis in video interviews amid concerns about potential inaccuracy and bias in facial and emotion recognition technology. The company said advances in natural language processing had improved its analysis of spoken language, and it no longer needed visual analysis. The tool now only analyzes interview transcripts.

Zuloaga said HireVue’s algorithms are regularly audited and adjusted for bias, something now required for AI hiring systems under a New York City law.

“We’re trying to improve upon a really broken thing,” she said, “and there may be missteps and there may be imperfections along the way.”

Because without more objective measures, employers can fall back on their biases, like focusing on college degrees, noted Audrey Mickahail at the nonprofit Opportunity@Work.

“We ask executives, ‘Think about the top three to five skills that you deploy every day in your job. Where did you gain those skills?'” Mickahail said. “And pretty much universally they’ll say, ‘Oh, you know, I learned them on the job'” — not in college.

More than 60% of U.S. workers don’t have four-year degrees, including an even greater share of Black and Latino workers. With proper guardrails to ensure fairness and accuracy, Mickahail said, pre-employment assessments could open doors and improve diversity in hiring, as long as they don’t become added barriers.

“When you are used to being blocked at every turn in your career journey, things that we might not fully appreciate the impact of can be incredibly demoralizing and act as disincentives,” Mickahail said.

She recommends that employers keep tests simple, explain what the tests are for and provide feedback about how applicants can improve. The assessments should be short, or applicants should be compensated for their time, Mickahail added.