Executive Grapevine | Executive Grapevine International Ltd

No woman, No AI

What went wrong with Amazon’s ‘sexist’ AI recruiter?

No woman no AI

Recruiters at Amazon were forced to pull the plug on an AI recruitment tool this month after it began to show a strong bias for male candidates. Does this mean AI in the office is still a distant dream?

What was the tool meant to do?

The team had been working on computer programs with the aim of mechanizing the search for top talent since 2014, according to a report in Reuters.

The group focused on creating software that could recognise 50,000 terms that showed up on past candidates’ resumes, and therefore determine which particular phrases were linked to candidates that went on to be hired, and those that lead to a candidate ultimately being dismissed.


What went wrong?

The machine was trained by being shown the CVs of successful applicants from over the past decade – but, because the tech industry has a slant towards more male employees, the AI inadvertently ‘learned’ that CVs belonging to men were more likely to be successful.

It’s important to note that the CVs the AI was selecting as appropriate were not necessarily the ‘best’ out of the ones submitted – they were the ones which most closely matched the CVs that it had been programmed to view as the most likely to result in a hire.

As a result, words like ‘women’s’ (as in ‘captain of the women’s basketball team’) counted against the applicant, rather than potentially being viewed a positive.

“Amazon's developers most likely did not code their AI to be biased, but sadly this was inherited by the fact the coding rules were entered in by men, with an unintentional bias,” Arran Stewart, Co-owner of blockchain recruitment platform Job.com, told Recruitment Grapevine. “The women's resumes in that process were by no means less relevant or wrong in any way, it was the machine's rules that had been set to a mild bias towards male tendencies and because of this, the system was flawed.”

“The women's resumes in that process were by no means less relevant or wrong in any way.”


Does this mean recruiters will never be able to use AI to find top talent?

While this may seem disheartening, Charles Hipps, CEO of Oleeo, explained that it should be possible to iron out bugs like the Amazon team found, so long as the AI does not base its decisions on historical data (eg the CVs of former applicants) alone.

“Top employers continue to receive huge numbers of applications for each open position and ultimately, the majority of applicants – upwards of 95% – are turned down,” he said. “With AI, employers should have an even greater ability to quickly flag candidates that have certain key indicators of success to streamline the selection process and nurture this talent ahead of competitors.