Our Chief Scientific Officer, Dr Boris Altemeyer, comments on why Amazon’s AI recruitment tool developed a sexist bias.
It is very interesting to see that a tech giant with, arguably, access to the absolute best talent in this area has admitted defeat. Their AI was simply automating bias, rather than removing it.
Based on our experience, it is not entirely surprising that this occurred. Relying on CV information for job fit is a limited, risky and biased approach. It is affected by factors such as language proficiency, the locus of control (how much you attribute things happening to you as being a result of your own actions), as well as education and social economic status.
Many high profile individuals rely on professional writers to fine-tune or completely ghostwrite their CVs. Therefore using mere a CV as an indicator of potential job performance relies on one – questionable – assumption: what you did in the past and are able to document coherently is indicative of what you are likely to do in a complex and changing environment.
It has been shown in research – meta-studies in particular – that CVs are not a reliable indicator of performance. Therefore, Cognisess starts at the point of which decisions are made: the brain.
The Cognisess Deep Learn™ framework that underlies many of our features doesn’t take the same approach as an AI CV selection tool. We are interested in learning as much as possible about a person to make a considered decision, whilst assessing their individual potential. However, personal information such as CV data is only a fraction (and actually the least part) that we are interested in.
Objective metrics such as the ability to inhibit automatic responses, paradigm-shifting, or problem solving, are not impacted by your ability to express how good you are at them. Which is why Cognisess use accessible game mechanics to measure them.
Whilst one can argue that there are gender differences between certain aspects of brain preference and function, this is far less of a concern in our complex profilers. If one were to select an employee purely based on one attribute, it wouldn’t paint a full picture of a candidate. However, when we build profilers with up to 140 separate aspects, each of which can have individual weights, target values, and thresholds, these effects arguably become insignificant.
The wealth of data that can be generated on cognition, emotion, behaviour and emotion detection in videos via AI, allows employers to take the crucial step to calibrate what ‘best’ truly looks like. In many cases, the Amazon AI recruiting tool will lead to additional scrutiny on what has caused the bias in workforce diversity including the poorly established or biased KPIs.