Biased AI systems produce biased humans

Human decision-making biases are pervasive, influencing areas such as finance, law and medicine. Reliance on Artificial Intelligence (AI) systems has been offered as a solution to reduce such systematic errors. However, in judgments ranging from perception to emotion, AI systems can also exhibit biases themselves. For example, a hiring algorithm used by Amazon was inevitably trained to favor men over women and an algorithm used in US hospitals to estimate the need for additional medical care favored white patients over black patients.
While the danger of biased algorithms automating and perpetuating existing human biases is known, here we suggest the problem may be even greater. Critical decisions are often made jointly by AI and humans (e.g., AI systems provide support to human physicians in the diagnosis of life-threatening diseases). These human-AI interactions provide a mechanism by which, not only biased humans generate biased AI systems, but biased AI systems can alter human decision-making processes, leaving humans more biased than they were before.
Across multiple experiments (N = 1,022), we reveal a feedback loop where human-AI interactions alter the processes underlying human perceptual, emotion and social judgments that subsequently amplify biases in humans. This amplification is significantly greater than observed in human-human feedback loops. Strikingly, the participants underestimate the substantial impact of the biased algorithm on their judgment, which could leave them more susceptible to its influence. Finally, using computational modeling, we show that humans learn from such interactions to become more biased, potentially altering how their decisions are made.
These findings demonstrate that biased algorithms not only produce biased evaluations, but crucially increase such biases in humans, creating a feedback loop. The findings highlight the pressing need to increase public awareness of how AI systems influence human decision-making in order to reduce bias contagion and improve AI-human interaction.