.png){kind=small}
Algorithms don’t fail everyone equally.
Humans are flawed decision-makers. Decades of research show that we’re bad with numbers and easily influenced by cognitive and social biases that operate beneath our awareness. Racism still creeps into human decisions about hiring, housing, and credit.
As a result, many of us are tempted to think we’re better off having machines make inferences directly from data. But that turns out to be a dangerous assumption, as we explore in this episode of Glad You Asked.
The many subjective choices that data scientists make as they select and structure training data can increase or decrease racial bias in machine learning systems. After all, engineers are humans with biases and blind spots like everyone else.
For other data sets, historical discrimination and systematic inequality will color the data no matter how diligent the collection process. Data on crime, for instance, is ultimately derived from the choices law enforcement officers make on which neighborhoods to patrol and who to arrest. All AI writing systems are trained on human writing samples, which reflect the perspectives of the dominant writing groups.