Definition
Data bias is when the data an artificial intelligence (AI) learns from leans toward certain groups or conditions, so the AI ends up making unfair or distorted predictions. The computer hasn't developed prejudice on its own. It happens when the training data carries real-world imbalances as they are, when it fails to capture some groups at all, or when human judgment creeps in during collection and labeling.
A kid who has only seen white cats won't know a black cat
Imagine you show a baby nothing but photos of white cats their whole life, saying "This is a cat." The first time that child meets a black cat or a calico on the street, they won't think it's a cat. It's not because their eyesight is bad. It's because everything they ever learned was white.
AI learns about the world the same way. It doesn't gain experience and build common sense on its own. It's a program that studies the huge pile of data people hand it and finds the rules they share. If the amount or variety of that data leans one way, the AI will mistake a distorted rule for the plain truth.
In the end, an AI's judgment depends heavily on the quality of its data. Computer science has a famous saying: "Garbage in, garbage out" (GIGO). An AI fed biased data can't help but make biased decisions.
When real-world prejudice shows up in the data mirror
A well-known case involved an AI hiring tool built by a major global company. The team trained it on resumes the company had received over the previous 10 years. The program then began to rate resumes containing words related to "women" lower. Most past applicants for technical jobs had been men, so the AI wrongly learned that features common on men's resumes were good signs. In the end, the company chose not to use the tool.
Face recognition runs into similar problems. This technology is supposed to recognize faces from all over the world. But if it's trained mostly on photos of men with lighter skin tones, it makes more mistakes telling apart people with darker skin tones, and women's faces too.
Data is like a mirror held up to the world we live in. If inequality or outdated habits in our society are still sitting in that data, the AI will learn real-world imbalance as the right answer, and it can make that bias even bigger and harder to undo.
A bit more precisely: where does it come from?
To be more precise, data bias doesn't come only from a shortage of data. It seeps in, in many forms, throughout the whole process of collecting data, sorting it, and testing the model.
One classic type is "sampling bias." For example, a survey run only through a smartphone app leaves out older people who aren't comfortable with digital devices, so their opinions go missing entirely. Another common type is "label bias" (also called annotator bias). Labeling means attaching the right answer to data, like putting name tags on photos or text, and the workers' own opinions or stereotypes can slip into that step.
To fix this, AI developers use techniques that artificially fill in data for underrepresented groups. They are also actively adopting methods that build fairness requirements into the algorithm itself. Reducing data bias is more than a way to make AI technically more accurate. It's a core challenge if AI is going to earn society's trust.
Common misconceptions
AI is just a machine, so it doesn't have biases or preconceptions like humans do.
AI learns directly from data that humans collected and processed. So if that data holds human prejudice or social imbalance, the AI will copy that bias just the same.
Where you meet it
An AI hiring system trained on past resumes, mostly from male applicants, that rated resumes containing women-related words lower
A smartphone camera trained mostly on lighter skin tones that has trouble recognizing the faces of people with darker skin tones
In one sentence
AI learns by looking into the mirror of data, so when the data is lopsided, the AI's decisions become unfair too.