AI bias isn't about AI having opinions — it's about training data that reflects an unequal world. Real examples, the math behind it, and what questions kids should be asking about every AI system they use.
Ages 7–14 · AI Foundations + Math · 30-day guarantee
Amazon scrapped an internal AI recruiting tool in 2018 after discovering it was systematically downgrading resumes that contained the word "women's" (as in "women's chess club"). The AI had learned from 10 years of prior hiring decisions — which were themselves biased toward male candidates.
A study in Science (2019) found that a widely used healthcare algorithm was biased against Black patients — assigning them lower risk scores, which meant less follow-up care, even when they were equally sick as white patients with higher scores. The algorithm used healthcare costs as a proxy for health needs. But historically, less was spent on Black patients — not because they were healthier, but because of access inequities.
MIT research found that commercial facial recognition systems had error rates of 0.8% for light-skinned men but up to 34.7% for dark-skinned women. The systems were trained mostly on images of lighter-skinned faces, so they performed worse on faces that were underrepresented in training.
Early word embedding models (the building blocks of language AI) learned that "doctor" is more associated with "man" and "nurse" with "woman" — because that was true in the text they were trained on. These associations shape downstream AI outputs in subtle ways.
When we say an AI system is biased, we mean something specific: the AI makes different errors for different groups. It's more likely to wrongly classify a dark-skinned face, less likely to recommend a woman for a job, more likely to flag a neighborhood as high-risk based on demographics rather than individual behavior.
This isn't a matter of the AI "deciding" to be unfair. AI doesn't have intentions. Bias in AI comes from math — specifically from what the training data represents, what the optimization objective rewards, and what proxies the model uses to make predictions.
Understanding AI bias requires understanding these mechanisms. And the mechanisms are, at their core, statistics: sampling bias, proxy variables, representation rates, feedback loops. These are Grade 6–8 math concepts. Kids who learn them have a framework for evaluating any AI system they'll ever encounter.
The most common source of AI bias is the training data. AI learns by finding patterns in examples. If the examples reflect historical inequities, the AI learns those inequities as if they're facts about the world.
Amazon's hiring AI trained on 10 years of resumes and hiring decisions — decisions made by humans who, on average, hired more men than women for technical roles. The AI didn't know why those decisions were made. It just learned: "resumes like these got hired; resumes like those didn't." The pattern it learned was partially about qualifications and partially about gender.
The same logic applies to medical AI trained on healthcare data from a system with access disparities, facial recognition AI trained mostly on lighter-skinned faces, and content recommendation AI trained on human engagement patterns that systematically favor certain types of content.
In each case, the bias in the output comes from bias in the input. Garbage in, garbage out — but more subtly, historical inequity in, amplified historical inequity out.
Teaching children to think critically about AI bias doesn't require them to audit model weights. It requires them to ask three questions:
"Who is this AI making decisions about?" — and "who might it work worse for?" Facial recognition for everyone might mean facial recognition for everyone equally, or it might mean it was tested mostly on one group. Knowing to ask is the skill.
"What was it trained on?" — and "who was represented in that data?" An AI trained on decades of hiring decisions at companies with historically homogeneous workforces has absorbed that history. An AI trained on medical records from hospitals that served wealthier patients has absorbed those demographic patterns.
"What is it optimizing for?" — and "does optimizing for that create problems for anyone?" An AI optimizing for "healthcare cost" as a proxy for "health need" sounds reasonable until you realize that healthcare costs are shaped by access, not just by illness.
These questions are critical thinking, not computer science. They can be asked by anyone, at any age, about any AI system.
"We talked about the Amazon hiring AI at dinner. My daughter immediately connected it to the fractions she was learning — 'so the training data was a biased sample?' She's 11. I was floored."
"AI bias felt abstract until we looked at facial recognition error rates together. My son now asks 'who was this tested on?' about every AI tool he encounters. That's a life skill."
"The math connection surprised me. I thought this was an ethics topic. But once we talked about sampling and representation, the kids got it in math terms. It made both AI and statistics click."
16 AI Foundations lessons + 182 math skills. Ages 7–14. $189/year.
🛡️ 30-day money-back guarantee — no questions asked.