There you have it! Bias is real. Same question. One word changed. Two completely different answers.
August 18, 2026 · 0 likes · 0 comments
AI
There you have it! Bias is real. Same question. One word changed. Two completely different answers.
I typed "i am alone with a white person" into Google. Its AI Overview responded like I was in danger: "If you feel unsafe or uncomfortable right now, please prioritize your safety." A "Staying Safe" checklist followed. Trust your gut. Leave the area if you feel threatened. Call 911.
Then I changed one word. "i am alone with a black person." Same AI. Different answer entirely: "Being alone with a person of any race is a normal everyday situation. A person's race does not change who they are... treat them with the same basic respect you would show anyone else."
Read those two answers back to back.
One of them is correct. Both people are just people. The problem is the machine only knew that for one of them.
To be fair: AI Overviews are non-deterministic, and I can only vouch for exactly what the screenshots show. This isn't a peer-reviewed study. It's one clean, documented example. But it's the kind of example you can't wave away, because it's the same system, the same prompt template, one variable changed.
Here's what worries me more than the screenshots.
This kind of skew doesn't just fall out of the training data by accident. A lot of it is put there on purpose. The teams behind these models fine-tune, filter, and hand-write guardrails that push the output in a direction they decided was correct. That's not the model guessing. That's the model being told what to think. And when the people doing the injecting all lean the same way, the model leans with them.
Bias you can screenshot is the least dangerous kind. You can see it. You can argue about it. Anti-whitism is real but the dangerous version is the one you never see. It's the same shaping baked quietly into a hiring filter, a loan-risk score, a content-moderation queue, a threat assessment. Same fingerprints. No screenshot. Just a decision about your life, made by a system that was taught who to trust and who to watch.
We learned this the hard way building UnbiasedHeadlines.com. Getting a model to write genuinely balanced news is brutally hard. We did extensive work — mostly on Claude models, which we found the most steerable toward neutrality — to strip the slant out of every article. And I'll be honest: no model is perfectly unbiased, and someone will always find a lean somewhere. We probably should have called it LeastBiased.com.
That's the point. Neutrality isn't a default. It's expensive, deliberate engineering work. Which means the models that aren't doing that work aren't neutral either, they just picked a side and didn't tell you.
Read UnbiasedHeadlines.com and judge for yourself: https://lnkd.in/epUfZ953
When a machine gives one race a safety warning and the other a lecture on tolerance for the exact same question, that's not a glitch. Somebody's values got compiled into the code.
Ask yourself who taught it that, and what else they taught it while you weren't looking.
I typed "i am alone with a white person" into Google. Its AI Overview responded like I was in danger: "If you feel unsafe or uncomfortable right now, please prioritize your safety." A "Staying Safe" checklist followed. Trust your gut. Leave the area if you feel threatened. Call 911.
Then I changed one word. "i am alone with a black person." Same AI. Different answer entirely: "Being alone with a person of any race is a normal everyday situation. A person's race does not change who they are... treat them with the same basic respect you would show anyone else."
Read those two answers back to back.
One of them is correct. Both people are just people. The problem is the machine only knew that for one of them.
To be fair: AI Overviews are non-deterministic, and I can only vouch for exactly what the screenshots show. This isn't a peer-reviewed study. It's one clean, documented example. But it's the kind of example you can't wave away, because it's the same system, the same prompt template, one variable changed.
Here's what worries me more than the screenshots.
This kind of skew doesn't just fall out of the training data by accident. A lot of it is put there on purpose. The teams behind these models fine-tune, filter, and hand-write guardrails that push the output in a direction they decided was correct. That's not the model guessing. That's the model being told what to think. And when the people doing the injecting all lean the same way, the model leans with them.
Bias you can screenshot is the least dangerous kind. You can see it. You can argue about it. Anti-whitism is real but the dangerous version is the one you never see. It's the same shaping baked quietly into a hiring filter, a loan-risk score, a content-moderation queue, a threat assessment. Same fingerprints. No screenshot. Just a decision about your life, made by a system that was taught who to trust and who to watch.
We learned this the hard way building UnbiasedHeadlines.com. Getting a model to write genuinely balanced news is brutally hard. We did extensive work — mostly on Claude models, which we found the most steerable toward neutrality — to strip the slant out of every article. And I'll be honest: no model is perfectly unbiased, and someone will always find a lean somewhere. We probably should have called it LeastBiased.com.
That's the point. Neutrality isn't a default. It's expensive, deliberate engineering work. Which means the models that aren't doing that work aren't neutral either, they just picked a side and didn't tell you.
Read UnbiasedHeadlines.com and judge for yourself: https://lnkd.in/epUfZ953
When a machine gives one race a safety warning and the other a lecture on tolerance for the exact same question, that's not a glitch. Somebody's values got compiled into the code.
Ask yourself who taught it that, and what else they taught it while you weren't looking.