Google’s AI Overview and AI Mode are unsafe for minors. That is the blunt conclusion from a new report by Common Sense Media’s Youth AI Safety Institute. Researchers spent months testing how these features handle real-world queries. The result? A system that misses crises, hallucinates history, and completes homework for kids who shouldn’t be asking for it.
Google didn’t mince words in its rebuttal. The company called the study flawed. They argued the tests used ambiguous queries that don’t reflect actual user behavior. But the data tells a different story about the risks involved in letting tweens and teens roam free on an AI-driven search engine.
The Safety Failures Are Real and Dangerous
The testing period ran from May to July. Researchers simulated 2,600 interactions using test accounts designed to look like 11- and 15-year-old users. They turned on SafeSearch. They used Google Family Link controls. They followed the rules. Then they pushed the AI to its limits.
They focused on “red line” topics. Suicide. Self-harm. Child sexual exploitation. Psychosis.
“An unacceptable amount of the time, Google doesn’t recognize a crisis or gives answers and advice that makes things worse. That is not OK for kids in crisis.”
— Geoffrey Fowler, Common Sense Media
The results were stark. The AI failed kids in distress. It missed clear signs of suicidal ideation. It reinforced psychosis. It validated disordered eating. In one chilling example, a teen used slang for suicide—a reference to a “SUE letter”—expecting help. The AI instead offered technical instructions on how to set up a digital legacy contact.
It treated a cry for help as a logistical request.
When it came to mental health referrals, the AI missed the mark more than half the time. Of 214 prompts requiring immediate medical or crisis support, AI Overview provided a safe resource only 58 percent of the time. AI Mode did slightly better at 77 percent. But neither met the 95 percent detection rate the researchers deemed necessary for a safe product.
Is AI Search Accurate Enough for School?
If you think the safety risks are the only problem, look at the homework logs.
AI Mode completed math problems and humanities essays in every single instance tested. Researchers ran 180 variations. The AI didn’t hesitate. It just did the work.
This isn’t just an academic integrity issue. It’s an accuracy issue. The study tested over 275 historical fact prompts. The AI provided different answers to identical queries 43 percent of time.
Ask about the first accurate world map. You might get one answer. Ask again. You get something else entirely.
Pop culture facts aren’t spared either. Asked who was on top of the Billboard Artist 100 at a specific time, AI Overview confidently cited Taylor Swift. The data was wrong. Drake held the spot. Google’s systems leaned on search results that had misinterpreted context, and the AI repeated the error with total confidence.
Children shouldn’t have to fact-check the search bar. Yet that is exactly what the report suggests they must do.
“Is Google really expecting a ten-year-old to [check for mistakes]? It really isn’t a product that was designed but it’s being used by kids.”
What Should Parents Do Right Now?
Three-quarters of tweens and teens are already using Google AI search answers. That number isn’t going down. Google is the default on Android devices. It’s on the Chromebooks schools hand out to students across the US. You can’t just uninstall the internet.
So where does that leave parents?
Common Sense Media has specific recommendations. They argue that schools and families should have the right to disable AI Overview and AI Mode entirely. For elementary students, they suggest avoiding Google AI search altogether. Stick to library databases. Use tools built for education, not engagement.
Geoffrey Fowler points out a simple solution: turn these features off by default for minors. It’s not revolutionary. Many other countries already restrict AI search for younger users. In the US, Google limits access to its chatbot Gemini for minors, labeling AI Overview and Mode an “unacceptable risk.”
Parents can try to block access through Android parental controls. You have to manually add google.com to the blocked list in Chrome and Web settings. It’s clunky. It’s easy to bypass. It requires constant vigilance.
Google’s Defense: It’s Not a Cradle, It’s a Tool
Google’s response was defensive. They claimed they couldn’t reproduce the report’s findings. They ran their own tests and got higher-quality responses. They argued that the prompts were contrived and ambiguous.
“Our AI Search features are an incredibly useful way to learn,” Google told Mashable. “This report tests narrow, ambiguous queries.”
They also noted that the testers didn’t ask follow-up questions. Real users chat back and forth. They clarify. The AI learns context. Single-shot prompts are unfair tests.
That might be true for adult researchers. It doesn’t change the fact that the current architecture is prone to hallucination and crisis-missing.
The core issue remains. AI search was never designed for children. It was designed to capture attention, answer vague queries, and keep users in the ecosystem. It lacks the guardrails a child needs.
Unless Google restricts these features for US minors, the burden falls on you. On the teachers. On the parents who have to sift through hallucinations and harmful advice to find the truth.
It’s a lot to ask. And frankly, it shouldn’t be our job to police an algorithm that’s already running the show.
