Google puts users at risk by downplaying health disclaimers under AI Overviews

Google did not deny its disclaimers fail to appear when users are first served medical advice, or that they appear below AI Overviews and in a smaller, lighter font.

Google is putting people at risk of harm by downplaying safety warnings that its AI-generated medical advice may be wrong.

When answering queries about sensitive topics such as health, the company says its AI Overviews, which appear above search results, prompt users to seek professional help, rather than relying solely on its summaries. “AI Overviews will inform people when it’s important to seek out expert advice or to verify the information presented,” Google has said.

However, the company does not include any such disclaimers when users are first presented with medical advice.

Google only issues a warning if users choose to request additional health information and click on a button called 'Show more'. Even then, safety labels only appear below all of the extra medical advice assembled using generative AI, and in a smaller, lighter font.

“This is for informational purposes only,” the disclaimer tells users, who click through for further details after seeing the initial summary, and navigate their way to the very end of the AI Overview. “For medical advice or a diagnosis, consult a professional. AI responses may include mistakes.” 

Google did not deny its disclaimers fail to appear when users are first served medical advice, or that they appear below AI Overviews and in a smaller, lighter font. AI Overviews “encourage people to seek professional medical advice”, and frequently mention seeking medical attention within the summary itself “when appropriate”, a spokesperson said.

AI experts and patient advocates said they were concerned. Disclaimers serve a vital purpose, they said, and should appear prominently when users are first provided with medical advice.

“The absence of disclaimers when users are initially served medical information creates several critical dangers,” said Pat Pataranutaporn, an assistant professor, technologist, and researcher at the Massachusetts Institute of Technology (MIT) and a world-renowned expert in AI and human-computer interaction.

First, even the most advanced AI models today still hallucinate misinformation or exhibit sycophantic behaviour, prioritising user satisfaction over accuracy. In healthcare contexts, this can be genuinely dangerous.

“Second, the issue isn’t just about AI limitations — it’s about the human side of the equation. Users may not provide all necessary context or may ask the wrong questions by misobserving their symptoms.

“Disclaimers serve as a crucial intervention point. They disrupt this automatic trust and prompt users to engage more critically with the information they receive.” 

Gina Neff, a professor of responsible AI at Queen Mary University of London, said the “problem with bad AI Overviews is by design” and Google was to blame. “AI Overviews are designed for speed, not accuracy, and that leads to mistakes in health information, which can be dangerous.”

Neff said the investigation’s findings showed why prominent disclaimers were essential. “Google makes people click through before they find any disclaimer,” she said. “People reading quickly may think the information they get from AI Overviews is better than what it is, but we know it can make serious mistakes.” 

Google has since removed AI Overviews for some but not all medical searches.

Sonali Sharma, a researcher at Stanford University’s centre for AI in medicine and imaging, said: “The major issue is that these Google AI Overviews appear at the very top of the search page and often provide what feels like a complete answer to a user’s question at a time where they are trying to access information and get an answer as quickly as possible.

"For many people, because that single summary is there immediately, it basically creates a sense of reassurance that discourages further searching, or scrolling through the full summary and clicking ‘Show more’ where a disclaimer might appear.

What I think can lead to real-world harm is the fact that the AI Overviews can often contain partially correct and partially incorrect information, and it becomes very difficult to tell what is accurate or not, unless you are familiar with the subject matter already.

A Google spokesperson said: “It’s inaccurate to suggest that AI Overviews don’t encourage people to seek professional medical advice. In addition to a clear disclaimer, AI Overviews frequently mention seeking medical attention directly within the overview itself, when appropriate.” 

Tom Bishop, head of patient information at Anthony Nolan, a blood cancer charity, called for urgent action. “We know misinformation is a real problem, but when it comes to health misinformation, it’s potentially really dangerous,” said Bishop.

“That disclaimer needs to be much more prominent, just to make people step back and think … ‘Is this something I need to check with my medical team rather than acting upon it? Can I take this at face value or do I really need to look into it in more detail and see how this information relates to my own specific medical situation?’ Because that’s the key here.” 

He added: “I’d like this disclaimer to be right at the top. I’d like it to be the first thing you see. And ideally it would be the same size font as everything else you’re seeing there, not something that’s small and easy to miss.”

Guardian