A randomized study published on February 9, 2026 in Nature Medicine throws cold water on the idea that medical chatbots are ready for the public. The research shows a stark split between how LLMs perform on their own and how they perform when everyday users actually consult them — a gap that matters as more people turn to AI for health questions.
The study, “Reliability of LLMs as medical assistants for the general public,” tested 1,298 adults in the UK across ten physician‑designed scenarios. Participants were assigned either an LLM — GPT‑4o, Llama 3, or Command R+ — or told to use whatever sources they would normally rely on. The goal was simple: identify the likely condition and decide the right level of care, from staying home to calling an ambulance.
When researchers prompted the models directly, performance looked impressive: the LLMs identified relevant conditions in 94.9% of cases and picked the correct disposition 56.3% of the time. But once people interacted with the same systems, accuracy fell off a cliff. Participants using the LLMs identified relevant conditions in fewer than 34.5% of cases and chose correct dispositions in fewer than 44.2% — results no better than the control group. The takeaway is not that models “know nothing,” but that the human‑AI interaction itself is the weak link.
That interaction problem matters because public‑facing tools don’t live in laboratory conditions. The paper argues that standard benchmarks and simulated patient tests fail to predict what happens when real users ask real questions, and it explicitly calls for systematic user testing before deployment. In short: strong exam‑style scores don’t translate into safe, reliable medical guidance for the public.
404 Media’s reporting frames the findings bluntly: the hype around AI “doctor” chatbots doesn’t hold up when people try to use them. The article notes that the study’s participants received incorrect or conflicting advice and that the researchers found the tools weren’t ready to replace professional judgment. That lines up with the study’s core argument — the models can appear competent in isolation, but real‑world use exposes a fragile chain of communication.
There’s another uncomfortable detail in the Nature Medicine paper: it points out that many people already use AI chatbots for sensitive health questions. That means the risk isn’t hypothetical. When tools feel confident but are inconsistently helpful, the outcome can be either false reassurance or unnecessary alarm — and both can push users toward the wrong care decisions.
For readers, the lesson isn’t “never touch AI.” It’s closer to “don’t outsource judgment.” The evidence suggests that if you use a chatbot, treat it like a first‑pass brainstorming tool — then verify with trusted medical sources or a professional. That’s especially true when the right decision is about urgency, not just diagnosis.
There’s also a another angle that’s easy to miss. When you paste your symptoms, lab results, or medication details into consumer chatbots, they’re moving highly sensitive data into systems with unclear retention and sharing policies. Even without a breach, that data can be exposed through account compromises, phishing, or misconfigured storage. From a security standpoint, the safest move is to avoid sharing identifying health details unless you’re using a service with explicit medical‑grade protections and clear data handling terms.
Related reading: our breakdown of why sensitive personal data needs extra protection, the differences between a data breach and a data leak, and our coverage of an AI data leak in the wild.
