AI May Not Be Ready to Run Emergency Rooms
By Dennis Thompson HealthDay Reporter
TUESDAY, Oct. 8, 2024 -- AI isn’t ready to run a hospital’s emergency room just yet, a new study concludes.
ChatGPT likely would ask for unnecessary x-rays and antibiotics for some patients, and admit others who don’t really need hospital treatment, researchers reported Oct. 8 in the journal Nature Communications.
“This is a valuable message to clinicians not to blindly trust these models,” said lead researcher Chris Williams, a postdoctoral scholar with the University of California, San Francisco.
“ChatGPT can answer medical exam questions and help draft clinical notes, but it’s not currently designed for situations that call for multiple considerations, like the situations in an emergency department,” Williams added in a UCSF news release.
For the new study, researchers challenged the ChatGPT AI model to provide the sort of recommendations an ER doctor would make after initially examining a patient.
The team ran data from 1,000 prior ER visits past the AI, drawn from an archive of more than 251,000 visits.
The AI had to answer “yes” or “no” as to whether each patient should be admitted, sent for X-rays or prescribed antibiotics.
Overall, ChatGPT tended to recommend more services than were actually needed, results showed.
The ChatGPT-4 model was 8% less accurate than human doctors, and ChatGPT-3.5 was 24% less accurate.
This tendency to overprescribe might be explained by the fact that the AI models are trained on the internet, Williams said. Legitimate medical advice sites aren’t designed to answer emergency medical questions, but to forward patients to a doctor who can.
“These models are almost fine-tuned to say, ‘seek medical advice,’ which is quite right from a general public safety perspective,” Williams said. “But erring on the side of caution isn’t always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients.”
To be more useful in the ER, AI models will need better frameworks built by designers who can thread the needle between catching serious illnesses while not asking for unnecessary exams and treatments, Williams said.
“There’s no perfect solution,” he said, “But knowing that models like ChatGPT have these tendencies, we’re charged with thinking through how we want them to perform in clinical practice.”
Sources
- University of California, San Francisco, news release, Oct. 8, 2024
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2025 HealthDay. All rights reserved.
Posted October 2024
Read this next
Recalled: More Than 67,000 Cases of Deodorant Sold at Dollar Tree, Walmart, and Amazon
TUESDAY, July 22, 2025 — More than 67,000 cases of Power Stick deodorant have been recalled due to an undisclosed manufacturing issue. The recalled deodorants, made by A.P...
Dogs Can Sniff Out Parkinson's Disease, Study Shows
TUESDAY, July 22, 2025 — Dogs’ noses are sensitive enough to track down fleeing convicts, locate human remains in hidden burial sites and detect illicit drug...
For Some, Gluten Intolerance Is Psychological, Study Says
TUESDAY, July 22, 2025 — Gluten intolerance might be all in the minds of some people with irritable bowel syndrome (IBS), results from a small-scale experiment...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.