ChatGPT Capable of Clinical Reasoning -- Maybe Better Than Clinicians
By Lori Solomon HealthDay Reporter
THURSDAY, April 4, 2024 -- A chatbot outperforms physicians in clinical reasoning, according to a research letter published online April 1 in JAMA Internal Medicine.
Stephanie Cabral, M.D., from the Beth Israel Deaconess Medical Center in Boston, and colleagues compared a large language model’s reasoning abilities against human performance using standards developed for physicians. Responses were compared for selected cases queried in GPT-4 (OpenAI) in August 2023 and from 21 internal medicine attending physicians and 18 residents.
The researchers found that median Revised-IDEA (R-IDEA) scores were 10 (range, 9 to 10) for chatbot, 9 (6 to 10) for attendings, and 8 (4 to 9) for residents. The chatbot had a significantly higher estimated probability of achieving high R-IDEA scores than attendings and residents and had significantly higher R-IDEA scores than attendings and residents. There were no significant differences in attendings’ and residents’ scores. For diagnostic accuracy, the chatbot performed similarly to attendings and residents. Scores were also similar for correct clinical reasoning and cannot-miss diagnosis inclusion. However, the chatbot had more frequent instances of incorrect clinical reasoning (13.8 percent) than residents (2.8 percent) but not attendings (12.5 percent).
"There are multiple steps behind a diagnosis, so we wanted to evaluate whether large language models are as good as physicians at doing that kind of clinical reasoning," coauthor Adam Rodman, M.D., also from Beth Israel, said in a statement. "It's a surprising finding that these things are capable of showing the equivalent or better reasoning than people throughout the evolution of clinical case."
Two authors disclosed ties to industry.
Abstract/Full Text (subscription or payment may be required)
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2024 HealthDay. All rights reserved.
Posted April 2024
Read this next
Long COVID Definitions, Care Models Are Evolving
MONDAY, May 20, 2024 -- Definitions of long COVID and care models are evolving, but considerable variability is seen in these models, according to a review published online May 21...
Recommendations Developed for People With HIV Wanting to Breastfeed
MONDAY, May 20, 2024 -- In an American Academy of Pediatrics clinical report published online May 20 in Pediatrics, recommendations are presented for breastfeeding among people...
Social Determinants of Health Impact Poststroke Mortality Risk
MONDAY, May 20, 2024 -- Social determinants of health have a cumulative impact on poststroke mortality, according to a study presented at the the annual European Stroke...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.