GPT-4 With Vision Has Poor Accuracy for Image-Based Radiology Questions
By Elana Gotkine HealthDay Reporter
FRIDAY, Sept. 6, 2024 -- The large language model GPT-4 with vision (GPT-4V) has high accuracy for text-only radiology questions, but much lower accuracy for image-based questions, according to a study published online Sept. 3 in Radiology.
Nolan Hayden, M.D., from Henry Ford Health in Detroit, and colleagues examined the performance of GPT-4V on radiology in-training examination questions to gauge the model's baseline knowledge in radiology. The September 2023 release of GPT-4V was assessed using 386 retired questions (189 image-based and 197 text-based) from the American College of Radiology Diagnostic Radiology In-Training Examinations; 377 questions were unique.
The researchers found that GPT-4V answered 65.3 percent of the unique questions correctly, with significantly higher accuracy observed on the text-only versus the image-based questions (81.5 versus 47.8 percent). For text-based questions, differences were seen between prompts, with chain-of-thought prompting outperforming long instruction, basic prompting, and the original prompting style by 6.1, 6.8, and 8.9 percent, respectively. For image-based questions, there were no differences seen between prompts.
"We found that while GPT-4V shows relatively good performance on text-based questions, it shows deficits in accurately interpreting key radiologic images. This highlights the model's limitations in visual radiology analysis," the authors write. "We also noted an alarming tendency for GPT-4V to provide correct diagnoses based on incorrect image interpretations, which could have significant clinical implications."
Editorial (subscription or payment may be required)
Disclaimer: Statistical data in medical articles provide general trends and do not pertain to individuals. Individual factors can vary greatly. Always seek personalized medical advice for individual healthcare decisions.
© 2025 HealthDay. All rights reserved.
Posted September 2024
Read this next
Declining Childhood Vaccination May Increase Risk for Vaccine-Preventable Infections
WEDNESDAY, April 30, 2025 -- Declining childhood vaccination rates may increase outbreaks of eliminated vaccine-preventable infections within the United States, leading to a...
AACR: Incidence-Based Mortality Dropping for Young Women With Breast Cancer
TUESDAY, April 29, 2025 -- Incidence-based mortality (IBM) declined from 2010 to 2020 among women aged 20 to 49 years diagnosed with breast cancer, according to a study presented...
AACR: Nonsurgical Treatment Feasible for Mismatch Repair-Deficient Tumors
TUESDAY, April 29, 2025 -- A neoadjuvant programmed cell death 1 (PD-1) blockade enables nonoperative management among patients with early-stage mismatch repair-deficient (dMMR)...
More news resources
- FDA Medwatch Drug Alerts
- Daily MedNews
- News for Health Professionals
- New Drug Approvals
- New Drug Applications
- Drug Shortages
- Clinical Trial Results
- Generic Drug Approvals
Subscribe to our newsletter
Whatever your topic of interest, subscribe to our newsletters to get the best of Drugs.com in your inbox.