Capabilities of GPT-5 on Multimodal Medical Reasoning
Free while signed in. Answers cite the passages they came from.
A controlled, zero‑shot CoT evaluation positions GPT‑5 as a generalist medical reasoner across text and image inputs. Using standardized prompts and splits, GPT‑5 consistently beats GPT‑4o and smaller GPT‑5 variants, with especially large gains on multimodal expert‑level QA.
Unified zero‑shot CoT setup. The authors fix prompts, exemplars, and answer formats for both QA and VQA to isolate the model upgrade.
Text benchmarks: new highs. On MedQA (US 4‑option) GPT‑5 reaches 95.84% (+4.80% vs GPT‑4o). On MedXpertQA‑Text, GPT‑5 improves reasoning by +26.33% and understanding by +25.30% over GPT‑4o. MMLU‑Medical is at or near ceiling, with notable gains in Medical Genetics and Clinical Knowledge.
USMLE practice sets: strong clinical management. GPT‑5 tops all baselines on Steps 1–3, with the largest margin on Step 2 (+4.17%), averaging 95.22% (+2.88% vs GPT‑4o).
Multimodal reasoning: big leap and human‑plus. On MedXpertQA‑MM, GPT‑5 gains +29.26% in reasoning and +26.18% in understanding over GPT‑4o and surpasses pre‑licensed human experts by +24.23% and +29.40%, respectively. A worked case shows coherent synthesis of CT findings and clinical context to recommend a Gastrografin swallow.
Caveats and outlook. GPT‑5 is slightly below GPT‑5‑mini on VQA‑RAD, possibly due to calibration on small radiology datasets. The discussion cautions that standardized tests differ from messy clinical reality and calls for prospective studies and deployment calibration.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack