Towards Expert-Level Medical Question Answering (Med-PaLM 2)

MedQA SOTA: Scored up to 86.5% on the MedQA dataset (USMLE-style questions) - a new state-of-the-art matching expert physicians.

Multi-benchmark leadership: Approaches or exceeds SOTA across MedMCQA, PubMedQA, and MMLU clinical topics datasets.

Human evaluation quality: Physician evaluators rated Med-PaLM 2 answers as comparable to those of other physicians on most axes.

Medical AI frontier: Set the bar for medical LLMs and informed FDA's thinking on AI-assisted clinical workflows.