Medprompt
Free while signed in. Answers cite the passages they came from.
Microsoft researchers show that careful prompt engineering can push general-purpose GPT-4 to state-of-the-art on medical benchmarks, no domain fine-tuning required.
General-purpose prompting: Uses purely general-purpose prompt-engineering techniques (CoT, dynamic few-shot, choice-shuffling ensembling) with no medical-domain specialization.
Medprompt recipe: Combines k-nearest-neighbor example selection, GPT-4-generated chain-of-thought rationales, and choice-shuffling to cancel answer-position biases.
SoTA on 9 benchmarks: Achieves state-of-the-art on all nine benchmarks in MultiMedQA, beating Med-PaLM 2 and other specialized medical models.
Broader lesson: Reopens the question of whether domain-specific pretraining is actually necessary when a frontier base model is paired with strong prompting - a framing that has recurred in later debates.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack