Large language models (LLMs) are improving in their ability to generate final diagnoses, but significant shortcomings in clinical reasoning persist, hindering their readiness for clinical deployment. Specifically, these models struggle with differential diagnoses, which involve identifying and ruling out alternative conditions and causes of symptoms. This limitation is critical, as accurate diagnoses rely on a thorough consideration of multiple potential explanations for a patient's symptoms. Researchers have evaluated the performance of off-the-shelf LLMs in clinical settings, concluding that while they show promise, they are not yet suitable for prime-time clinical use1. The inability of LLMs to replicate the nuanced reasoning of human clinicians undermines their potential to support medical decision-making. So what matters to practitioners is that LLMs require substantial refinement before they can be trusted to inform clinical judgments.