Medical AI Benchmarks vs Custom Evals

The main public benchmarks

Sizes are as reported at release; later versions and leaderboards may differ. HealthBench and MedHELM scores depend on model graders or juries and are not directly comparable to accuracy.

Contamination: why static public test sets age badly

Preprint evidence (not yet peer-reviewed): a 2025 preprint argues that benchmarks derived from public exams raise “contamination bias that can inflate performance” (arXiv:2510.09275 (2025 preprint)), and a 2026 preprint describes test sets that “inadvertently leak into training corpora, leading to inflated performance estimates” (LiveMedBench, arXiv:2602.10367 (2026 preprint)). Peer-reviewed work on leakage in machine learning generally found it in 17 fields and 294 papers (Kapoor S, Narayanan A, Patterns 2023); that study is not specific to medical LLM benchmarks.

Why custom evals matter

A systematic review of 519 studies of LLMs in health care found that only 5% used real patient care data, 95.4% used accuracy as the primary dimension, and calibration and deployment considerations were rarely measured (Bedi S et al., JAMA 2025). MedHELM’s authors note that near-perfect exam scores “inadequately reflect” real clinical practice. And locally, a widely deployed sepsis model scored an AUC of 0.63 on one health system’s data (Wong A et al., JAMA Intern Med 2021): performance has to be checked on your own population.

What it means for startups

• Use public benchmark scores as a first filter, not evidence of fit for your use case.
• Build a held-out eval from your own task, users and de-identified data, with clinician-defined rubrics; measure safety and calibration, not only accuracy.
• Keep the custom test set private and versioned, and re-run it when the underlying model changes.

Go deeper

• How to evaluate a health LLM
• Health AI validation claims: 8 red flags
• Clinical prediction model validation

Sources (accessed October 27, 2026)

1. Jin D et al., 2020
2. Pal A et al., 2022
3. Jin Q et al., 2019
4. Singhal K et al., Nature 2023
5. Arora RK et al. (OpenAI), 2025
6. Bedi S et al., 2025
7. arXiv:2510.09275 (2025 preprint) (preprint)
8. LiveMedBench, arXiv:2602.10367 (2026 preprint) (preprint)
9. Kapoor S, Narayanan A, Patterns 2023
10. Bedi S et al., JAMA 2025
11. Wong A et al., JAMA Intern Med 2021

Related resources

• Healthcare AI consulting
• Expert clinical labeling
• Medical data labeling