Microsoft AI publishes MAI-DxO sequential diagnosis study in Science and open-sources SDBench

Microsoft AI has published peer-reviewed studies of the Microsoft AI Diagnostic Orchestrator (MAI-DxO) in Science (doi:10.1126/science.aeh7930) and BMJ Digital Health and AI, alongside the open-source Sequential Diagnosis Benchmark (microsoft.github.io/sdbench).

Multi-agent debate and cost control

Unlike static medical benchmarks such as USMLE where all patient findings are handed to a model at once, SDBench forces diagnostic systems to start from an initial complaint, ask follow-up history questions and order laboratory or imaging tests sequentially while tracking cumulative US healthcare test costs.

MAI-DxO wraps frontier language models in a structured virtual panel of five specialist roles: hypothesis generation, test selection, devil’s-advocate challenge, cost stewardship and quality checklist review. Across rounds of chain-of-debate reasoning, the panel updates a differential diagnosis and weighs expected information gain against the dollar cost of each test before committing to a final diagnosis.

Clinical evaluation next steps

On 304 New England Journal of Medicine (NEJM) clinicopathological cases, MAI-DxO achieved 83% diagnostic accuracy compared with 22% for 21 practising US and UK physicians working without reference materials (3.7 times as accurate) and 36% for physicians with textbooks and internet search (2.3 times as accurate); a cost-constrained configuration reached 81% at less than half the test spend. Microsoft AI is now launching a prospective clinical study with Beth Israel Deaconess Medical Center in Boston.