MAI-DxO Beats Doctors on Hard Cases in Science Study

Microsoft AI’s diagnostic system, MAI-DxO, has made it into Science. In the peer-reviewed study, published on 22 October, the system reached the correct diagnosis in 83% of 304 complex cases taken from the case records of the New England Journal of Medicine. Physicians who worked the same cases with textbooks and the internet managed 36%, and 22% without them. On a second set of more ordinary emergency-department presentations, drawn from the MIMIC-IV-ED dataset, MAI-DxO scored 80% against the physicians’ 47%. Microsoft set out the results in a post titled Building Medical Superintelligence, which is where the figures below come from; the paper itself sits behind Science’s paywall.

MAI-DxO is not a model of its own. It is an orchestrator that runs a language model as what Microsoft calls a panel of virtual physicians: each member proposes diagnoses, picks the next question or test, and challenges the emerging view over several rounds of debate, weighing what each test is likely to reveal against what it costs. The cases are presented step by step, as in a real work-up, on a benchmark Microsoft calls SDBench (Sequential Diagnosis Benchmark), which scores accuracy, the evidence requested and the estimated cost of the tests ordered. Microsoft says the system reached its results at “half or even one third of the cost”, that one cost-optimised configuration scored 81% on the NEJM cases at less than half the cost of comparable systems, and that no other human or AI approach was both more accurate and cheaper on either dataset.

How much of it is the orchestrator?

Less than the headline suggests, by Microsoft’s own data. TechCrunch, which went through the study’s per-model results, reports that Microsoft ran MAI-DxO on models from OpenAI, Google, Anthropic, Meta and DeepSeek, and that the best results came from OpenAI’s. GPT-5 on its own, with a short basic prompt, already scored 78.9% on the complex cases, at about $8,400 a case; under MAI-DxO it gained about four points and cost about $3,000 less. GPT-5.5 climbed from 72% to nearly 87%, while Gemini 2.5 Flash went from 46.4% to nearly 69%. On the emergency-department cases GPT-5.5 scored about 85% with or without the orchestrator. The spread between weak and strong models is wider than anything the orchestration adds, and Microsoft’s researchers say the gains shrink as models near the ceiling.

The comparison also has its limits. The physicians were primary-care doctors and general internists, each working alone, in a simulation; the specialists and multidisciplinary teams who normally handle cases this complex were not part of it. Microsoft frames the work around what it calls medical superintelligence, which it describes as care that would ultimately outdo the strongest clinical teams and health systems now operating, and its argument for that goal is laid out in a companion paper in BMJ Digital Health & AI. Mustafa Suleyman, the chief executive of Microsoft AI, told TechCrunch he expects health models to become an order of magnitude more capable within 24 to 36 months.

From case records to patients

The next test is a live one. Microsoft has started a prospective study with Beth Israel Deaconess Medical Center in Boston, in which consenting patients receive standard care and also talk to MAI-DxO directly, with its recommendations compared against the physicians’ own assessments. Beth Israel is also where Google ran the clinical study of its AMIE diagnostic agent that it published in The Lancet in October, so the two companies’ systems are now being judged in the same hospital. Microsoft is also open-sourcing SDBench for a selection of the NEJM cases, including the gatekeeper, judge and cost-estimation code, and TechCrunch reports that the healthcare foundation model it is training with Mayo Clinic is planned for next year. It sits alongside the general-purpose models Microsoft has been building in-house, such as MAI-Base-2.

Until the Boston results arrive, the evidence is case records read by software. The figure that will matter is how MAI-DxO does when the person answering its questions is a patient in an emergency room rather than a write-up in a journal.