Mustafa Suleyman says Microsoft's MAI Diagnostic system is 1.7x more accurate than emergency department physicians, with the results published in Science
Our MAI Diagnostic system is now 1.7x more accurate than physicians in the ED, even when they have access to the web to look things up. It's also lower cost & does fewer medical tests to get to its diagnosis. Published in the journal Science today...
Context
It is a peer reviewed medical AI paper in Science, but the accuracy figures are Microsoft's own, come from a text only benchmark and not a clinical trial, and the paper itself says they do not establish superiority in real world practice.
Suleyman is CEO of Microsoft AI, and his post summarizes a Science paper titled Sequential diagnosis with language models, which Science lists as accepted August 10, 2026 and published in print October 22, 2026. The paper calls the system DxO for Diagnostic Orchestrator, while Microsoft's blog calls it MAI DxO. According to the paper, all authors are Microsoft employees who hold company equity, and no outside funds were used.
Deep dive
Suleyman's post says the MAI Diagnostic system is now 1.7x more accurate than physicians in the emergency department, even when they have access to the web, that it costs less and orders fewer medical tests, and that the work is published in Science. The attached image is the cover of the October 22 issue, themed AI in Medicine. Microsoft's blog post of the same date describes MAI DxO as working like a panel of virtual physicians that propose diagnoses, choose questions and tests, and challenge the emerging view over several rounds.
The paper reports results on SDBench, a benchmark built from 304 New England Journal of Medicine clinicopathological conference cases and 325 emergency department to admission encounters from MIMIC IV ED. On the emergency cases, DxO with GPT 5 reached 80.3% accuracy against 46.8% for participating generalist physicians, which is the roughly 1.7x ratio. The paper's figure caption labels those emergency physician results as open book, and elsewhere describes open book as having search and reference tools. On the NEJM cases it reported 83.2% against 36.1% for physicians with search tools and 22.3% without.
On cost, the paper is more nuanced than the post. The unconstrained configuration that scored 80.3% had a mean testing cost of $4979 per emergency case, against $3400 for physicians. Budget constrained configurations scored 73.2% at $2634 and 69.5% at $2037, both below the physician figure. Costs are estimates from a Kaiser Permanente Washington price file.
The authors say these are controlled, text only comparisons with individual generalist physicians and do not establish superiority over specialists, multidisciplinary teams or real world practice. They also say the benchmark has no imaging or waveforms, uses only pathological cases, and measures diagnosis alone, not treatment, safety or outcomes. All authors are Microsoft employees.
What to expect next
Microsoft says it will test the system in a prospective study with Beth Israel Deaconess Medical Center in Boston, where patients interact with the system directly and performance is measured against physicians' independent assessments. It also says it is open sourcing SDBench for select NEJM cases so other researchers can reproduce and challenge the evaluation, and the paper says the emergency case subset will be released on PhysioNet subject to credentialing. Watch for independent replications, comparisons against specialist teams, and whether any clinical deployment follows. Microsoft's own page says the true test is performance in real clinical environments.