Microsoft says its latest medical AI can diagnose illnesses more accurately than human doctors, but the technology has yet to prove itself in a real clinical setting. The company's Diagnostic Orchestrator, known as MAI-DxO, achieved substantially higher diagnostic accuracy than generalist physicians in a peer-reviewed study published in Science.
However, the results come from a controlled test using existing medical records rather than patients seeking treatment. The distinction matters because a correct answer in a research exercise does not necessarily translate into safer or more effective care.
Microsoft AI Chief Executive Mustafa Suleyman has already predicted major changes to healthcare, telling The Verge that 'Next year we're going to see extremely impressive deployments of intelligent systems in production.'
The company is now preparing to test its approach in hospital settings, where its performance can be assessed under real clinical conditions. Until that evidence emerges, the findings offer a promising research result rather than proof that AI is ready to take over medical diagnosis.
Microsoft AI Diagnostic Orchestrator Achieves Higher Accuracy Than Doctors
The study evaluated MAI-DxO using two sets of medical cases: 304 complex cases published in The New England Journal of Medicine and 325 emergency department encounters drawn from the MIMIC-IV-ED database.
According to the research published in Science, MAI-DxO achieved 83.2 per cent diagnostic accuracy on the complex cases. Generalist physicians who could consult reference materials scored 36.1 per cent in the study's comparison, while their accuracy without those resources was 22.3 per cent.
The comparison was with individual generalists, not specialist teams working together on difficult cases. On the emergency department cases, the AI scored 80.3 per cent, while the doctors achieved 46.8 per cent.
How the System Performed Monetarily
The system also performed well in tests that considered the estimated costs of reaching a diagnosis, although the results depended on how it was configured.
In the complex-case comparison, the highest-accuracy version achieved 83.2 per cent accuracy at an estimated cost of US$5,431 (approximately £4,000) per case, compared with US$4,725 (approximately £3,500) for generalist physicians using reference materials.
Microsoft also tested more budget-conscious versions that cost less and remained more accurate than the physician averages, although their accuracy was lower than that of the highest-scoring setup. These figures are estimates based on the study's methodology, not savings demonstrated in hospitals.
One important feature of the research is that MAI-DxO was not simply handed a complete medical history and asked to select an answer. Instead, it worked through each case in stages, deciding what information to request, which investigations to pursue and how to revise its assessment as new findings became available.
The system simulates a panel of virtual clinicians. This arrangement allows different diagnostic approaches to be considered before the system settles on a conclusion. The researchers argue that this process more closely resembles clinical reasoning than conventional AI tests that provide all the relevant information upfront.
The findings should also be considered in light of the study's authorship. The research was conducted by Microsoft researchers, and the authors are company employees who hold equity in Microsoft.
Although the paper was peer-reviewed and published in Science, independent replication would help establish whether the results hold up beyond the team's own evaluation.
Why Microsoft's Medical AI Still Needs Real-World Testing
The central limitation is that the research measured performance on historical case records, not how MAI-DxO would behave when interacting with patients during actual medical care.
The benchmark used text-based case information and simulated the process of gathering further details. It did not involve direct consultations with the patients whose cases were assessed, physical examinations conducted by the AI or live interpretation of medical images.
It therefore could not fully reproduce the uncertainty and interaction involved when patients describe their symptoms during actual medical care.
Nor did the study establish whether using the system would improve recovery rates, prevent complications or reduce the number of missed diagnoses.
There is also a difference between identifying the correct diagnosis and deciding what to do for a patient. Doctors must weigh treatment options, explain risks, account for individual circumstances and respond when a patient's condition changes. The study did not establish that MAI-DxO can perform these responsibilities as effectively as clinicians.
In its announcement of the research, Microsoft outlined plans to work with Beth Israel Deaconess Medical Center in Boston to evaluate the system in a clinical environment. The proposed work will examine its diagnostic reasoning in settings that involve real patients rather than relying solely on archived cases.
The findings have also been reported by Reuters, which noted that the system is designed to gather information progressively, much like a doctor investigating a case.
Researchers will need to establish how reliably the system handles incomplete information, whether its recommendations are safe and how it performs alongside healthcare professionals.
For now, Microsoft has demonstrated that its AI can achieve high diagnostic accuracy under controlled testing conditions. Whether it can deliver the same results in the unpredictable environment of everyday healthcare remains an open question.
