At the end of June, Babylon Health’s AI has demonstrated its ability to provide health advice which is on-par with practicing clinicians from the Royal College of Physicians in London in a series of robust tests. The tests included the relevant sections of the MRCGP exam, which is the final test for trainee General Practitioners (GPs), set by the Royal Collegeof General Practitioners (RCGP).
Babylon took a representative sample-set of questions testing diagnostic skills from publicly available RCGP sources, as well as independently published examination preparation materials, and mapped these to the current RCGP curriculum in order to ensure the questions resembled actual MRCGP questions as closely as possible. The average pass mark over the past five years for real-life doctors was 72 percent. In sitting the exam for the first time, Babylon’s AI scored 81 percent. As the AI continues to learn and accumulate knowledge, Babylon expects that subsequent testing will produce significant improvements in terms of results.
Important though exams are, doctors are presented with a much wider range of illnesses and conditions in their daily practice. Therefore, to further test the AI’s capabilities, Babylon collaborated with medical professionals to test Babylon’s AI alongside seven highly-experienced primary care doctors using 100 independently-devised symptom sets (or ‘vignettes’). Babylon’s AI scored 80 percent for accuracy, while the seven doctors achieved an accuracy range of 64-94 percent. The accuracy of the AI was 98 percent when assessed against conditions seen most frequently in primary care medicine. In comparison, when Babylon’s research team assessed experienced clinicians using the same measure, their accuracy ranged from 52-99 percent.
“Tonight’s results clearly illustrate how AI-augmented health services can reduce the burden on healthcare systems around the world. Our mission is to put accessible and affordable health services into the hands of every person on Earth,” said Dr. Ali Parsa, Babylon Health’s founder and CEO. “Babylon’s latest artificial intelligence capabilities show that it is possible for anyone, irrespective of their geography, wealth or circumstances, to have free access to health advice that is on-par with top-rated practicing clinicians.”
Source: Babylon Health