Doctors diagnose better than Internet services or applications

Powerful computer systems are now successfully competing with people in areas such as playing chess, go, composing music, or even creating movie trailers. But can a car make a diagnosis better than a human doctor?

Project participants from Harvard Medical School believe that this is not always the case. A group of scientists conducted a study, the purpose of which is to find out who makes the diagnosis more precisely - people or specialized applications. The results of the study were published in the authoritative edition of JAMA Internal Medicine.

As it turned out, human doctors make the correct diagnosis twice as often as most of the medical applications offered on the network. In total, the project participants checked the correctness of the work of 23 such programs . This work, according to its authors, is the first to compare the diagnostic work of doctors and popular applications.
')
Over the past 20 years, medical applications, designed in the form of interactive questionnaires, have become very popular. Every year, hundreds of millions of people use such programs instead of going to the doctor, believing that this way you can get the correct diagnosis without wasting time. In addition, sometimes this software is free. Nobody has conducted any research on the effectiveness of such applications, and scientists from Harvard decided to become the first.

The study invited 234 doctors to participate. They were asked to evaluate 45 different clinical cases. This number included diseases with the usual symptoms, as well as cases in which a particular disease was accompanied by a somewhat distorted symptomatic picture. For each of the proposed option, the doctors were asked to make a diagnosis.

It was allowed to make the main diagnosis and two alternative. As it turned out, the doctors made the correct diagnosis in 72% of cases with obvious symptoms. But diagnostic applications coped with the task much worse, giving the correct answer only in 34% of the options. 84% of doctors put the correct answer, assessing it as the main one. Only 51% of applications handled the same task.

Even worse, the applications showed themselves in cases where the symptoms were not too pronounced, or the disease manifested itself as distorted symptoms.



“It’s clear now that people are winning programs in terms of diagnostic accuracy. Now one of the most important issues is improving the accuracy of applications and services, ”said Ateyev Mehrota, head of the work.

Despite the fact that human doctors showed much better results than applications, specialists also have something to improve. According to statistics, doctors are mistaken in the diagnosis of diseases in 10-15% of cases. 15% of errors is quite a lot. In many cases, such a mistake can cost a person health or even life.

“In clinical diagnosis today there is as much experience and skill as science. But there are excellent prospects for diagnostic technology. In such tools there is a real technological appeal, ”says the author of the project.


As for applications that can diagnose, we are talking about official programs and services developed by the Harvard Medical School, various hospitals, insurance companies and government organizations. Such programs ask the user for a list of symptoms, analyze the answers and make a diagnosis. Also, the diagnostic program usually advises a patient with difficult problems to immediately go to the doctor.

The results of the programs themselves are also different. According to the authors of the study, the most accurate are such programs as Isabel, iTriage, Mayo Clinic, and Symcat. “These programs can be useful for those who want to determine whether they need to go to the doctor. But in this case, the testimony of the program should not be taken on faith immediately, ”says the head of the study.

Cognitive systems like IBM Watson are also engaged in diagnostics. In this case, the work is done differently than in assessing the condition of a person using diagnostic programs. Such a system is able to analyze huge arrays of unstructured data, identifying interrelated elements using the latest data for each specific case.

Now it is safer for the patient to go to the doctor, and not to use some kind of program (albeit certified by the hospital) to get a diagnosis. Perhaps in the near future the situation will change. Scientists talk about the need to improve the accuracy of diagnostic programs and services in order to develop in the future the second, third and subsequent generations of such software systems. Only if the programs become more accurate, will it be possible to talk about the possibility of replacing communication with a doctor with a diagnostic computer.

DOI: 10.1001 / jamainternmed.2016.6001