Software bugs for MRI scanners question 40,000 scientific studies

A thorough study of the methodology of research using magnetic resonance imaging makes invalid the results of an entire branch of science. For several decades, neuroscientists and cognitive psychologists have used the AFNI , SPM, and FSL statistical programs to analyze fMRI data. As it turned out, due to incorrect algorithms, these programs can return up to 70% false positive results instead of the expected 5%.

Thus, approximately 40,000 scientific papers published in recent decades on the basis of fMRI data have been questioned for one hour. In addition, a new assessment of validity can have a strong impact on the interpretation of the results of neuroimaging.

Functional magnetic resonance imaging has been used in medicine for more than 25 years, and it is surprising that so far the most commonly used statistical methods used in MRI software have not been confirmed with real data, the authors of a study published June 27, 2016 in Proceedings of the National Academy of Sciences .
')
Statistical methods are the basis for interpreting fMRI results. This method of neuroimaging allows you to determine the activation of a certain area of ​​the brain during its normal functioning under the influence of various physical factors (for example, body movement) and under various pathological conditions.

In the process of neuroimaging, according to fMRI scanning results, the brain is divided into tiny sections (voxels) on the obtained high-resolution scanning results. The software then scans the voxels and combines them into “active” clusters that correspond to the activation of a specific region of the brain.


Image of fMRI with yellow clusters in which "increased activity is noted"

The problem is that this statistical procedure is performed incorrectly and, as it turned out, does not meet the scientific requirements for maximum error of statistical data. The authors of the scientific paper write: "Our results show that the main cause of incorrect clustering is spatial autocorrelation functions that do not correspond to the assumed Gaussian distribution."

For example, a bug in 3dClustSim software (part of the AFNI package) was present for 15 years and was fixed only in May 2015, during the preparation of this study, say the authors of a research paper from Linkoping University (Sweden) and the University of Warwick (United Kingdom).

During the verification of the results of the work of the programs, the data of functional magnetic resonance imaging at rest of 499 healthy patients from the control group were compared to obtain 3 million tomograms. It turned out that the percentage of false-positive results is much higher than the estimated 5% and reaches 70%. The researchers conclude that parametric statistical methods that are used in popular software packages SPM, FSL and AFNI are in fact unsuitable for cluster analysis of fMRI results, because the results do not correspond to the normal distribution.

This is what happens when research is carried out without checking the correctness of medical instruments for compliance with scientific standards of statistics.

Authors of scientific papers from the medical field may now have to conduct research again in order to repeat the result obtained using reliable tools using valid statistical methods.

So when you meet another study with fMRI, you should take these results with caution. There is even an XKCD comic on this topic.



Surprisingly, the comics were drawn long before the study of statistical methods for the fMRI voxels clustering by scientists from Linkoping University and the University of Warwick.