Abstract
Methodological research rarely generates a broad interest, yet our work on the validity of cluster inference methods for functional magnetic resonance imaging (fMRI) created intense discussion on both the minutia of our approach and its implications for the discipline. In the present work, we take on various critiques of our work and further explore the limitations of our original work. We address issues about the particular event‐related designs we used, considering multiple event types and randomization of events between subjects. We consider the lack of validity found with one‐sample permutation (sign flipping) tests, investigating a number of approaches to improve the false positive control of this widely used procedure. We found that the combination of a two‐sided test and cleaning the data using ICA FIX resulted in nominal false positive rates for all data sets, meaning that data cleaning is not only important for resting state fMRI, but also for task fMRI. Finally, we discuss the implications of our work on the fMRI literature as a whole, estimating that at least 10% of the fMRI studies have used the most problematic cluster inference method (p = .01 cluster defining threshold), and how individual studies can be interpreted in light of our findings. These additional results underscore our original conclusions, on the importance of data sharing and thorough evaluation of statistical methods on realistic null data.
Highlights
In our previous work (Eklund, Nichols, & Knutsson, 2016a), we used freely available resting state functional magnetic resonance imaging data to evaluate the validity of standard fMRI inference methods
We found that parametric statistical methods (e.g., Gaussian random field theory [GRFT]) perform well for voxel inference, where each voxel is separately tested for significance, but the combination of voxel inference and familywise error (FWE) correction is seldom used due to its low statistical power
For cluster inference, where groups of voxels are tested together by looking at the size of each cluster, we found that parametric methods perform well for a high cluster defining threshold (CDT; p = .001) but result in inflated false positive rates for low CDTs (e.g., p = .01)
Summary
In our previous work (Eklund, Nichols, & Knutsson, 2016a), we used freely available resting state functional magnetic resonance imaging (fMRI) data to evaluate the validity of standard fMRI inference methods. We found that parametric statistical methods (e.g., Gaussian random field theory [GRFT]) perform well for voxel inference, where each voxel is separately tested for significance, but the combination of voxel inference and familywise error (FWE) correction is seldom used due to its low statistical power. For this reason, the false discovery rate is in neuroimaging (Genovese, Lazar, & Nichols, 2002) often used to increase statistical power. The nonparametric permutation test is not based on these assumptions (Winkler, Ridgway, Webster, Smith, & Nichols, 2014) and, produced nominal results for all two-sample t tests, but in some cases failed to control FWE for one-sample t tests
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
