Do “Ten simple rules for neuroimaging meta-analysis” receive equal attention and accurate quotation? An examination on the quotations to an influential neuroimaging meta-analysis guideline

Abstract

The collection of recommendations and guidelines for conducting and reporting neuroimaging meta-analyses, called “Ten simple rules for neuroimaging meta-analysis” by Müller et al., has been published for a few years. Here, the papers citing this reference were examined to evaluate the rationale of the quotations and what quotation errors existed. In May 2023, an online query via Scopus identified 386 papers citing this reference, 2 of which were inaccessible. The resultant 384 papers were checked to identify the total number of quotations to the reference, the exact quotations, which of the ten recommendations/rules was concerned by each quotation, and if any quotation error existed. Results found that the reference by Müller et al. were quoted 804 times by the 384 papers, meaning an average of 2.1 quotations per paper. Out of the 804 quotations, the three rules that the researchers most frequently referred were the power of the meta-analysis (Rule #2, 14.1%), the consistency of the search coverage and reference space (Rule #4, 13.8%), and the statistical threshold (Rule #8, 10.2%). Overall, 63 quotations from 51 papers contained some errors. In other words, 7.8% (63/804) of the quotations contained errors and they involved 13.3% (51/384) of the papers. The commonest quotation errors were dealing with a failure to substantiate the assertion, unrelated to the assertion, and oversimplification of the original notion. Some notable quotation error examples were to quote Müller et al. to substantiate the assertion of having at least 10 datasets to be considered to have adequate power for ES-SDM meta-analysis (no such recommendation), and having a misquoted primary cluster-forming threshold of p < 0.05 or p < 0.005 (should be p < 0.001). The neuroscience community should be cautious and double-check the accuracy of assertions, even with a quotation.