A standardized checklist on meta-analysis reporting in the open science era

Abstract

Meta-analysis is a crucial tool for accumulating evidence in basic and applied research. In the open science era, meta-analysis becomes an important way for integrating open data from different sources. Meanwhile, because of researchers’ high degree of freedom introduced by multiple-steps and multiple-choices in each step of meta-analysis, the openness and transparency are crucial for reproducing the results of meta-analysis. To reflect the transparency and openness of meta-analysis reports published in Chinese journals and improve the transparency and openness of future meta-analysis by Chinese researchers, we developed a Chinese version of a checklist for meta-analysis (Preferred Reporting Items for Open and Reproducible Meta-analysis, PRIOR-MA), which was based on the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) and the principle of openness and transparency. We then surveyed the methods and results of 68 meta-analysis papers published in mainstream Chinese psychological journals in the last five years. Our results revealed that the openness and transparency of Chinese meta-analysis reports need to be improved, especially in the following aspects: the date/time and limitation of literature search, the details of screening and data collection, the flow chart of article screening, the details of effect size transformation, and the evaluation of individual research bias. The checklist, which lists almost all aspects that an open meta-analysis should include, can be used as a guide for future meta-analysis. This paper provides a comprehensive checklist (PRIOR-MA) for open meta-analysis, and provides a reference for future meta-analytic research.