Abstract
Temporal changes in magnitude of effect sizes reported in many areas of research are a threat to the credibility of the results and conclusions of meta‐analysis. Numerous sequential methods for meta‐analysis have been proposed to detect changes and monitor trends in effect sizes so that meta‐analysis can be updated when necessary and interpreted based on the time it was conducted. The difficulties of sequential meta‐analysis under the random‐effects model are caused by dependencies in increments introduced by the estimation of the heterogeneity parameter τ 2. In this paper, we propose the use of a retrospective cumulative sum (CUSUM)‐type test with bootstrap critical values. This method allows retrospective analysis of the past trajectory of cumulative effects in random‐effects meta‐analysis and its visualization on a chart similar to CUSUM chart. Simulation results show that the new method demonstrates good control of Type I error regardless of the number or size of the studies and the amount of heterogeneity. Application of the new method is illustrated on two examples of medical meta‐analyses. © 2016 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.
Highlights
Meta-analysis is a statistical technique used to combine results from related but independent studies in order to provide an estimate of the overall treatment effect
We propose the use of Gombay (2003) truncated cumulative sum (CUSUM)-type test statistic with critical values estimated by the bootstrap
When K = 20, the values of Type I errors achieved by GDL and GH are somewhat higher compared with GMP and GREML, but as K increases to 50 and 100, there is very little difference between the four tests, as is clearer from Figure S7 in Web Appendix
Summary
Meta-analysis is a statistical technique used to combine results from related but independent studies in order to provide an estimate of the overall treatment effect. Similar temporal changes have been reported in education (Hyde et al, 1990), medicine (Gehr et al, 2006), psychology (Brugger et al, 2011; Twenge et al, 2008; Grabe et al, 2008), to mention but a few. These changes in effect sizes can be dramatic and often lead to the loss or gain of statistical significance (Kulinskaya and Koricheva, 2010). In case of a monotonic temporal trend, meta-regression with time as a covariate can be used to evaluate such a trend and to adjust for it, see Shi and Copas (2004); Baker and Jackson (2010)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
