The Significance Level of Meta-Regression's Standard Hypothesis Test

Abstract

Meta-regression is routinely used in the context of meta-analysis to assess the potential impact of covariates on the treatment effect. The limitations and pitfalls of this type of analysis have been documented, however, in particular, it has been claimed that the actual significance levels of the resulting tests can be higher than nominal levels. In order to examine the validity of this claim analytically, the distribution of meta-regression's standard test statistic, in the simplified scenario where the studies are the same size, is derived. The resulting significance levels can be used to give an indication of these more generally and it is found that relatively large numbers of studies are needed to provide actual significance levels that are sufficiently similar to the corresponding nominal levels. These findings can be used to inform those performing standard meta-regression hypothesis tests, so that they can avoid adopting the usual procedure in situations where this is likely to be inappropriate.