A Comparison of Fixed-Effects and Random-Effects Models for Multivariate Meta-Analysis Using an SEM Approach

Abstract

This study compared fixed-effects (FE) and random-effects (RE) models in meta-analysis for synthesizing multivariate effect sizes under the framework of structural equation modeling. Monte Carlo simulations were conducted to examine the performance characteristics of the two models under different data conditions. The results indicated that, for the homogeneous case, there was little difference between the FE model and the RE model applications. But the FE model had better performance in standard error estimation when number of studies is not large and the sample size of primary studies is small. Furthermore, under the heterogeneous case, FE model exhibited biased estimates of population parameters and extreme levels of inflated Type I error in testing the effect size estimates. However, RE model maintained unbiased estimates of the population parameters, and controlled Type I error well under various data conditions investigated. These findings provided empirical evidence that it is likely that RE model application in a meta-analysis would be preferred when the number of primary studies and the sample sizes in the primary studies are reasonably large, and FE model could be favored for situations with smaller numbers of studies and uniformly small sample sizes of primary studies.