Modeling multiple dependent variables in meta-analysis of single-case experimental design using multilevel modeling.

Abstract

Although meta-analyses of single-case experimental design (SCED) often include multiple types of dependent variables (DVs), multiple DVs are rarely considered within models in the analysis. Baek et al. (Journal of Experimental Education, 90(4), 934-961, 2022) identified several statistical issues that arise when researchers fail to model multiple DVs in meta-analyses of SCED data. However, the degree to which non-modeling of multiple DVs impacts the results of the meta-analysis of SCED has not been fully examined. In this simulation study, we have systematically investigated the impact of non-modeling of multiple DVs when analyzing meta SCED data using multilevel modeling. The result demonstrates that modeling multiple DVs has advantages over the non-modeling option for meta-analysis of SCED. Modeling multiple DVs enables the determination of precise effects from different DVs in addition to the unbiased and accurate average effect and accurate estimates and inferences for the error variances at the study level as well as the observation level. The current study also reveals potential factors (i.e., the number of DVs, degree of heterogeneity in the level-1 error variances and autocorrelation, and presence of the moderator effect) that impact the precision and accuracy of the variance parameters.