Meta-analysis of single-case experimental designs: How can alternating treatments and changing criterion designs be included?

Abstract

This text focuses on the quantitative integration of studies using alternating treatments designs (ATDs) and changing criterion designs (CCDs) with other types of single-case experimental designs (SCEDs), such as multiple-baseline and withdrawal, which have received more attention in terms of statistical developments. First, a review of how published meta-analyses have dealt with ATDs and CCDs suggests a variety of analytical strategies and insufficient transparency in reporting the exact comparisons performed. Second, we review data-analytical techniques for ATDs and CCDs, looking for alternatives that can be useful for meta-analysis. Third, without losing sight of the underlying logic of the ATDs and CCDs, we propose using multilevel models, comparing data paths in ATDs and the baseline to the last intervention subphase in CCDs. Furthermore, we suggest additional evaluations of the data pattern, beyond the quantification of the magnitude of effect. We also advocate for transparent reporting of how exactly the conditions are being compared for the different SCEDs. This is possible by specifying the design matrices for the multilevel models used. Fourth, in the context of one of the meta-analyses reviewed, we comment on a series of analytical decisions that need to be made before carrying out a meta-analysis using multilevel models.