Imbalances in Two-Level Hierarchical Linear Modeling of Single-Case Data: A Monte Carlo Simulation Study

Abstract

Hierarchical linear modeling (HLM) is a promising approach that can be applied to explain variability in intervention effectiveness between participants in single-case experimental design (SCED) research. This approach allows the inclusion of participant characteristics as moderators to account for variability in intervention effectiveness. However, little is known about the performance of HLM for analysis with the inclusion of imbalanced intervention starting points and imbalanced moderators. Therefore, the goal of this study is to empirically evaluate the statistical properties of this model through a large-scale Monte Carlo simulation study under these scenarios. The results indicate that imbalanced intervention starting points have no impact on the statistical properties of estimating intervention effects. On the other hand, imbalanced moderators have an impact on estimating the intervention and moderator. How the conditions of imbalanced moderators influence the performance of two-level HLM depends on how the moderators are coded. Two general conclusions can be made. The model results in more favorable statistical properties when (i) a larger number of participants are included and (ii) the variability in continuous moderators is larger. Practical implications for the design and analysis of SCEDs are discussed.