Assessing Intervention Effects in the Presence of Missing Scores

Chao-Ying Joanne Peng,Li-Ting Chen

doi:10.3390/educsci11020076

Chao-Ying Joanne Peng, Li-Ting Chen

Open Access

https://doi.org/10.3390/educsci11020076

Copy DOI

Abstract

Due to repeated observations of an outcome behavior in N-of-1 or single-case design (SCD) intervention studies, the occurrence of missing scores is inevitable in such studies. Approximately 21% of SCD articles published in five reputable journals between 2015 and 2019 exhibited evidence of missing scores. Missing rates varied by designs, with the highest rate (24%) found in multiple baseline/probe designs. Missing scores cause difficulties in data analysis. And inappropriate treatments of missing scores lead to consequences that threaten internal validity and weaken generalizability of intervention effects reported in SCD research. In this paper, we comprehensively review nine methods for treating missing SCD data: the available data method, six single imputations, and two model-based methods. The strengths, weaknesses, assumptions, and examples of these methods are summarized. The available data method and three single imputation methods are further demonstrated in assessing an intervention effect at the class and students’ levels. Assessment results are interpreted in terms of effect sizes, statistical significances, and visual analysis of data. Differences in results among the four methods are noted and discussed. The extensive review of problems caused by missing scores and possible treatments should empower researchers and practitioners to account for missing scores effectively and to support evidence-based interventions vigorously. The paper concludes with a discussion of contingencies for implementing the nine methods and practical strategies for managing missing scores in single-case intervention studies.

Highlights

To fill in the gaps in the literature, we aim to address three issues surrounding missing data in this paper: (A) the prevalence of missing data in single-case design (SCD) studies, (B) problems caused by missing data to the validity of conclusions, and (C) treatments of missing data in intervention studies and needs for additional research
All 90 articles chose to analyze remaining or available data, leaving missing data untreated. Another 2% did not provide sufficient information to help determine if missing data existed
Class-level assessment presented in Section 5.1 led us to conclude that the RC intervention was effective in decreasing the intervals with disruptive behaviors from SSR1 to

Summary

Introduction

The N-of-1 or single-case design (SCD) studies have long been used for identifying and implementing effective interventions in school settings [1,2]. SCD studies in education are characterized by repeated measures of an outcome behavior over time, participants serving as their own controls, and smaller samples as compared to group design studies. A counselor may examine effects of a motivational interviewingbased program on children’s classroom behavior [4]. SCD studies may include no interventions by researchers [6]. In such cases, a few participants are observed repeatedly over time and the primary purpose is to examine relationships between observed outcome and predictors

Objectives

Results

Discussion

Conclusion