Addressing Missing Data Due to COVID-19: Two Early Childhood Case Studies

Abstract

One part of COVID-19’s staggering impact on education has been to suspend or fundamentally alter ongoing education research projects. This article addresses how to analyze the simple but fundamental example of a multi-cohort study in which student assessment data for the final cohort are missing because schools were closed, learning was virtual, and/or assessments were canceled or inconsistently ­collected due to COVID-19. We argue that current best-practice recommendations for addressing missing data may fall short in such studies because the assumptions that underpin these recommendations are violated. We then provide a new, simple decision-making framework for empirical researchers facing this situation and provide two empirical examples of how to apply this framework drawn from early childhood studies, one a cluster randomized trial and the other a descriptive longitudinal study. Based on this framework and the assumptions required to address missing data, we advise against the standard recommendation of adjusting for missing outcomes (e.g., via imputation or weighting). Instead, we generally recommend changing the target quantity by restricting to fully observed cohorts or by pivoting to focusing on an alternative outcome.