Addressing Inadequacies of the Observation Survey of Early Literacy Achievement

Abstract

AbstractThe authors used nationally based, random sample data from three different years (2009–2010, 2011–2012, and 2014–2015) for nearly 20,000 first‐grade students (n = 9,760, 3,657, and 3,121, respectively) to examine long‐reported inadequacies of a commonly used early literacy assessment tool, the Observation Survey of Early Literacy Achievement (OSELA), chief among them the skewness and nonequal interval nature of the scores that are obtained on its six individual tasks. Such inadequacies prevented the individual task scores from being used for program evaluation, screening, and progress monitoring. To mitigate these OSELA limitations, the authors employed Rasch analysis to create a scale that can be used to track a student's literacy achievement based on the combination of OSELA task scores. Dimensionality analyses revealed that the OSELA measures one factor, which supported the decision to combine the individual tasks to compute one total score. The equal‐interval total score was normally distributed at the beginning, middle, and end of first grade. Further, the authors conducted a predictive validation study of the total score to identify a range of cut scores that can be used in the fall of first grade to predict reading failure by year end. The authors maintain that the total score provides a more precise and efficient means of screening young students for reading failure and evaluating their progress over time. Implications for using the total score to make screening decisions and measure early reading progress are discussed.