Reading Assessment Data Research Articles

Inadequate foundational decoding skills constrain global literacy goals for pupils in low- and middle-income countries.

Learning to read is the most important outcome of primary education. However, despite substantial improvements in primary school enrolment, most students in low- and middle-income countries (LMICs) fail to learn to read by age 10. We report reading assessment data from over half a million pupils from 48 LMICs tested primarily in a language of instruction and show that these pupils are failing to acquire the most basic skills that contribute to reading comprehension. Pupils in LMICs across the first three instructional years are not acquiring the ability to decode printed words fluently and, in most cases, are failing to master the names and sounds associated with letters. Moreover, performance gaps against benchmarks widen with each instructional year. Literacy goals in LMICs will be reached only by ensuring focus on decoding skills in early-grade readers. Effective literacy instruction will require rigorous systematic phonics programmes and assessments suitable for LMIC contexts.

How curriculum and assessment policies affect the role of reading in an assessment culture: a New Zealand case study

ABSTRACT Reading assessment is a critical feature of education systems. The aim of this study is to explore reading assessment policy in the New Zealand (NZ) assessment culture. We reviewed research publications, national policy documents and test protocols on reading for year levels 6–9 (ages 9–13). We found that assessment data are broadly conceptualised and used for multiple purposes, that the representation of reading assessment data is located in an improvement-oriented environment and that the use of reading assessment for improvement purposes involves multiple actors and organisation levels. We discuss to what extent reading assessment policy supports student learning and teacher professional learning.

The Impact of a High-Dosage Tutoring Program on Reading Achievement of Beginning Readers: A Multi-Level Analysis

The purpose of this randomized control trial (RCT) was to examine the impact of one-to-one high-dosage tutoring on reading achievement of kindergarten and first grade students, many at risk of reading failure. This nearly two-year study collected reading assessment data from 185 students, from across 13 classrooms, randomly assigned to either the one-to-one high-dosage tutoring group or the teacher-instruction-only group. Treatment students received tutoring support 2-3 times per week across the school year from specially trained tutors using a digital literacy platform. Tutoring focused on sequenced skills beginning with letter identification and sounds and moved to reading stories. The results of the Hierarchical Linear Modeling (HLM) analysis, from both Chapter One assessments as well as the nationally recognized assessments, indicate that Chapter One’s high impact tutoring program is highly effective in helping at-risk kindergarten and first grade students achieve reading proficiency. Tutored students outperformed non-tutored students on multiple measures of reading achievement.

Preschool participation and students’ learning outcomes in primary school: Evidence from national reform of pre-primary education in Ethiopia

This study examines whether a large expansion of pre-primary education in Ethiopia affected subsequent students’ learning outcomes during the national reform of pre-primary education. The study utilizes two comparable, representative early grade reading assessment data that straddle the reform period from 2010 to 2016, during which enrolment rates in pre-primary education soared by nearly ten times nationwide. We find that associations between preschool participation and literacy outcomes were positive and significant after the expansion, yet no such relationships were observed before the reform. However, there was little heterogeneity in the gains of the preschool participation by gender, urbanity, and parental literacy. We discuss implications for ongoing reform, including strategic and inclusive policy designed to close the learning gap between children from advantaged and disadvantaged backgrounds.

Bayesian Item Response Theory Models With Flexible Generalized Logit Links.

In educational and psychological research, the logit and probit links are often used to fit the binary item response data. The appropriateness and importance of the choice of links within the item response theory (IRT) framework has not been investigated yet. In this paper, we present a family of IRT models with generalized logit links, which include the traditional logistic and normal ogive models as special cases. This family of models are flexible enough not only to adjust the item characteristic curve tail probability by two shape parameters but also to allow us to fit the same link or different links to different items within the IRT model framework. In addition, the proposed models are implemented in the Stan software to sample from the posterior distributions. Using readily available Stan outputs, the four Bayesian model selection criteria are computed for guiding the choice of the links within the IRT model framework. Extensive simulation studies are conducted to examine the empirical performance of the proposed models and the model fittings in terms of "in-sample" and "out-of-sample" predictions based on the deviance. Finally, a detailed analysis of the real reading assessment data is carried out to illustrate the proposed methodology.

No learning loss in Sweden during the pandemic: Evidence from primary school reading assessments

The COVID-19 pandemic has led to worldwide school closures, with a risk of learning loss. Sweden kept primary schools open, but it is unknown whether student and teacher absence and pandemic-related stress factors affected teaching and student progress negatively. In this study, reading assessment data from 97,073 Swedish primary school students (grades 1-3) were analysed to investigate potential learning loss. Results showed that word decoding and reading comprehension scores were not lower during the pandemic compared to before the pandemic, that students from low socio-economic backgrounds were not especially affected, and that the proportion of students with weak decoding skills did not increase during the pandemic. Study limitations are discussed. We conclude that open schools benefitted Swedish primary school students.

A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models

Diagnostic classification models (DCMs) are restricted latent class models with a set of cross-class equality constraints and additional monotonicity constraints on their item parameters, both of which are needed to ensure the meaning of classes and model parameters. In this paper, we develop an efficient, Gibbs sampling-based Bayesian Markov chain Monte Carlo estimation method for general DCMs with monotonicity constraints. A simulation study was conducted to evaluate parameter recovery of the algorithm which showed accurate estimation of model parameters. Moreover, the proposed algorithm was compared to a previously developed Gibbs sampling algorithm which imposed constraints on only the main effect item parameters of the log-linear cognitive diagnosis model. The newly proposed algorithm showed less bias and faster convergence. An analysis of the 2000 Programme for International Student Assessment reading assessment data using this algorithm was also conducted.

Impact of time of day on radiology image interpretations

To examine the impact of the time of day on radiologists' mammography reading performance. Retrospective mammographic reading assessment data were collected from the BreastScreen Reader Assessment Strategy database and included timestamps of the readings and reader-specific demographic data of 197 radiologists. The radiologists performed the readings in a workshop setting with test case sets enriched with malignancies (one-third of cases were malignant). The collected data were evaluated with an analysis of covariance to determine whether time of day influenced radiologists' specificity, lesion sensitivity or the jackknife alternative free-response receiver operating characteristic (JAFROC). After adjusting for radiologist experience and fellowship, specificity varied significantly by time of day (p=0.027), but there was no evidence of any significant impact on lesion sensitivity (p=0.441) or JAFROC (p=0.120). The collected data demonstrated that specificity during the late morning (10.00-12.00) was 71.7%; this was significantly lower than in the early morning (08.00-10.00) and mid-afternoon (14.00-16.00), which were 82.74% (p=0.003) and 81.39% (p=0.031), respectively. Specificity during the late afternoon (16.00-18.00) was 73.95%; this was significantly lower than in the early morning (08.00-10.00) and mid-afternoon (14.00-16.00), which were 82.74% (p=0.003) and 81.39% (p=0.031), respectively. The results indicated that the time of day may influence radiologists' performance, specifically their ability to identify normal images correctly.

George F. Bereday Award for 2019

Previous articleNext article FreeAwardGeorge F. Bereday Award for 2019PDFPDF PLUSFull Text Add to favoritesDownload CitationTrack CitationsPermissionsReprints Share onFacebookTwitterLinked InRedditEmailQR Code SectionsMoreEach year the Bereday Award Committee selects the most outstanding Comparative Education Review (CER) article for the George F. Bereday Award. The selection committee includes scholars who are chosen for their breadth of methodological skills and intellectual rigor. This year’s committee was chaired by Thomas Luschei (Claremont Graduate University), and included Yi-Lin Chiang (National Chenchi University), Marlaine Lockheed (Princeton University), Payal Shah (University of South Carolina), and Rolf Straubhaar (Texas State University). From the 23 research articles published in volume 63 (2019) of CER (selected from almost 200 annual submissions), the Committee awarded the honor to Amy Jo Dowd and Lesley Bartlett for their article “The Need for Speed: Interrogating the Dominance of Oral Reading Fluency in International Reading Efforts,” published in May. The committee reported the following: “The article examines the use of correct words per minute (CWPM) as a measure of the impact of reading interventions. Drawing on reading assessment data from 11 country sites in monolingual and multilingual populations, the authors conclude that a global CWPM standard ‘rests upon untenable assumptions.’ … The committee noted the article’s clarity and strength of argument, rigorous conceptual framework, and the strong potential impact of the research on education policy and the field of comparative education.” The CER editors congratulate Amy Jo Dowd and Lesley Bartlett on this honor.The committee also noted the high quality of the other articles in volume 63 and offered an honorable mention to two articles: Tracy Brunette, Benjamin Piper, Rachel Jordan, Simon King, and Rehemah Nabacwa, “The Impact of Mother Tongue Reading Instruction in Twelve Ugandan Languages and the Role of Language Complexity, Socioeconomic Factors, and Program Implementation,” published in November, and Sarah Dryden-Peterson and Celia Reddick, “‘What I Believe Can Rescue That Nation’: Diaspora Working to Transform Education in Fragility and Conflict,” published in May.George Zygmunt Fijalkowski Bereday, born in Warsaw in 1920, was both the founding editor of CER and a cofounder of the Comparative Education Society, along with his close colleague William Brickman. A graduate of London and Oxford (while also serving in the British army during World War II), Professor Bereday subsequently arrived in the United States for PhD studies at Harvard’s Graduate School of Education. Later he received a JD from Columbia Law School, where he studied while teaching comparative education, sociology, and juvenile law at Columbia Teachers College from 1955 until his untimely death in 1983. William Brickman (writing in the fall of that year in Western European Education) called Bereday an extraordinarily talented and gifted personality: “A polyglot, he read, spoke, comprehended, and lectured in several Slavic, Germanic, and Romance languages … and he attained a practical ability in the use of Japanese.” Brickman also enthused that “Professor Bereday represented a rare blend of the classical and the modern, of Eastern and Western European education and scholarship, and of the Occidental and Oriental cultures” and that he “exhibited endless evidence of humanism toward other cultures and humanitarianism toward persons of all ethnic, racial, and national backgrounds.” Previous articleNext article DetailsFiguresReferencesCited by Comparative Education Review Volume 64, Number 2May 2020 Sponsored by the Comparative and International Education Society Article DOIhttps://doi.org/10.1086/708238 © 2020 by the Comparative and International Education Society. All rights reserved.PDF download Crossref reports no articles citing this article.

Advances in modelling visual symptoms and visual skills

Though it seems to be a natural process of physical development, not everyone learns on their own how to use their eyes effectively. Poor binocular vision skills negatively impact a wide range of daily life activities, including reading. This study extends previous work using existing data in multidimensional causal Rasch models of vision. Optometry, survey, and reading assessment data are evaluated in a common frame of reference. Thirty-three students performed vergence and tracking exercises in about 12 to 38 sessions each involving a sequence of five computerized training modules. The number of training modules experienced in any given session ranged from one to five. There were 4, 064 total measured cases, with about 123 measures per student. Paired comparison t-tests of early and late measures show significant gains in vision quality. Two regression models predict reading scores on the basis of vision data. Results support further study of how training in Functional Binocular Vision might contribute to improved outcomes in reading education.

Teacher inquiry of using assessments and recommendations in teaching early reading

Previous research point to difficulties for teachers to interpret reading assessment data with regard to instructional decisions. This study explored Swedish primary teachers' use of assessments and recommendations, in order to be able to target individual needs. Eight teachers participated in a reading program and were interviewed in focus-group meetings. The analysis of teacher narratives stemming from assessment use resulted in three themes: Awareness of student learning, Changes in the organization of teaching, but not regarding individualized content and Strengthened teacher role, but modest professional growth. The themes indicated that the teachers had become aware of their students’ learning, had employed teaching based on informed decisions, and showed initial professional growth.However, the assessment details and the recommendations allowed for more adjustments than was evident in the teachers’ narratives. The results point to the relative difficulty of targeting individual needs in the general classroom education, and to the challenges of changing teaching practices.

A Systems Approach to International Education: How Administrative Data is Being used to Track the Progress of USAID's Education Strategy

While rigorous evaluation of donor funded education interventions are quite common in developing countries, the data collected and analyzed to assess the interventions are not always published. Even when published, the data often lack codebooks and technical documentation, guidance on how to correctly use the data, and most frustratingly only contain meta-information pertaining to the design of the intervention and evaluation in free text in PDF reports. Such practices reduce the overall quality of the data and meta-data making it very difficult to be used for secondary analysis to answer policy relevant questions. To address such challenges, this paper describes the integrated data system developed to securely receive, review, harmonize, ingest, curate, and publish standardized public use Early Grade Reading Assessment data and meta-data to track the progress of the 2011-2015 USAID Education Strategy. The paper discusses the underlying methodological framework for systematically reviewing the data for quality and completeness as well as the approach to profile the design of the intervention and evaluation to assess its rigor using the system. Lastly, the paper illustrates how the meta-data created through the systematic review process combined with the data collected and processed through the system can be used for developing dynamic data products that reduce the technical barriers for such data to be used by a wide range of stakeholders to inform critical policy decisions.

Student-Teacher Racial Match and Its Association With Black Student Achievement: An Exploration Using Multilevel Structural Equation Modeling

This study examines student-teacher “racial match” for its association with Black student achievement. Multilevel structural equation modeling was used to analyze 2013 National Assessment for Educational Progress (NAEP) Grade 4 Reading Assessment data to examine interactions of teacher race and student race in their associations with student achievement (N = 165,410 students, 23,710 teachers). Effects on Level 1 random slope parameters suggest that a match of Black male students with Black teachers is associated with higher reading scores for this group, as is a match of Black female students with a Black or Hispanic teacher. Level 2 effects suggest that greater classroom composition of Black males is associated with lower reading achievement. Moderation by gender and implications for educational policy are discussed.

An Exploration of Elementary Teachers’ Views of Informal Reading Inventories in Dual Language Bilingual Programs

ABSTRACTThis study examines how elementary teachers (grades three through five) in dual-language, bilingual programs (Spanish/English) view informal reading inventories (IRIs) to support their students’ reading growth. The research, conducted in an urban district in the Northeastern United States, draws on interviews with 20 teachers in these programs. One significant finding is that although teachers in the sample collected IRIs in the two languages of instruction, they did not examine English and Spanish reading assessment data side by side in order to construct a unified portrait of their students as bilingual readers. This study highlights the finding that IRIs are currently viewed as monolingual assessments rather than as a lens into students’ biliteracy, thus bypassing a powerful way to assist teachers in making instructional decisions in support of students’ bilingual reading development.

Interpreting Reading Assessment Data

This article is designed to help teachers interpret reading assessment data from DIBELS beyond individual subtests to better support their students’ needs. While it is important to understand the individual subtest measures, it is more vital to understand how each fits into the larger picture of reading development. The underlying construct of each DIBELS subtest is unpacked, followed by a demonstration of how teachers can (a) interpret results and (b) design high-quality interventions around the reading constructs, rather than teaching to individual subtests. Two classroom examples are presented that were designed to illustrate realigned interpretation of assessment results.

Compensatory Reading among ESL Learners: A Reading Strategy Heuristic

This paper aims to gain an insight to the relationship of two different concepts about reading comprehension, namely, the linear model of comprehension and the interactive compensatory theory. Drawing on both the above concepts, a heuristic was constructed about three different reading strategies determined by the specific ways the literal, reorganisation and inferential skills of comprehension interact. The concepts of apt and smart reading strategies were introduced, which refer to two possible ways of compensatory reading. Applying the reading strategy heuristic to the secondary analysis of a large reading assessment data from Malaysian secondary school 3567 ESL learners, compensatory reading was found to be used by a significant group of those students who struggle with poor reading skills. Furthermore, one of the compensatory strategies, namely, smart reading, was found to be positively correlated with the learners' motivation to read and their belief about own reading competence, the proportion of positive answers among smart readers being 3.5% and 3.9% higher than among the mainstream readers, respectively. The findings suggest that the language talent of apt and smart readers (18.2% of the current sample to be discovered and cultivated in the L2 classroom, especially among low-achiever learners who would most benefit from the recognition of compensation as a legitimate skill of reading comprehension.

A model of online reading engagement: Linking engagement, navigation, and performance in digital reading

A model of online reading engagement is outlined. This model proposes that online reading engagement predicts dedication in digital reading. Dedication in digital reading according to the model is reflected in task-adaptive navigation, and task-adaptive navigation predicts digital reading performance over and above print reading skill. Information engagement is assumed to positively predict task-adaptive navigation, while social engagement is assumed to negatively predict task-adaptive navigation. These hypotheses were tested using OECD PISA 2009 Digital Reading Assessment data from 17 countries and economies (N=29,395). Individual task responses served as the primary unit of analysis. Linear mixed models were used to predict navigation behavior from the interaction of information and social online reading engagement with navigation demands. High information engagement was associated with more task-adaptive navigation behavior, as shown by significant positive interactions between information engagement and tasks’ navigation demands. In contrast, high social engagement was associated with less adaptive navigation behavior, as shown by negative interactions between social engagement and navigation demands. Generalized linear mixed models were used to predict task performance by the interaction of navigation demands and navigation behavior. Adaptive navigation behavior predicted digital reading task performance, as shown by significant interactions between navigation behavior and navigation demands. These results are in support of the proposed model of online reading engagement.

Oral reading fluency and comprehension in Kenya: reading acquisition in a multilingual environment

Reading research has shown that variable relationships exist between measures of oral reading fluency and reading comprehension, depending on whether the language of the text is the reader's first language or an additional language. This paper explores this phenomenon, using reading assessment data for 2,000 Kenyan children in two or three languages: English, Kiswahili and one of two mother tongues, Dholuo or Gikuyu. The assessment data allowed us to compare reading and comprehension rates across languages. The data indicated that many children could read English words more easily than words in Kiswahili or their mother tongue; nevertheless, their reading comprehension was significantly lower in English than in Kiswahili, Dholuo or Gikuyu. The paper concludes that emphasising English reading fluency is an inefficient route to gaining reading comprehension skills because pupils are actually attaining minimal oral reading fluency in English and only modest comprehension skills in their own languages. The evidence also demonstrates that Kenya's national language policy of mother tongue as a medium of instruction in the early primary grades is consistently ignored in practice.

A Bifactor Multidimensional Item Response Theory Model for Differential Item Functioning Analysis on Testlet-Based Items

A differential item functioning (DIF) detection method for testlet-based data was proposed and evaluated in this study. The proposed DIF model is an extension of a bifactor multidimensional item response theory (MIRT) model for testlets. Unlike traditional item response theory (IRT) DIF models, the proposed model takes testlet effects into account, thus estimating DIF magnitude appropriately when a test is composed of testlets. A fully Bayesian estimation method was adopted for parameter estimation. The recovery of parameters was evaluated for the proposed DIF model. Simulation results revealed that the proposed bifactor MIRT DIF model produced better estimates of DIF magnitude and higher DIF detection rates than the traditional IRT DIF model for all simulation conditions. A real data analysis was also conducted by applying the proposed DIF model to a statewide reading assessment data set.

An Analysis of Consistency Between Team Decisions and Reading Assessment Data Within an RTI Model

Data-based decision making by teams is central to implementation of response to intervention (RTI) models. Few studies have examined the actual decision-making process within RTI systems of service delivery. The purpose of this study was to examine the tier assignment decisions for students across grade-level teams in three K–5 elementary schools implementing identical RTI models for reading. Decisions of grade-level teams primarily composed of teachers were compared against the recommendation made by the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) universal screening measure during fall and winter assessments across a 2-year period. Results showed teams had high overall levels of agreement with the DIBELS recommendations, especially after Grade 1 and during the winter of each year. When teams initially disagreed with the DIBELS recommendations, increased agreement in the use of data-based decisions was present over time. Implications for the data decision-making process within an RTI model are discussed.