Curriculum-based Measurement Research Articles

Systematic Progress Monitoring of Individualized Education Program Goals in Mathematics

The review of Endrew F. v. Douglas County School District case revealed that an Individualized Education Program (IEP) requires inclusion of specific measurable goals to ensure educational benefit. The purpose of this article is to provide a systematic approach to progress monitoring IEP goals in mathematics, which monitors whether a student is benefiting from the mathematics instruction being provided. Using curriculum-based measures (CBM) is discussed as a feasible tool for monitoring the progress of students and creating specific measurable IEP goals. The article includes step-by-step strategies and guidelines for teachers to collect CBM data, create strategic IEP goals, and evaluate student progress.

Explaining reading variance by student subgroup: should we move beyond oral reading fluency?

BackgroundStudies have demonstrated strong predictive and concurrent validity of curriculum‐based measures (CBMs) with high‐stakes reading tests, mainly using oral reading fluency (ORF) to predict outcomes. Some studies have found that CBMs' predictive accuracy might vary across demographic subgroups of students. In this study, we investigate whether additional CBMs of reading comprehension account for variance beyond ORF and whether these reading variables predict reading comprehension differently for demographic subgroups.MethodsCurriculum‐based measures were administered to 9,967 students at the end of Grade 3 in the United States, concurrent with a high‐stakes standardised state reading comprehension achievement test (RCAT). Hierarchical regression models with RCAT as the outcome were used to test (a) whether CBMs of reading comprehension accounted for significant variance beyond ORF and (b) if the reading variables differentially predicted RCAT for different subgroups of students.ResultsOral reading fluency explained significant variation in RCAT, and the addition of comprehension CBMs accounted for small amounts of unique variance. The contribution of ORF in explaining variation in reading comprehension varied by demographic subgroups.ConclusionsThe contributions of comprehension CBMs in predicting high‐stakes standardised reading comprehension tests were not meaningful, only accounting for small amounts of unique variance. Additionally, the contribution of ORF in explaining variation in the high‐stakes standardised reading comprehension test varied by student demographics, likely due to complex reasons specific to those individual factors.HighlightsWhat is already known about this topic CBMs of ORF account for significant variance in reading comprehension achievement on high‐stakes standardised reading tests. Failure to include some measure of comprehension in CBMs could be problematic because fluency does not ensure comprehension. When comprehension CBMs have been included, findings have been mixed. Some studies have pointed to differences in how well ORF predicts reading comprehension for various subgroups of students. What this paper adds We examine the contribution of comprehension CBMs over and above ORF in predicting student performance on a high‐stakes reading comprehension test using a large, diverse sample in the United States. We replicate others' findings on the contribution of ORF in predicting variance in students' high‐stakes reading comprehension test scores. We find that the contributions of ORF differ by student subgroups, specifically socioeconomic status, limited English proficiency, and race/ethnicity. Implications for theory, policy or practice Conducting and analysing assessments takes a significant amount of teachers' time. Our findings suggest a lack of utility in using additional CBMs beyond a measure of ORF. Given the consistency of our findings with prior studies, teachers should use ORF as a screening measure and then shift to diagnostic measures to better understand students' performance. Because ORF differentially predicts performance on a high‐stakes reading comprehension test based on subgroups of students, alternative assessments should be considered for particular groups of students.

Observing Two Reading Intervention Programs for Students with Dyslexia

ABSTRACT This exploratory study examined the nature of instruction provided in two reading intervention programs designed for elementary-grade students with dyslexia (The Multisensory Teaching Approach and Reading RULES!). In addition to documenting the proportion of time dedicated to particular content components (i.e., letter-name knowledge, phonological awareness, phonics/decoding, encoding, sight word recognition, passage reading/fluency, comprehension, vocabulary), the research team also documented the degree to which instruction (a) included teacher explanation/modeling, guided practice, group practice, and individual practice, (b) addressed particular types of decoding/encoding procedures, including multi-sensory (defined as tactile/kinesthetic) procedures and use of orthographic rules, and (c) made use of given instructional materials. The team conducted 12, ~45-minute observations. There were many similarities between the two programs (e.g., both were explicit, systematic, and sequential; both included curriculum-based measures of student learning that informed instruction; both dedicated approximately equal amounts of time to decoding and encoding instruction). However, our observations indicated a statistically significant difference between the programs (p < .002) in proportion of time dedicated to letter-name knowledge, text reading, and comprehension instruction. The programs also differed in their emphasis on articulating/applying orthographic rules and on particular procedures for decoding and encoding words.

Reading Curriculum-Based Measures for Universal Screening in Monolingual Spanish First Graders

ABSTRACT Research findings: Universal screening is the first stage in identifying students at risk for reading difficulties within a Response to Intervention model. However, there is a lack of validated screening tools for assessing reading abilities in first-grade students from Spain. This pilot study examines the technical adequacy, classification accuracy, and best predictors within a set of curriculum-based measures (CBMs) in Spanish. A sample of 178 first graders from urban and peripheral areas of Santa Cruz de Tenerife (the Canary Islands, Spain) was assessed in the fall, winter, and spring. Receiver operating characteristic curves and logistic regression models were conducted to evaluate the predictive validity of each CBM and their composite score. In addition, students’ learning growth on each CBM was analyzed using hierarchical linear models. Although results suggested that most of the CBMs had adequate reliability and validity throughout first grade and were able to detect students’ growth, some measures showed low reliability and validity coefficients. Practice or police: Some of the studied CBMs could potentially be used as universal screening to early detect reading difficulties in this population. A two-stage gated screening procedure is proposed for future research and practical implementation using oral reading fluency in the first step.

School readiness screening and educational achievement at 9-10 years of age.

To determine whether a multi-domain school readiness screening, the Before School Check (B4SC), identifies children at risk of low educational achievement and to compare the educational outcomes between those referred for intervention and those with B4SC concerns who were not referred. In this longitudinal cohort study of children born at risk of neonatal hypoglycaemia (N331), the B4SC was performed at 4.5 years of age and a standardised curriculum-based measure of educational achievement was completed at 9-10 years of age. Outcomes of school readiness screening were categorised into 'school readiness concern' or 'no school readiness concern' while 'below standard' and 'well below standard' ratings of educational achievement were combined into a single category of 'low educational achievement'. Overall, 52% of children had ≥1 school readiness concerns at the B4SC, predominantly about behaviour (46%). Having ≥1 school readiness concern was associated with a nearly twofold increase in the likelihood of low academic achievement (OR1.85, 95% CI 1.14, 3.02), which was apparent only for behaviour concerns. Of the 128 children with behaviour concerns, only 10 (8%) were referred for further interventions. There was a statistically non-significant increase in the rates of low academic achievement among those referred than those non-referred (60% vs. 47%). Identification of behaviour concerns during B4SC is associated with a moderate increase in the likelihood of low academic achievement at 9-10 years. Further, research is needed to determine how academic achievement can be improved in children with behaviour concerns at school entry.

Specific Learning Disability Identification Practices in the USA: a Survey of Special Education Administrators

Specific learning disorder (SLD) identification is arguably one of the most significant controversies in school psychology. One component of this issue is that guidelines and practices vary substantially across professionals and states, which leads to inconsistent identification rates and outcomes for students. To assess the extent of this problem at the district level, we surveyed special education administrators from 725 US school districts regarding their districts’ current specific learning disability (SLD) identification practices in the USA. Significant differences in SLD identification methods were reported across districts. Significant differences in SLD identification methods were also found across regions and between districts that used different educators to collect academic achievement test data. Curriculum-based measures were also used more frequently in districts that identify SLD using response to intervention. Implications of these findings for policy and for practice are considered.

Accounting for Levels of Language in Narrative and Expository Writing

AbstractWe conducted a skills analysis on written expression curriculum-based measurement (WE-CBM) tasks completed by 117 second-grade students. As part of the skills analysis, we scored two WE-CBM...

Developing and Investigating the Promise of Early Measurement Screeners

This study investigated the technical characteristics of four early measurement curriculum-based measures (EM-CBMs) designed to assess concepts related to linear measurement and iteration. The sample consisted of 221 first grade students. Data were collected at two time points approximately 10 weeks apart. Reliability and concurrent and predictive validity correlations were in the low to moderate range. We discuss study results related to screening for risk status including limitations to the current work and future directions for research. Impact Statement The importance of measurement in mathematics development is garnering increased attention. Exploring measures to screen students for risk status is critical to enable schools to allocate resources to students in need of intervention services. The findings in this manuscript represent a first exploratory attempt to develop screening measures in the area of measurement.

FABLe: A Mobile Application to Assess and Build Foundational Literacy Skills for All Children, Including Children with Disabilities in India

ABSTRACT Fluency Assessment and Benchmarking for Literacy in Education (FABLe) is the first mobile application being developed for the assessment of foundational literacy skills in India. FABLe passages are currently available in English for Grades 2, 3 and 4, with plans to expand to other Indian languages. FABLe is designed to administer curriculum-based measures for reading (R-CBMs) that have been validated in the Indian context. This paper describes the preliminary benchmarking procedures being undertaken for the Indian student population and how FABLe screens for students at-risk for reading difficulties. Student performance data based on initial FABLe results are compared with international benchmarks and teacher perception data. Overall, FABLe scores are lower than international benchmarks. FABLe was able to accurately identify students needing additional support, with the majority of students identified at-risk falling below the 19th percentile in FABLe.

Graph Construction and Visual Analysis: A Comparison of Curriculum-based Measurement Vendors

Curriculum-based measurement (CBM) represents a critical strategy for data-based decisionmaking within educational settings. Visual analysis is frequently used to analyze CBM data; thus, CBM vendors often automatically generate graphs based on student data to facilitate analysis. Differences in graph formatting are apparent across CBM vendors, and currently, the extent to which these differences may influence rater interpretation of data is unknown. As such, the current study sought to evaluate whether there were differences in rater evaluation of presence and magnitude of an intervention effect when identical data were plotted on graphs of four CBM vendors: AIMSweb, FastBridge Learning, easyCBM, and DIBELS Next. Results of the study indicated that visual analysts rated. Results of the study found that probability of identifying the presence of a treatment effect was similar across vendors. However, differences emerged with respect to ratings of magnitude of effect, with raters indicating the greatest magnitude of intervention effect for easyCBM and DIBELS Next graphs. As data graphs generated by CBM vendors may be utilized by school personnel to make decisions regarding the continuation, discontinuation, or alteration of intervention procedures, the results of the study indicate the need for increased consideration in the manner in which data are presented visually such that accurate decisions are made regarding student progress and response to intervention.

Bringing Data-Based Individualization to Scale: A Call for the Next-Generation Technology of Teacher Supports

The purpose of this narrative synthesis of the curriculum-based measure (CBM) instructional utility literature is to deepen insight into the supports required to enrich teachers’ instructional decision-making within curriculum-based measure–data-based individualization (CBM-DBI) in ways that enhance the learning outcomes of students with intensive intervention needs, including students with learning disabilities. We begin by summarizing a recent meta-analysis of CBM-DBI studies focused on academic outcomes. We then reconsider studies from that meta-analysis to further explore the supports required to enrich teachers’ instructional decision-making within CBM-DBI and improve student learning. We next draw conclusions and propose a renewed program of instructional utility CBM-DBI research for capitalizing on technology’s potential to enhance productive instructional decision-making for students who require intensive intervention, fulfill DBI’s potential, and bring CBM-DBI to scale.

Validity of automated text evaluation tools for written-expression curriculum-based measurement: a comparison study

Existing approaches to measuring writing performance are insufficient in terms of both technical adequacy as well as feasibility for use as a screening measure. This study examined the validity and diagnostic accuracy of several approaches to automated text evaluation as well as written expression curriculum-based measurement (WE-CBM) to determine whether an automated approach improves technical adequacy. A sample of 140 fourth grade students generated writing samples that were then scored using traditional and automated approaches and examined in relation to the statewide measure of writing performance. Results indicated that the validity and diagnostic accuracy for the best performing WE-CBM metric, correct minus incorrect word sequences, and the automated approaches to scoring were comparable, with automated approaches offering potentially improved feasibility for use in screening. Averaging scores across three time points was necessary, however, in order to achieve improved validity and adequate levels of diagnostic accuracy across the scoring approaches. Limitations, implications, and directions for future research regarding the use of automated scoring approaches for screening are discussed.

External evaluations for dyslexia: do the data support parent concerns?

A grassroots movement of parents who fear that their children's reading struggles are going unrecognized at school has led to dyslexia laws in all but three states in the U.S. The current study was undertaken to provide data relevant to this topic by characterizing the reading profiles of 71 children referred for testing at a center specializing in the assessment of reading disabilities. These children were receiving instruction and intervention in reading across the tiers of instructional support in general and special education within their schools. On average, the children demonstrated equivalent deficits in print literacy skills on norm-referenced assessments regardless of the intensity of their reading support, and the majority of children who were only receiving tier 1 instruction exhibited characteristics of dyslexia. Moreover, 69% of children only receiving tier 1 instruction, and all remaining children, performed below benchmark expectations on a curriculum-based measure of oral reading fluency. While these data are not an evaluation of the implementation of the state's dyslexia laws or the statewide implementation of RTI, they provide data characterizing the real struggles and lack of identification of children whose parents seek an external evaluation of their children's reading skills. However, they are set in the context of a state in which 66% of public-school children cannot read proficiently by the end of the third grade. The reading struggles highlighted in this clinic referral sample are unexceptional in the larger state context.

A Systematic Review of CBM Professional Development Materials: Are Teachers Receiving Sufficient Instruction in Data-Based Decision-Making?

The purpose of this study was to examine the amount of attention devoted to data-based decision-making in Curriculum-Based Measurement (CBM) professional development materials. Sixty-nine CBM instructional sources were reviewed, including 45 presentations, 22 manuals, and two books. The content of the presentations and manuals/books was coded into one of four categories: (a) general CBM information, (b) conducting CBM, (c) data-based decision-making, and (d) other. Results revealed that only a small proportion of information in the CBM instructional materials was devoted to data-based decision-making (12% for presentations and 14% for manuals/books), and that this proportion was significantly smaller than (a) that devoted to other instructional topics, (b) that expected were information to be equally distributed across major instructional topics, and (c) that recommended by experienced CBM trainers. Results suggest a need for increased attention to data-based decision-making in CBM professional development.

Identification of struggling readers or at risk of reading difficulties with one-minute fluency measures

BackgroundTo identify readers who are struggling or at risk of reading difficulties, reference standards in oral reading fluency (ORF) are used to conduct an assessment that is based on a widely reported method known as curriculum-based measurement (CBM), which itself is based on 1-min fluency measures. The purpose of this study was to evaluate students’ ORF (with a 1-min fluency measure) to characterize their fluency and to determine references of appropriate development in reading at the 50th percentile.MethodFor this study, a database of readings made available by the Learning Studies Research Laboratory was used. This database consisted of 365 readings by elementary-school students from the third to fifth grades in two cities in the interior of the state of São Paulo from two different public school systems that use the same teaching methodology. The data consisted of digital audio recordings of the passage “The Umbrella” (text suitable for schooling levels) of the Protocol for Assessment of Reading Comprehension procedure. For this procedure, three steps were performed: step 1—listening to the 365 readings and assessing the scores for the number of words read correctly per minute; step 2—the calculation of the mean and percentiles for each grade; and step 3—the adaptation of the reference table to indicate students eligible to receive reading fluency intervention.ResultsThird-year students who correctly read 86 or more words per minute, fourth-year students who correctly read 104 or more words per minute, and fifth-year students who correctly read 117 or more words per minute were considered students who had made adequate progress in reading.ConclusionIt was possible to classify students based on the 1-min fluency measures, with reference intervals of words read correctly per minute per school year (for the third, fourth, and fifth years) for those who were making adequate progress in reading and reference intervals for those who were considered readers who were struggling or at risk of reading difficulties.

Academic Achievement Test Use and Assessment Practices: A National Survey of Special Education Administrators

This study examined current academic achievement test use and assessment practices in the United States. Special education administrators from 725 US school districts were surveyed using an online platform to determine current trends in the use of norm-referenced academic achievement tests (NRTs) and curriculum-based measures (CBMs). Administrators reported that school psychologists most commonly administered the NRTs used to help determine special education eligibility, although special education teachers also fill this role frequently, particularly in Western states. General and special educators are the most likely staff to administer CBMs. A detailed accounting of academic achievement assessment practices is discussed including which tests are used and the differences in assessment roles by region and district locale. Implications of these findings for the training of educators and for research are considered.

Does CBM maze assess reading comprehension in 8-9-year olds at-risk for dyslexia?

Recent research has reported the word-level, code-related focus of curriculum-based measures (CBM) of reading comprehension such as Maze (Muijselaar et al., 2017) with typically developing readers, but research has yet to examine whether this finding also applies to children at-risk for dyslexia. We administered a collection of cognitive, linguistic, CBM, and norm-referenced measures to children whose word reading and decoding fluency fell below the 25th percentile and were, therefore, considered at-risk readers. We found that language comprehension contributed additional variance beyond decoding (fluency and accuracy measures) to reading comprehension as assessed by the WIAT-III, but that decoding explained the most variance in children's performance on the CBM Maze task (vis à vis the simple view of reading). The findings have practical implications to the use of CBM Maze as a formative assessment with children at-risk for dyslexia and elucidate the need for additional or alternative assessments to capture the reading comprehension construct.

Accuracy of Automated Written Expression Curriculum-Based Measurement Scoring

We examined the extent to which automated written expression curriculum-based measurement (aWE-CBM) can be accurately used to computer score student writing samples for screening and progress monitoring. Students ( n = 174) with learning difficulties in Grades 1 to 12 who received 1:1 academic tutoring through a community-based organization completed narrative writing samples in the fall and spring across two academic years. The samples were evaluated using four automated and hand-calculated WE-CBM scoring metrics. Results indicated automated and hand-calculated scores were highly correlated at all four timepoints for counts of total words written ( rs = 1.00), words spelled correctly ( rs = .99–1.00), correct word sequences (CWS; rs = .96–.97), and correct minus incorrect word sequences (CIWS; rs = .86–.92). For CWS and CIWS, however, automated scores systematically overestimated hand-calculated scores, with an unacceptable amount of error for CIWS for some types of decisions. These findings provide preliminary evidence that aWE-CBM can be used to efficiently score narrative writing samples, potentially improving the feasibility of implementing multi-tiered systems of support in which the written expression skills of large numbers of students are screened and monitored.

CBM language measures as indicators of foreign-language learning: Technical adequacy of scores for secondary-school students

Students with and without learning disabilities often struggle to learn a foreign language (FL). Teachers could benefit from a measure designed to screen and identify students at risk for FL learning difficulties. In this study, we examined the reliability and validity of scores from four curriculum-based measures (CBM) as potential indicators of English FL learning: reading aloud, maze selection, and English-to-Dutch and Dutch-to-English word translation. Participants were 133 Dutch students in Grade 8. Criterion variables were English course grades and scores on a standardized achievement test (Cito-VAS). Alternate-form reliability ranged from r = .77 to .87. Correlations between CBM and criterion measure scores ranged from r = -.04 to .65. Scores from maze selection and reading aloud alone predicted English-language proficiency better than a combination of scores from the four measures, explaining 29.7% and 23.6% of the variance, respectively. Implications for the use of CBM for FL screening and progress-monitoring are discussed.

Technical Features of Slopes for Curriculum-Based Measures of Secondary Writing

Compared to other content areas, there is a dearth of research examining curriculum-based measurement of writing (CBM-W). This study conducted a conceptual replication examining the reliability, stability, and sensitivity to growth of slopes produced from CBM-W. Eighty-nine (N = 89) eighth-grade students responded to one CBM-W probe weekly for 11 weeks. Probes were scored using four different CBM-W scoring procedures: words written, words spelled correctly, correct word sequences, and correct minus incorrect word sequences. We found limited evidence for reliability and stability of these slopes. Further, the slopes were not sensitive to growth, as defined as a slope significantly greater than zero. We recommend caution when using CBM-W to progress monitor eighth-grade students. Future research should examine the same technical features with a sample of students who are low-performing or at-risk for writing failure.