Progress Monitoring Data Research Articles

Comprehensive Assessment and Evaluation of Students with Learning Disabilities: A Paper Prepared by the National Joint Committee on Learning Disabilities

The National Joint Committee on Learning Disabilities (NJCLD)1 strongly supports comprehensive assessment and evaluation of students with learning disabilities by a multidisciplinary team for the identification and diagnosis of students with learning disabilities. Comprehensive assessment of individual students requires the use of multiple data sources. These sources may include standardized tests, informal measures, observations, student self-reports, parent reports, and progress monitoring data from response-to-intervention (RTI) approaches (NJCLD, 2005). Reliance on any single criterion for assessment or evaluation is not comprehensive, nor is a group assessment, such as universal screening or statewide academic assessment tests, sufficient for comprehensive assessment or evaluation. This paper is intended to inform administrators, educators, parents, and others concerned about the effective identification and education of students with learning disabilities about the components, processes, and participants necessary for comprehensive assessment and evaluation, as well as optimal practices that should further enhance the education of students with learning disabilities. The NJCLD has long recognized that inappropriate assessment and evaluation practices may result in questionable incidence rates for learning disabilities (NJCLD, 2001a). Similarly, the NJCLD (2001a, 2001b) has provided a solid foundation for addressing the issues of assessment, evaluation, identification, and eligibility of students with learning disabilities.

Defensible Progress Monitoring Data for Medium- and High-Stakes Decisions

Within a response to intervention model, educators increasingly use progress monitoring (PM) to support medium- to high-stakes decisions for individual students. For PM to serve these more demanding decisions requires more careful consideration of measurement error. That error should be calculated within a fixed linear regression model rather than a classical test theory model, which has been more common. Seven practical skills are described for improving the use of PM data for medium- to high-stakes decisions: (a) estimating a static performance level in PM, (b) fitting a level of confidence to an educational decision, (c) expressing an estimated score ( Yhat) with its measurement error, (d) judging reliable improvement from one time to a later time, (e) properly using slope versus trendedness, (f) expressing “rate of improvement” (slope) with error, and (g) controlling autocorrelation. An example data set and PM graphs are used to illustrate each.

Reliability of Decision-Making Frameworks for Response to Intervention for Reading

The current study examines the consistency of two response-to-intervention (RTI) decision-making models. Weekly progress monitoring data for 30 students participating in a Tier II intervention were collected for 30 weeks. The data were examined by comparing them to an aimline with a yearly goal and by computing a dual discrepancy (DD) using numerical slope and postintervention reading level. A κ coefficient was computed between the two models regarding student growth using three categorical decisions of making sufficient progress, not making sufficient progress, and exceeding expected progress. The resulting coefficient of .29 represents fair agreement. Data were also assigned to one of two sets in an every-other order, with resulting new aimlines and new DDs. The consistency between the two sets of data resulted in coefficients of κ = .29 for aimline and κ = .58 for DD. Finally, the standard error of measurement for the baseline data point affected the decision made (κ = .13) for aimline but not for DD (κ = .93). Implications for research, practice, and special education decision making are included.

Using DIBELS: A Survey of Purposes and Practices

Using a mail survey and face-to-face interviews, this study explored educators' use of and perceptions about DIBELS, a widely used reading assessment and intervention instrument. Analysis included tabulations and a conceptual analysis of extended responses. Most frequent uses for DIBELS included identification of at-risk students, intervention development, and progress monitoring. However, respondents were less clear about how progress monitoring data informed their instruction and whether DIBELS was aligned with state-mandated testing. Time issues were perceived as both positive (quick to administer) and negative (administered individually and repeatedly) factors. Mirroring the literature, respondents expressed both strong positive and negative views about DIBELS.

Progress Monitoring as Essential Practice within Response to Intervention

Response to intervention (RTI) encompasses a process for evaluating whether students react to evidence-based instruction as expected. Typically considered a multitiered, prevention-intervention system, successive levels of instructional support are provided when a student's response to the academic program is sufficiendy poor, particularly as compared to his or her peers' responses. In 2004, the reauthorization of the Individuals with Disabilities Education Improvement Act (IDEIA; P. L. 108-446) allowed for a student's response to research-based intervention to be part of the process for identifying students with specific learning disabilities (SLD). Rather than requiring the traditional aptitude-achievement discrepancy approach to identification, which sometimes necessitated years of poor academic achievement before a student might qualify for special education services (i.e., a wait-to-fail model), IDEIA allows for continued poor response to validated instruction as a means for documenting that a student's disability may require specialized services to produce appropriate learning outcomes. In other words, if a student continues to perform poorly despite the implementation of scientifically validated instruction, then inadequate instruction is eliminated as a cause for the student's insufficient learning. Instead, the student's lack of appropriate response to otherwise generally effective instruction is, in part, evidence for the presence of a disability. Within a multitiered RTI system, students are likely to receive help at earlier stages in their learning with perhaps some disabilities even being prevented from developing or their overall impact lessened. This preventive aspect has prompted many schools to adopt an RTI framework as a means for reforming their educational practices (e.g., see Tilly, 2006). As potential LD identification is embedded within this preventive framework, however, schools are faced with questions about how best to operationalize this process. Professional concerns expressed about using RTI as a part of SLD identification typically revolve around issues related to instruction and assessment. Although sound instruction is paramount to successful implementation of RTI, assessment data should drive decision making. Therefore, we argue that progress monitoring comprises one of the most critical features of successful RTI implementation. Careful progress monitoring aids teachers and student support teams in making instructional decisions throughout all levels of the RTI system and provides data to corroborate SLD identification. We propose a sample framework for RTI implementation within the context of elementary-level instruction in reading that is based on available research. We identify critical aspects of both instruction and assessment that must be addressed by schools as they operationalize RTI. Also within this multitiered framework, we describe how progress monitoring data can be used to provide salient information regarding the presence of a learning disability. Last, we discuss additional challenges rural schools may face when carrying out RTL Instruction and Assessment Within an RTI Framework Tier 1: Primary Prevention Instruction. Most models of RTI involve multiple tiers, or levels, of instruction aimed at preventing (a) inadequate instruction from being implemented over sustained periods of time and (b) disabilities from developing or becoming more severe. The first tier of such a framework occurs as general education instruction. Because data used from an RTI process for potentially identifying students with SLD must show lack of adequate response to scientifically validated instruction, Tier 1 must necessarily involve implementation of instructional practices that have been tested empirically. Schools must be able to defend that the core programs and instructional procedures used by their teachers have been generally effective in promoting student achievement or that specific instructional components within diese programs have empirical validation for improved achievement. …

When Teachers Work to Use Progress Monitoring Data to Inform Literacy Instruction

The phenomenon of using progress monitoring data to inform literacy instruction was explored in the context of four schools during their 2nd year of Reading First implementation. Open-ended interviews and surveys from 10 teachers, purposefully selected for maximum variation in effectiveness and skills and resistance to using data to inform instruction, were primary data sources. Across schools, teachers reported varying levels of success with using data to inform instruction. A grounded theory model of how K-1 teachers worked with reading coaches to try to use data to inform instruction was developed through axial coding. The coach's role was to help teachers access and interpret data and make informed links to reading curricula. Perceived barriers to the practice of using the data included lack of time and classroom management difficulties.

Evaluating curriculum-based measurement from a behavioral assessment perspective.

Curriculum-based measurement for reading (CBM-R) procedures were developed in the late 1970's and early 1980's as a set of standardized assessment tools for gauging students' academic in reading. It was developed in order to provide teachers with an efficient, easily understood measurement system yielding relevant data about students' level of performance, as well as their reading growth over time (Deno, 1985). Educators and researchers have noted that an important characteristic of is its ability to measure both inter-individual differences in groups of students as well as intra-individual change within specific students (Fuchs & Fuchs, 1998; Fuchs, Fuchs, & Speece, 2002). While data were initially used exclusively to guide low-stakes educational decisions (Deno, 1985; Deno, 1986; Deno, Marston, & Tindal, 1985; Deno & Shinn, 1989), data are now being used for making high-stakes decisions (i.e., special education eligibility) within Response to Intervention (RtI) models. Several features distinguish procedures from other standardized measures used to assess students' reading. First, the assessment materials are relatively cheap and it requires little time to administer probes to students. Second, is meant to be a measurement of students' global reading performance, which allows for practitioners to evaluate how students' are progressing toward towards long-term goals (Deno, Fuchs, Marston, & Shin, 2001; Fuchs, Fuchs, Hamlett, Walz, & Germann, 1993). Finally, as described by Deno et al. CBM-R departs from conventiona l psychometric applications by integrating the concepts of standardized measurement and traditional reliability and validity with features from behavioral and observational assessment methodology: repeated sampling, fixed time recording, graphic display of times-series data, and qualitative descriptions of performance (Deno et al., 2001, p. 508) These characteristics makes ideal for use within an RtI model, as an instrument used to both identify students at-risk for academic problems and to evaluate individual students' response to instruction. Within the Fuchs and Fuchs (1998) dual discrepancy RtI model, data are first used to identify students who are at-risk for academic problems based upon a comparison of their to that of their normative group (i.e., nomothetic context). Such decisions are relatively low-stakes decisions, given that the identification of a student at-risk simply results in the student being provided with supplemental instruction. data modeling individual student's response to the supplemental instruction is then used as a primary source of data for making high-stakes decisions. Students' response to supplemental instruction is generally evaluated using progress monitoring procedures, which entails the frequent administration of probes and plotting of collected data in time-series fashion. Progress monitoring data are evaluated by comparing the plotted data to a pre-established goal line or an estimate of weekly growth calculated using ordinary least square regression techniques. Regardless of the method of comparison, these evaluations of data are within an idiographic context in which an individual's data are compared to his/her previous performance(s). Based upon the evaluation of data, one of the following high-stakes decisions is made: (a) the intervention was successful, therefore the student is not eligible for special education and the intervention should be terminated, (b) the intervention was not adequate and a more intense intervention is needed, or (c) supplemental interventions of varying levels of intensity have not been successful and the student is eligible for special education. The importance of the psychometric adequacy of is especially salient now that data are being used as a primary source for making high-stakes educational decisions (Ardoin & Christ, in press; Christ & Ardoin, 2007). …

The effectiveness of classroom-based instructional assessments for progress monitoring purposes in Texas Reading First schools

The purpose of this article was twofold. First, the assessment materials from four core reading programs were informally reviewed for their adherence to Texas Reading First progress monitoring guidelines for kindergarten and the beginning of Grade 1. Conclusions support existing concerns regarding the variability in the quality and quantity of assessment materials within and across programs as well as concerns about whether the materials lend themselves to progress monitoring purposes at 2-week intervals for students who are struggling readers. Second, recommendations are made for the development of progress monitoring tools that are concise, easy for teachers to use, and that provide clear guidelines regarding the use of progress monitoring data to drive instruction. Accordingly the recent development of a progress monitoring tool that parallels the Texas Primary Reading Inventory (TPRI), which is an assessment tool used in approximately 95% of Texas Reading First schools was discussed.

Selecting at-risk readers in first grade for early intervention: A two-year longitudinal study of decision rules and procedures.

Response to intervention (RTI) models for identifying learning disabilities rely on the accurate identification of children who, without Tier 2 tutoring, would develop reading disability (RD). This study examined 2 questions concerning the use of 1st-grade data to predict future RD: (a) Does adding initial word identification fluency (WIF) and 5 weeks of WIF progress-monitoring data (WIF-Level and WIF-Slope) to a typical 1st-grade prediction battery improve RD prediction? and (b) Can classification tree analysis improve the prediction accuracy compared to logistic regression? Four classification models based on 206 1st-grade children followed through the end of 2nd grade were evaluated. A combination of initial WIF, WIF-Level, and WIF-Slope and classification tree analysis improved prediction sufficiently to recommend their use with RTI.

Using Curriculum-Derived Progress Monitoring Data as Part of a Response-to-Intervention Strategy: A Case Study

The revised Individuals with Disabilities Improvement Education Act (2004) and subsequent Federal Regulations promote the use of alternative process of identifying students with specific learning disabilities based on how well a student responds to researched-based interventions. As these strategies are implemented, school psychologists have the opportunity to expand their roles and to assume leadership positions in implementing a response-to-intervention (RtI) model. A central element of all RtI approaches is the universal monitoring of students’ academic progress. As part of a general effort to implement a data-driven system, multiple sources of information may be used. This article contributes to these efforts by presenting a case study demonstrating how a school psychologist took the first steps to implement a low-cost, continuous progress monitoring procedure in one urban school. This was accomplished by using data readily available at the school site (reading probes included with the district reading curriculum) to develop a systematic way to monitor progress by creating local school norms and using existing reading benchmarks.

Generating Reading Interventions Through Experimental Analysis of Academic Skills: Demonstration and Empirical Evaluation

This article describes the application of experimental analysis methods for identifying reading fluency interventions for two elementary school students (fourth and fifth grade) referred for reading problems. For each student the experimental analyses examined use of rewards, instruction, and a treatment package containing both reward and instructional components across difficulty levels to determine the condition that led to the most efficient increases in reading fluency for each student. Based on the experimental analyses, individualized reading packages were developed that required minimal supervision. These interventions were carried out over time during normal instructional routines, and progress monitoring data were gathered to determine the effects of the interventions. Results are discussed in terms of how practitioners can use experimental analyses of academic skills within a broader response-to-intervention framework. Also, recommendations for future research in this area are made.

Effects of Reading Curriculm-Based Measurement (R-CBM) Teacher Feedback in General Education Classrooms

This study examined the effects of teacher feedback from Reading Curriculum-Based Measurement (R-CBM) progress results for low-performing students in general education classrooms. Participants included 44 second-grade teachers and 184 students in their low reading groups. After 5 weeks of progress monitoring, teachers in the two experimental groups were given progress results of (a) a single student in the reading group or (b) all students in the reading group. Teachers in the control group received no progress monitoring feedback. Progress feedback did not affect subsequent progress as hypothesized. However, students' reading progress in all three groups improved reliably during the second half of the study, suggesting some reactive benefits of progress monitoring. The need to increase the type and quality of teacher feedback and to support teachers to use progress monitoring data in general education settings to make instructional changes is discussed.

Monitoring Implementation of Reciprocal Peer Tutoring: Identifying and Intervening With Students Who Do Not Maintain Accurate Implementation

This study evaluated students' daily implementation of a reciprocal peer tutoring procedure for mathematics and methods for supporting student implementation. Additionally, this study evaluated the reliability of progress monitoring data collected by students. The peer tutoring procedure was designed such that completion of each treatment component produced a unique permanent product. Student integrity was defined as the percentage of treatment components completed. The effect of performance feedback for students with poor implementation was evaluated. Five of the 37 students who implemented the peer tutoring procedure were provided additional follow-up support due to poor implementation. The accuracy of students' implementation of the peer tutoring procedure improved markedly when they were provided performance feedback. With regard to the reliability of progress monitoring data collected by students, a very high level of agreement was obtained between adults who scored probes for digits correct per minute and tutor scoring of those probes. Implications for practice employing peermediated interventions and further research are discussed.

Effecting Superior Achievement Using Curriculum-Based Measurement: The Importance of Individual Progress Monitoring

The purpose of this study was to examine the importance of designing students' programs based on individual progress-monitoring data. For 20 weeks, 22 special education teachers monitored the mathematics progress of 42 students with mild to moderate disabilities, most of whom had learning disabilities, using curriculum-based measurement (CBM). For these 42 students in Grades 2 through 8, teachers made instructional adjustments based on CBM data. In addition, for each of the 42 students, teachers made the same instructional adjustments for a matched partner. Results indicated that students for whom teachers tailored instructional adjustments based on those students' own CBM data performed significantly better on a global achievement test than did their partners whose instructional adjustments were not based on their own assessment data. Implications for using CBM to effect superior achievement are discussed.

Relative Contribution of Progress Monitoring Strategies, Student Characteristics, and Program Variables to Instructional Outcomes in Resource Programs

The purpose of this study was to identify variables, aside from curriculum and instruction, that make a contribution to the effectiveness of special education resource room instruction. Of particular interest was whether different approaches to teachers use of progress monitoring data related to differences in achievement. A total of 343 students identified as learning disabled by their schools from grades one through six within 35 schools in a large city school district participated. Results of descriptive and regression analyses revealed students with learning disabilities made progress in reading proficiency. The results of this investigation implied that one non-alterable variable—initial level of performance—and one alterable variable—instructional group size—were the two factors consistently related to student reading progress. Although no achievement differences were obtained related to the data utilization approach used, frequency of instructional changes was related to the approach for evaluating progress data. Implications for monitoring student progress in reference to alterable and unalterable elements of instruction are considered.