Treatment Fidelity in Applied Educational Research: Expanding the Adoption and Application of Measures to Ensure Evidence-Based Practice

Abstract

Abstract In intervention research, treatment fidelity is defined as the strategies that monitor and enhance the accuracy and consistency of an intervention to ensure it is implemented as planned and that each component is delivered in a comparable manner to all study participants over time. Reviews of the literature in special education and other disciplines reveal that reports of treatment fidelity are limited. In this article, we examine some recommendations made by the National Institutes of Health Behavior Change Consortium that may be adapted to document treatment fidelity in educational research. We discuss the critical importance of planning for, collecting, and reporting treatment fidelity data at each stage of intervention research and discuss the implications of these practices for validity issues, efficacy and effectiveness studies, and cost-benefit considerations. Throughout the article, we use our own classroom-based research to provide examples of expanding treatment fidelity in randomized field trials. ********** There is increased emphasis on investigative and experimental rigor for educational researchers because of the No Child Left Behind Act of 2001 (NCLB) and the reauthorization of the Elementary and Secondary Education Act, which demands the use of "scientifically-based research" as the basis for educational programming and classroom practices. As in the early developmental stages of research-based fields such as medicine and psychology, education has often relied on tradition, anecdotal evidence, and a collective sense of expert opinion to guide its instructional practices. These sources of evidence, however, do not effectively provide educators with precise or accurate enough information to determine which educational practices are truly effective, for whom, and under what conditions. Scientifically- based research, on the other hand, "... means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs" (NCLB, 2001). Such studies are judged on their scientific merit, which involves appropriate research design, thorough investigative methods, and valid and reliable measures to provide objective evidence from which to draw conclusions about an educational practice. Yet, scientifically-based research in education not only encompasses rigorously conducted efficacy studies (i.e., whether an intervention can work under specified conditions), but also includes effectiveness studies that show intervention outcomes under less controlled, authentic educational situations across a variety of conditions. As educational research continues to evolve, researchers need to determine how dissemination of scientifically-based effective practices can be achieved for the benefit of thousands of students, teachers, and other education professionals. A critical factor in determining the efficacy, effectiveness, and successful dissemination of an educational practice is ensuring that the professionals who are responsible for its implementation deliver an intervention under study with accuracy and conformity. Treatment fidelity is defined as the strategies that monitor and enhance the accuracy and consistency of an intervention to (a) ensure it is implemented as planned and (b) make certain each component is delivered in a comparable manner to all participants over time (Bellig et al., 2004; Detrich, 1999; Dumas, Lynch, Laughlin, Smith, & Prinz, 2001; Lane, Bocian, MacMillian, & Gresham, 2004). If treatment fidelity is not measured, researchers cannot ascertain with confidence whether study outcomes were due to treatment or to factors incidental to the intervention (Bellg et al., 2004). Lack of treatment fidelity data also makes replication of, and comparisons between, interventions problematic (Hester, Baltodano, Gable, Tonelson, & Hendrickson, 2003). …