Treatment Fidelity in Multisite Evaluation

Abstract

Program implementation data obtained from the repeated observation of teachers delivering one of two early childhood literacy programs to economically disadvantaged students in a large southwestern school district were analyzed to estimate protocol adherence levels at the onset of the intervention as well as the change in adherence over the intervention period. Application of multilevel growth models to the classroom observation data revealed that fidelity to program protocol varied within and between treatment sites during the initial observation and over time. An exploratory examination of select teacher, classroom, and site characteristics indicated that the background characteristics of teachers and contextual factors in the treatment environment were associated with the fidelity outcomes. These results provide some insight into the range of factors that are associated with protocol adherence and highlight the challenge of achieving and maintaining fidelity to a treatment intervention that is delivered by multiple providers over multiple treatment sites.