Cross-Level Mediation in School-Randomized Studies of Teacher Development: Experimental Design and Power

Abstract

Professional development for teachers is regarded as one of the principal pathways through which we can understand and cultivate effective teaching and improve student outcomes. A critical component of studies that seek to improve teaching through professional development is the detailed assessment of the intermediate teacher development processes that scaffold program content through three key types of outcomes—teacher knowledge, instruction, and student learning. Cross-level and sequential mediation strategies that probe and connect these processes and outcomes emerge as an important design consideration in these studies. We derive formulas that track the power with which school-randomized designs can detect professional development effects as they operate through a sequence of teacher-level mediators to affect student outcomes (e.g., school-randomized professional development studies). The results suggest that the sample sizes typically seen in well-planned experiments targeting total effects (e.g., 30–100 schools) can produce comparably high or disparately low levels of power for mediation effects—the similarity depends heavily on context and concomitant parameter values. The results are implemented in the PowerUpR package and in the PowerUpR Shiny application.