Innovation, evaluation design and typologies of professional learning

Abstract

ABSTRACTBackground: Current policy discourses emphasise the importance of evidence in education, including evidencing the impact of teacher professional learning on student outcomes. Randomised controlled trial (RCT) designs are promoted to measure ‘impact’. Recent debates about this reflect longer standing methodological disputes. Advocates of comparative approaches contend that these are uniquely capable of establishing causality. However, others dispute this and consider their application in education as often being flawed. Whilst acknowledging the importance of these debates, our concern is how RCTs and similar evaluation designs are specifically used to evaluate innovations in which professional learning is important.Purpose: Arguably, professional learning is often under-theorised within experimental and quasi-experimental designs. The purpose of this paper is to address this by encouraging developers of innovations and evaluators to consider a proposed typology of professional learning and other important relevant methodological issues. This is so that developers of innovations that involve professional learning are better able to theorise their endeavours and to support more appropriate design of RCTs and other forms of evaluation of innovations.Sources of evidence: Theoretical and methodological literature from diverse fields is drawn on, namely: descriptions of RCT implementation and process evaluation designs; research on effective professional development; and theoretical models of professional learning. Insights and theories from this literature are used to develop and illustrate the typology and to identify methodological concerns and potential ways to address these.Main argument: In trials of those innovations that involve professional learning, there is both assessment of the extent to which professional learning occurs and also of whether resulting changes in practice improve outcomes. A novel typology of three different ways that professional learning may occur in innovations is proposed. This is related to the centrality (or not) of professional learning to the innovation’s success and related to the form and purpose of the professional learning involved. The three analytical categories described are pedagogical professional learning, technical professional learning and curriculum professional learning. Based on this typology, features of professional learning that are likely to lead to impact on student outcomes are discussed. Tensions are identified between the implementation of experimental and quasi-experimental designs and interpretation of resulting evidence. Further, tensions are identified between the complex and recursive nature of pedagogical professional learning systems and the models of linear pathways in some RCT designs. This is illustrated by discussing examples of innovations and trials.Conclusion: The proposed typology and greater theorisation of professional learning can support more robust evaluation design. It is important to assess rigorously teacher learning alongside changes in student outcomes.