Towards more robust evaluation of policies and programmes in education: identifying challenges in evaluating DEIS and Reading Recovery

Abstract

ABSTRACT In recent years, countries including the UK and USA have seen advancements in the use of Randomised Controlled Trials in education, progress that has not been mirrored in Ireland. Ireland does not have a strong tradition of using experimental or quasi-experimental evaluation designs for monitoring and evaluation of education policy despite well-articulated commitments to the latter activities. An example is the evaluation of the Delivering Equality of Opportunity in Schools (DEIS) programme where little is known about the causal impact of DEIS on student outcomes, despite considerable State investment over almost 20 years. There appears to have been comparatively little critical review of the literacy and numeracy programmes offered to DEIS schools, with those included at the outset largely still on offer. Here, we draw on the wider literature on the evaluation of complex interventions and consider what lessons may be learned for Ireland, illustrating challenges and opportunities by focusing on DEIS and Reading Recovery (a literacy support programme widely used in DEIS schools). We advocate for greater use of experimental designs where feasible and appropriate; further gathering of comparable measures of achievement at pupil level to facilitate monitoring; and detailed consideration of quasi-experimental methods that can support causal conclusions.