A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses and challenges for 'Progress 8' raised by COVID-19.

Abstract

School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the ‘Progress 8’ secondary school accountability system for state‐funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measure; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi‐year averages. We then discuss challenges for Progress 8 raised by the COVID‐19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England. Context and implicationRationale for this studyThis study fulfils a need for a comprehensive statistical evaluation of the Progress 8 school accountability system in England, which has been in place since 2016.Why the new findings matterA review of Progress 8 is an important contribution as decisions on its calculation and presentation have important consequences for schools and pupils.Implications for educational researchers and policy makersOur findings on the statistical strengths and weaknesses of Progress 8 and the resulting recommendations made for improvements to the measure and school accountability have clear implications for informing policy makers in the educational system. This includes wider lessons extending beyond Progress 8 to other performance metrics and to other school systems internationally where similar issues apply. Additionally, the review highlights key concerns that the public and school practitioners may like to consider when drawing upon Progress 8 to make decisions around school choice and in assessing how well a school is performing.