Understanding validity issues surrounding test-based accountability measures in the US

Abstract

Purpose– This AERI-NEPC eBrief, the fourth in a series titled “Understanding validity issues around the world”, looks closely at issues surrounding the validity of test-based actions in educational accountability and school improvement contexts. The specific discussions here focus testing issues in the US. However, the general principles underlying appropriate and inappropriate test use in school reform and high stakes public accountability settings are applicable in both domestic and international settings. This paper aims to present the issues.Design/methodology/approach– This policy brief is based on a synthesis of conference proceedings and review of selected pieces of extant literature. It begins by summarizing perspectives of an invited expert panel on the topic. To that synthesis, the authors add their own analysis of key issues. They conclude by offering recommendations for test developers and test users.Findings– The authors conclude that recurring validity issues arise with tests used in school reform and public accountability contexts, because the external tests tend to be employed as policy instruments to drive reforms in schools, with unrealistic timelines and inadequate resources. To reconcile the validity issues with respect to educational assessment and forge a coherent understanding of validity among multiple public users with different agendas, the authors offer several recommendations, such as: adopt an integrated approach to develop content and standards of proficiency that represent a range of cognitive processes; support studies to examine validity of assessments and the effects of decisions taken with assessment data before results are fed into high stakes accountability-related actions that affect teachers, leaders or schools; align standards, curricula, instruction, assessment, and professional development efforts in schools to maximize success; increase capacity-building efforts to help teachers, administrators, policy makers, and other groups of test users learn more about assessments, particularly, about appropriate interpretation and use of assessment data and reports.Originality/value– Baker points out that in response to growing demands of reformers and policy-makers for more frequent and rigorous testing programs in US public education, results from a single test tend to get used to meet a variety of public education needs today (e.g. school accountability, school improvement, teacher evaluation, and measurement of student performance). While this may simply be a way to make things more cost-efficient and reduce the extent of student testing in schools, a consequence is inappropriate test use that threatens validity in practice settings. This policy brief confronts this recurring validity challenge and offers recommendations to address the issues.