The (Mis)measure of Schools: How Data Affect Stakeholder Knowledge and Perceptions of Quality

Abstract

Purpose/Objective Under the reauthorized Every Student Succeeds Act (ESSA), states and districts retain greater discretion over the measures included in school quality report cards. Moreover, ESSA now requires states to expand their measurement efforts to address factors like school climate. This shift toward more comprehensive measures of school quality provides an opportunity for states and districts to think intentionally about a basic question: What specific information should schools collect and report to their communities? Setting This study took place in the community surrounding a small, highly diverse urban school district. Population/Participants Forty-five local residents representing a range of demographic backgrounds participated in a modified deliberative poll with an experimental treatment. Intervention/Program/Practice We randomly assigned participants into two conditions. In the first, participants accessed the state web portal, which houses all publicly available educational data about districts in the state. In the second condition, participants accessed a customized portal that contained a wider array of school performance information collected by the research team. Research Design This mixed-methods study used a modified deliberative polling format, in conjunction with a randomized controlled field trial. Data Collection and Analysis Participants in both conditions completed a battery of survey items that were analyzed through multiple regressions. Findings/Results When users of a more holistic and comprehensive data system evaluated unfamiliar schools, they not only valued the information more highly but also expressed more confidence in the quality of the schools. Conclusions/Recommendations We doubt that more comprehensive information will inevitably lead to higher ratings of school quality. However, it appears—both from prior research, from theory, and from this project—that deeper familiarity with a school often fosters more positive perceptions. This may be because those unfamiliar with particular schools rely on a limited range of data, which fail to adequately capture the full range of performance variables, particularly in the case of urban schools. We encourage future exploration of this topic, which may have implications for school choice, parental engagement, and accountability policy.