The limits of measurement: misplaced precision, phronesis, and other Aristotelian cautions for the makers of PISA, APPR, etc.

Abstract

ABSTRACTQuantitative measures of student performance are increasingly used as proxies of educational quality and teacher ability. Such assessments assume that the quality of educational practices can be unambiguously quantitatively measured and that such measures are sufficiently precise and robust to be aggregated into policy-relevant rankings like league tables or employment-relevant effectiveness scores for teachers. In this paper I direct attention to a theoretical tradition which casts a long shadow of doubt over this enterprise. Drawing on Aristotelian and pragmatist scholarship, I argue that the classroom is a domain of practical knowledge or phronesis where quality is best assessed by a jury of experienced practitioner’s context-sensitive judgment. This is because phronesis is predominantly tacit and resists codification. It cannot be made explicit without major distortions. The current worldwide drive that aims at measuring educational quality in precise quantitative terms commits the fallacy of misplaced precision and violates the rule of “requisite variety” which suggests that an assessment regime is at least as complex as the system it assesses. The discussion is placed in the history of the controversy between proponents of educational efficiency, which was opposed by pragmatist philosophers like John Dewey and William James.