Meaningful assessment

Abstract

Assessment of student learning outcomes is a key process used in education to both evaluate students' level of achievement and to identify opportunities for continuous improvement. The most prevalent technique for analyzing data collected from direct assessment methods is to distill the data to a single measure of central tendency, typically the arithmetic mean. Despite well known awareness and understanding of the limitations of the arithmetic mean, it is still commonly used because it is easy to calculate from the readily available data and is familiar to most educators. This paper argues that use of arithmetic mean alone is a poor assessment practice, and an alternate evaluation technique is presented in detail. To illustrate our conceptual arguments, a case study involving the assessment of an intermediate, college-level information technology course is presented. For the evaluation of an outcome in this course, assessment of student performance for the embedded indicators of that outcome are shown using both the commonly used arithmetic mean and what we believe to be a better, more meaningful assessment technique that places individual student performance data points into categories using an Individual Indicator Metric and then evaluates the group's overall performance based on the distribution of these student performances across the categories using a Group Indicator Metric. The paper's concluding section briefly addresses integrating indirect (subjective) evidence, combining all data source evaluations to evaluate an outcome, identifying and acting on opportunities for improvement, and reassessing changes. The central theme of the paper is that the veracity of assessment can be significantly improved with minimal extra effort.