On the complementarity of holistic and analytic approaches to performance assessment scoring.

Abstract

A holistic approach to performance assessment recognizes the theoretical complexity of multifaceted critical thinking (CT), a key objective of higher education. However, issues related to reliability, interpretation, and use arise with this approach. Therefore, we take an analytic approach to scoring students' written responses on a performance assessment. We focus on the complementarity of holistic and analytic approaches and on whether theoretically developed analytical scoring rubrics can produce sub-scores that may measure the 'whole' performance in a holistic assessment. We use data from the Wind Turbines performance assessment, developed in the iPAL project this study where 55 students at a German university participated. The (sub)scores generated from the scoring scheme empirically reproduced the theoretically assumed structure of CT, with valid and reliable scores in a three-dimensional model. The proposed interpretation of CT as assessed with a performance assessment and measured by the rating scheme was supported preliminarily. Our results support the complementarity of holistic and analytic approaches to assessing CT. When combined, they provide interpretable scores for a complex, multifaceted construct useful in diagnostic contexts.