Comparison of standardized assessment methods: logistics, costs, incentives and use of data

Abstract

Critical thinking, problem solving and communication are fundamental elements of undergraduate education, but methods for assessing these skills across an institution are susceptible to logistical, motivational and financial issues. Queen’s University conducted two research studies investigating the use of standardised tests to assess cognitive skill development across the institution. Synthesis of results from implementing the Collegiate Learning Assessment (CLA+), the Critical Thinking Assessment Test (CAT), and the HEIghtenTM test found that student test effort was a significant factor, and effort level correlated with performance at r = .33. Test incentives were also a significant factor; effort levels for the $25 financial incentive group were one standard deviation higher than effort for the in-class test group. A dedicated computer lab was the preferred option for computer-based testing. A paper-based test was found to be much simpler to administer, but test results were not available for a long time, therefore limiting the usefulness of data. The true cost of tests was greater than the price of the instrument; recruitment, training, proctoring and marking costs need to be included in the calculation. Generally speaking, alignment of test objectives with student or course objectives, and timeliness of data, were key for participation and motivation.