Building up adjusted indicators of students’ evaluation of university courses using generalized item response models

Abstract

This article advances a proposal for building up adjusted composite indicators of the quality of university courses from students’ assessments. The flexible framework of Generalized Item Response Models is adopted here for controlling the sources of heterogeneity in the data structure that make evaluations across courses not directly comparable. Specifically, it allows us to: jointly model students’ ratings to the set of items which define the quality of university courses; explicitly consider the dimensionality of the items composing the evaluation form; evaluate and remove the effect of potential confounding factors which may affect students’ evaluation; model the intra-cluster variability at course level. The approach simultaneously deals with: (i) multilevel data structure; (ii) multidimensional latent trait; (iii) personal explanatory latent regression models. The paper pays attention to the potential of such a flexible approach in the analysis of students evaluation of university courses in order to explore both how the quality of the different aspects (teaching, management, etc.) is perceived by students and how to make meaningful comparisons across them on the basis of adjusted indicators.