Abstract
Students’ Evaluations of Teaching (SETs) are the most common way to measure teaching quality in Higher Education: they are assuming a strategic role in monitoring teaching quality, becoming helpful in taking the major formative and summative academic decisions. The majority of studies investigating SETs reliability focus on the instruments and the procedures adopted to collect students' evaluations rather than on the capability of the students as teaching quality assessors. In order to overcome this lack, a study has been carried out with the aim of measuring SETs reliability in terms of inter-student agreement and intra-student agreement. The results of our study show that the majority of students provided substantially repeatable evaluations whereas only a few students provided almost perfectly repeatable evaluations; the evaluations provided by different students generally slightly agreed, which means that the students did not share the same opinions and beliefs on teaching quality.
Highlights
Differently from the majority of available studies, which rather focus on the instruments and the procedures adopted to collect Student Evaluations of Teaching (SETs), our study aims at investigating the peculiar abilities of the students as teaching quality assessors by measuring SETs reliability in terms of interstudent and intra-student agreement
The intra-student agreement was generally higher than the inter-student agreement: 73% of students were at least substantially repeatable on both Numeric Rating Scale (NRS) and Verbal Rating Scale (VRS) whereas 19% of them were even almost perfectly repeatable on both NRS and VRS
The majority of students show over the years values of KWU|VRS higher than those of KWU|NRS for about half of them the repeatability on the two rating scales belong to the same agreement categories and only for few (i.e., 10) students KWU|NRS and KWU|VRS belong to no-adjacent categories of agreement
Summary
Measuring the student experience is assuming increasingly importance in Higher Education (hereafter, HE) representing a widespread method for evaluating teaching quality whose importance is relevant for taking the major formative and summative academic decisions (Berk, 2005; Gravestock & Gregor-Greenleaf, 2008; Onwuegbuzie et al, 2009).Student ratings, known as Student Evaluations of Teaching (SETs), have dominated as the primary measure of teaching quality over the past 40 years (e.g., Centra, 1979; Seldin, 1999; Emery at al., 2003; Gaertner, 2014) forming the basis for the rankings of HE institutions. The above considerations call into question the opportunity to consider the students as able to provide reliable evaluations on teaching quality For this reason, differently from the majority of available studies, which rather focus on the instruments and the procedures adopted to collect SETs, our study aims at investigating the peculiar abilities of the students as teaching quality assessors by measuring SETs reliability in terms of interstudent and intra-student agreement. Differently from the majority of available studies, which rather focus on the instruments and the procedures adopted to collect SETs, our study aims at investigating the peculiar abilities of the students as teaching quality assessors by measuring SETs reliability in terms of interstudent and intra-student agreement The former allows evaluating the students’ ability to provide the same score, on average, as the other students whereas the latter, known as repeatability, allows evaluating the students’ ability to score consistently a given quality item in different occasions
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
