Student Evaluations Of Teaching Research Articles

Academic entitlement and Ratemyprofessors.com evaluations bias student teaching evaluations: Implications for faculty evaluation and policy-lenient professors’ occupational health

This study explored relationships between academic entitlement (AE) and Ratemyprofessors.com (RMP) use. It also investigated, while controlling for AE, if RMP evaluation positivity influences students' intentions to ask for policy exemptions, beliefs professors would provide them, intentions to reward and punish professors contingent upon provision of policy exemptions by improving or lowering their student teaching evaluations, and intentions to evaluate and reenroll with professors. Following exposure to RMP evaluations, participants (n = 320) rated their intentions and beliefs toward a fictional professor. They also completed an AE measure. AE was related to frequency of writing RMP evaluations as well as participants' intentions to ask for exemptions, beliefs they would receive them, and intentions to reward and punish professors. RMP evaluation positivity affected participants' intentions to ask for and beliefs they would receive policy exemptions as well as intention to evaluate and reenroll with professors. Effects did not differ by professor or student gender. Participants reported intention to improve the evaluation of professors who provide any policy exemption. This study's findings suggest that student attitudes related to AE and impacted by RMP evaluations have significant implications for professors' occupational health via requests for policy exemptions and the consequences of professors' responses to them. These findings also contribute to the body of evidence that student teaching evaluations do not exclusively measure teaching effectiveness. Similar to grade leniency, policy leniency may bias student teaching evaluations. These contribute to the ongoing discussion of the use of student teaching evaluations in faculty personnel decisions and underscore the need for robust approaches to professor evaluation.

Predicting undergraduate student evaluations of teaching using probabilistic machine learning: The importance of motivational climate

The purpose of this study was to understand the complex interactions within a course among motivational climate variables and student evaluations of teaching (SETs) by developing online simulators using probabilistic machine learning. We used data from 2938 undergraduate students in 30 classes to create online simulators based on Bayesian Belief Networks. We created bubble charts, line graphs, and radar charts to show the relationships between the study variables. Findings showed that (a) the motivational climate variables—as measured by the MUSIC Model of Motivation variables—are the largest predictors of SETs, (b) student interest (in the coursework and instructional methods) is the overall largest predictor of SETs, (c) the relationships between the motivational climate variables and SETS are nonlinear, (d) the ease of the course and demographic variables are only weakly associated with SETs, and (e) the largest predictors of instructor and course rating are similar, but somewhat different.

A framework for using SET when evaluating faculty

ABSTRACT This paper presents a framework for utilizing Student Evaluations of Teaching (SET) in faculty evaluations. Recognizing the ongoing debate about the validity of SET as a measure of teaching effectiveness, the authors agree with scholars who propose viewing SET as a tool for gauging ‘student perceptions of learning’. They present a method that combines post-estimation residuals from an Ordinary Least Squares (OLS) model, which accounts for key confounding factors, with simple mean SET scores. This approach provides a more nuanced perspective on each faculty member's individual ratings. The authors draw on SET data from courses taught by tenured faculty in the Faculty of Arts and Sciences at a large, private university in the Northeastern United States. Their findings suggest that the proposed residual framework can help address some of the concerns associated with the use of SET in personnel decision-making. However, they caution that this approach requires careful explanation and should be used in conjunction with a comprehensive review of a professor's full teaching portfolio. The study contributes to the ongoing discourse on the use of SET in faculty evaluations, offering a more equitable and nuanced approach to interpreting SET data.

Measurement Invariance Analysis of Engineering and Mathematics Majors Students’ Understanding of Mathematics Courses Teaching Practices

This study attempts to understand the source of variation in the Students’ Evaluation of Teaching (SET) of mathematics courses by comparing the data structure of engineering major students and mathematics major students’ datasets. The sample considered in this study consists of 644 students distributed into two majors: 237 mathematics students and 407 engineering students who filled out a 20-item SET questionnaire to rate the teaching practices of three different mathematics courses offered by the Department of Mathematics. The hypothesis tested in this study is: that variation in students’ perceptions of mathematics course teaching practices is different based on students’ majors (Mathematics versus Engineering). Measurement invariance (MI) analyses were used to examine the source of variation in the datasets and to compare engineering and mathematics students’ perceptions of the teaching effectiveness of mathematics courses. While the results of this study provide evidence of the SET validity, it was found that engineering students differently perceive three out of twenty of the SET questionnaire items when compared with mathematics major students.

Academic developers’ roles and responsibilities in strengthening student evaluation of teaching for educational enhancement

ABSTRACT Student evaluation of teaching (SeT) is ubiquitous in higher education but has been criticized by many scholars because of low use for course improvement benefiting student learning. Academic developers (ADs) are responsible for pedagogical courses and to support leadership and academics in processes enhancing educational quality. We could therefore expect ADs to play a key role in SeT practice. This paper investigates how Norwegian ADs and Academic Development Units (ADUs) are engaged in SeT practice. Norwegian academic leaders of twelve ADUs are interviewed about ADs’ roles and responsibilities in evaluation policy and SeT practice. The empirical data are analyzed by using the terms ‘accountability’ and ‘professional responsibility’ aiming to better understand the use of SeT. Before the turn of the millennium, ADs were actively engaged in SeT practice, but our study found that they are no longer central actors and little time is spent on the topic in pedagogical courses. Today evaluation is dominated by accountability logic and is professionalized by administration and leadership. We will argue that there is a potential to strengthen SeT as a tool for educational development by inviting ADs into evaluation policy development and by including evaluation as part of pedagogical courses.

Course design as a stronger predictor of student evaluation of quality and student engagement than teacher ratings

AbstractResearch on Student Evaluation of Teaching (SET) has indicated that course design is at least as important as teachers’ performance for student-rated perceived quality and student engagement. Our data analysis of more than 6000 SETs confirms this. Two hierarchical multiple regression models revealed that course design significantly predicts perceived quality more strongly than teachers, and that course design significantly predicts student engagement independent of teachers. While the variable teachers is a significant predictor of perceived quality, it is not a significant predictor of student engagement. In line with previous research, the results suggest it is important to highlight the vital impact of course design. The results are discussed particularly in relation to improved teaching practice and student learning, but also in terms of how student evaluations of teaching can be used in meaningful ways.

Performance evaluation in teaching: Dissecting student evaluations in higher education

Numerous studies have highlighted the significant role of Student Evaluations of Teaching (SETs) as a key metric for assessing teaching quality in Higher Education (HE). Building upon these insights, our study introduces an innovative four-tiered model, derived from diverse research, to examine the reliability of SETs. This model addresses biases associated with SETs, delving into both statistical anomalies and cognitive biases, with particular emphasis on often-overlooked hidden context and timing factors. We reveal that these biases can distort SET scores, leading to potentially inaccurate representations of both individual and comparative academic performances. The implications of our research are significant for those influencing HE policy-making and performance evaluation. We echo previous calls for a more expansive approach to teaching effectiveness, essential for genuine insight into teaching quality. By adopting this perspective, HE can design better-informed strategies, ensuring policies and practices reflect the diverse nature of teaching excellence.

Validity of student evaluation of teaching in higher education: a systematic review

IntroductionData obtained from students regarding the quality of teaching are used by higher education administrators to inform decisions concerning tenure, promotion, course development and instructional modifications, among others. This article provides a review regarding studies conducted to examine the validity of student evaluation of teaching, specifically focusing on the following objectives: (1) identify the context where studies have been conducted on student evaluation of teaching; (2) find out the methodologies usually employed for assessing the validity of student evaluation of teaching; and (3) establish the sources of measurement error in student evaluation of teaching.MethodsThe systematic review was conducted based on the PRISMA checklist. The databases searched include Scopus, Web of Science (WoS), Google Scholar, PubMed, MEDLINE, ERIC, JSTOR, PsycLIT, EconLit, APA PsycINFO and EBSCO using some specific keywords. After applying the four eligibility criteria, 15 papers were left to be analyzed.ResultsIt was discovered that the generalizability theory approach was mostly used to understand the validity of student evaluation data. The review revealed that students were found at the centre of inconsistencies in the evaluation process.DiscussionThe general impression from the review is that the credibility and validity of teaching evaluation outcomes is questionable, considering the several sources of errors revealed. The study recommended closely studying these sources of errors (e.g., rating behaviours of students).

Student Evaluation of Teaching: A Tool for the Assessment of Teaching Learning Practices in Higher Education Institutions

Student Evaluation of Teaching (SET) is a way to evaluate the teaching effectiveness of teachers, and teaching-learning practices based on students' feedback. It is an evaluation of the teachers by students. SET is considered a tool to assess teaching effectiveness mostly in higher education institutions. Globally, universities use SET, especially in taking the decision of tenure and promotion of professors. This article attempts to gather students' responses regarding the teaching performance of their teachers from one of the campuses in Kailali district, Nepal. A survey research design was adopted to collect students’ responses. A random sampling method was employed to select undergraduate students from the selected campus. Students were asked to fill out a questionnaire consisting of the statements managed on a 5-point Likert scale. IBM SPSS Statistics 20 was used to manage and visualize the data, and the descriptive parameters were calculated for analysis. The results show that the teaching performance of the teachers was quite good in two categories, viz., management and motivation aspects, whereas the teaching performance in the instruction aspect was only satisfactory. This study shows that teachers need to improve their instructional strategies for the betterment of teaching and learning. The study will motivate other scholars and institutions to initiate the practice of SET for the enhancement of teaching and learning in their respective institutions.

Understanding the impact of biased student evaluations: an intersectional analysis of academics’ experiences in the UK higher education context

ABSTRACT This paper aims to offer an understanding of how (intersectional) bias in student evaluations of teaching affects academics’ mental health and career progression. Despite the widespread acceptance of student evaluations, an emerging stream of research has begun to highlight the biases and prejudices that underpin much of data collection when it comes to student evaluation surveys both in terms of who completes the evaluation and also in terms of who is being evaluated. While most studies provide analyses of students’ comments and illustrate the abuse that is directed at (marginalised) academics, very little research has focused on the impact of this process on academics. This study offers an overview of academics’ experiences with student evaluations by drawing on an intersectional analysis of 17 interviews with academics employed in UK Business Schools. The findings illustrate the detrimental impact of student evaluations on academics’ mental health and career progression. This study also shows how institutional pressures to keep students happy impact academics’ wellbeing and teaching approaches.

The Student Evaluation of Teaching Premium for Clinical Faculty in Economics

This article uses student evaluation of teaching (SET) data for 947 faculty members affiliated with 90 U.S. colleges and universities to study the presence of a teaching quality rating premium for clinical economics faculty relative to traditional tenure-track economics faculty. Based on OLS estimation, we find this difference ranges between 3.9% and 4.8% and is robust to different econometric model specifications. Moreover, the average treatment effect from a propensity score weighting approach suggests that the difference ranges between 5.8% and 6.1%. Lastly, our analysis produces an institutional ranking of economics departments based on department-level SETs. Overall, our findings are encouraging signs for the hiring and retention of clinical faculty in economics departments.

Design and Implementation of Remote Piano Teaching Based on Attention-Induced Multi-Head Convolutional Neural Network Optimized with Hunter–Prey Optimization

The continuous progress of multimedia technology in music educational institutions has led to the recognition of its importance in our country and society. The traditional approach to piano teaching has its limitations, which can be overcome by adopting alternative approaches to the instrument, using advances in science and technology. For pianist, expressing emotions and thoughts through music is crucial, and teachers can now use multimedia tools to exemplify their musical skills to students effectively. This manuscript proposes the Remote Piano Teaching Based on Attention-Induced Multi-Head Convolutional Neural Network Optimized with Hunter–Prey Optimization to improve the piano-teaching quality. At first, input data is taken from Piano Triad Wavset dataset. Afterward, the data are fed to preprocessing stage. The preprocessing stage involve data cleaning or scrubbing that is the process of identifying errors, inconsistencies, and incorrectness in a dataset with the help of adaptive distorted Gaussian matched filter. Then, the preprocessed output is fed to Attention-Induced Multi-Head Convolutional Neural Network (AIMCNN) for effectively predict the piano-teaching quality. The hunter–prey optimization (HPO) algorithm is proposed to optimize the parameters of Attention-Induced Multi-Head Convolutional Neural Network. The performance of the proposed technique is evaluated under performance metrics like accuracy, computational time, learning skill analysis, learning activity analysis, learning behavior analysis; student performance ratio and teaching evaluation analysis are evaluated. The proposed RPT-AIMCNN-HPO attains better prediction accuracy 12.566%, 12.075% and 15.993%, higher learning skill 15.86%, 15.26% and 16.25% compared with existing methods.

The Merdeka Curriculum Reform on Aspects of Historical Thinking Skills in History Subjects in Senior High Schools

The aims of this research are (1) to plan history learning using the Merdeka curriculum for historical thinking skills, (2) to improve the Merdeka curriculum in relation to aspects of history learning for historical thinking skills, and (3) As an evaluation of history teaching and learning activities based on thinking skills. Merdeka Curriculum Aspects of Historical Thinking Skills in SMA Pasundan 2 Cimahi, Especially in Schools that Implement the Merdeka Curriculum. This research applies qualitative methods with a case study approach. The data sources are informants: teachers, students, documents, curriculum, and historical thinking skills. Research findings show that (1) Merdeka Curriculum history learning planning is carried out by teachers who prepare their own ATP and learning modules; However, one of the schools where he could use KOSP was closed, and the teachers at both schools did not know the whereabouts and contents of his KOSP. (2) Implementation of history learning depends on student performance. There are several learning processes that do not follow learning principles, such as Education modules that are not prepared. PLP students take classes based on a unique curriculum. There are no student teaching evaluations for teachers. There are also monotonous learning methods, less situational learning, and unused social studies books. (3) In evaluating teaching and learning activities, there are several cases where the implementation does not follow the principles of teaching and learning evaluation based on the Merdeka Curriculum. Teaching and learning activities in history classes are not in line with assessment principles. There are no student self-assessments or peer assessments, no in-module assessment rubrics, and no post-test quizzes. However, teachers have tried to adhere to some principles and implement their own curriculum as closely as possible. Implementing the Merdeka Curriculum from the perspective of historical thinking skills in history education can be adapted to the implementation of the Merdeka Curriculum established by the Ministry of Education, Culture, Research and Technology of the Republic of Indonesia.

Sentiment Analysis of Student Evaluation for Teachers Using Valence-Aware Dictionary and Sentiment Reasoner

This study analyzed the quality of teaching using the Student Evaluation of Teaching (SET) and the Valence Aware Dictionary and Sentiment Reasoner (VADER) to assess textual comments, providing a comprehensive view of teaching effectiveness beyond numerical ratings. The objectives were to identify faculty strengths and areas for improvement based on student feedback, analyze sentiment toward teaching methods, and determine the optimal number of clusters within the dataset. The analysis included 28,222 student comments from three semesters, preprocessed through tokenization, stopword removal, part-of-speech tagging, and lemmatization. A word cloud visualized common terms, while K-means clustering and the Elbow method identified five as the optimal number of clusters. Results indicate that most comments are positive, emphasizing effective teaching methods' role in creating a positive educational experience. The findings suggest integrating machine learning with VADER and expanding the dataset for broader insights. Institutions should develop effective teaching strategies, prioritizing regular feedback collection and analysis.

Sentiment classification model for student teaching evaluation based on deep learning technology

Sentiment classification model for student teaching evaluation based on deep learning technology

Does a rose by any other name smell as sweet? Assessing student evaluation of teaching ratings pre- and during the COVID-19 lockdown: an Australian study

The sudden sweep of the COVID-19 pandemic that led to significant lockdowns, and only gradual re-openings of societal interactions, forced the higher education sector to make significant changes in teaching and learning practices. Australian universities were no exception to this process. The adoption of almost universal online teaching in the face of lockdowns was one of the main changes executed by these universities. During this transition, instructors and students faced a plethora of challenges, with the former needing to master the necessary technological tools in a short period of time, especially with respect to creating and teaching online courses, while the latter had to deal with the socio-economic impacts of being unable to interact with their milieu in an orthodox fashion. This study examines students’ opinions about the effectiveness of their learning experiences prior to and during the pandemic period, within an Australian context. Utilising a pooled OLS methodology, we estimate the impact of the COVID-19 pandemic on student evaluation of teaching for a number of courses taught at an Australian university. We find statistically higher evaluation of teachers in the area of organisation, focus and inspiration during the pandemic period (where online teaching predominated) compared to the pre-pandemic period. This bifurcation is accentuated in favour of instructors from English-speaking backgrounds (ESBs) and for those teaching postgraduate courses.

When to conduct student evaluation of teaching surveys: before or after the final examination?

Student evaluation of teaching (SET) involves numerous conceptual and methodological problems. This paper focuses on a specific methodological dilemma: whether to conduct SET surveys before or after the final examination. This decision is a critical administrative and practical issue that can affect the level and quality of SET results. To address this problem, a systematic literature search was initially performed. However, it found only 17 studies and inconclusive results. In the empirical component, a replication survey (n = 5,077) was conducted. It posed seven course-related questions to the same students before and after their final examination. A slightly higher but statistically significant (p < 0.001) average score was obtained in the after-examination survey (a 0.06 difference on a 1–5 scale). This increase was partly attributed to a positive examination experience. The overall test–retest correlation (r = 0.61) indicated relatively low reliability, suggesting that the before-examination survey better reflected students’ attitudes. The data quality was also higher in the before-examination survey, although the before-examination survey could not properly assess the examination-related aspects of the course. From a methodological perspective, the most comprehensive insight seems to be derived from conducting SET surveys before the examination and following up with a brief after-examination survey.

Students’ Evaluation of Teaching (SET) for Improving Learning and Teaching Quality in HE: Students’ and Teachers’ Perspectives

Various academic institutions in higher education (HE) allocate their resources differently to improve the quality of teaching and learning. To measure and identify the teaching and learning quality in any educational context, institution performance in this field must be evaluated using different techniques and instruments, one of which is Student Evaluation of Teaching (SET). The study aims to determine the use of SET to improve teaching quality based on the opinions of teachers (n = 6) and students (n = 413). This is achieved by investigating 1) the degree of alignment or divergence between teachers’ and students’ ratings of a particular course, 2) how teachers value students’ feedback on teaching to improve their teaching practises, 3) students’ perspectives on using SET in improving teaching quality and 4) the implications for implementing SET effectively in HE context. These were addressed using an adapted version of the Students’ Course Evaluation Questionnaire (SCEQ), and the Mann-Whitney test was used to analyse the data. The findings reveal a statistically significant correlation coefficient (r = 0.739) between teachers’ and students’ evaluations. The associated p-value (0.058) is slightly greater than the conventional threshold of p ≤ 0.05. The results of the Mann-Whitney U test indicated an overall concurrence between teachers’ and students’ evaluations, although certain inconsistencies were observed. The findings indicate a consensus among teachers and students regarding SET’s efficacy in enhancing instructional quality.

Veterinary Student Evaluations of Teaching: Scores and Response Rate When Administered before or after Final Exams

In higher education, instructors and administrators use student evaluations of teaching (SETs) as formative and summative assessments of instruction; thus, they need adequate response rates for optimal validity and reliability. Veterinary students are often requested to complete dozens of SETs each semester, and response rate is shown to decline as the number of SETs increases. Allowing students to complete SETs after final examinations has been suggested to help increase response; however, students’ knowledge of their final course grade has been previously shown to negatively influence SET scores. This case study explored how making SETs available to veterinary students after final exams affected quantitative item scores and response rate when compared to SETs administered during the final weeks of the semester, prior to final examinations. Participants ( n = 262) were randomly assigned to before finals or after finals groups, and 171 students completed 2,926 SETs. Students were more likely to complete evaluations before finals (vs. after), and first-year students completed more SETs than third-year students. Compared to the prior year, in which SETs were administered before finals, students in the study year completed 31% fewer SETs. Timing of SET delivery did not significantly affect SET item scores, but third-year students rated instructors higher than first-year students on five of 10 items. Students’ self-reported expected grade was positively correlated with all 10 SET items for both groups. In this study, timing of completion had no statistical effect on SET item score. However, when students completed SETs after final exams, response rates decreased.

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching

Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions.