Student Evaluations Of Teaching Research Articles

Institutional Approaches to Evaluate Teaching Effectiveness: The Role of Summative Peer Review of Teaching for Promotion and Tenure

A growing body of literature has identified student evaluations of teaching (SETs) as introducing bias against minority faculty members and not serving as a reliable or valid measure of teaching effectiveness. This lack of reliability and validity presents issues for university tenure and promotion committees, as these institutional processes necessarily require accurate, objective, and holistically informed modes of evaluation to recognize teaching achievements. Summative peer review of teaching (SPRT) is an alternative mode of assessment that aims to provide evidence of teaching effectiveness to inform promotion and tenure. SPRT, as an institutional practice, has been adopted at a small cohort of institutions of higher education, marking a potential shift in practice. This article examines SETs to articulate the problematic elements introduced by SETs, specifically to examine if SPRT can serve as a viable alternative. By describing the SPRT processes that four institutions have taken, the authors aim to articulate these emerging approaches to collecting evidence of teaching effectiveness. In this descriptive work, it is our secondary contention that SPRT, through intentional design and facilitation, can offer a process that does not introduce bias in the same way as SETs and thus, can also be used to satisfy the growing need for practices that help achieve, in part, institutional goals related to equity, diversity, and inclusion (EDI).

Impact of social media distraction on student evaluation of teacher effectiveness

PurposeThe study examines the impact of social media distraction on the relationship between student engagement and student evaluations of teachers (SET) in management education, part of the STEM disciplines. While it is widely accepted that student engagement and evaluation of teachers are two important constructs of active learning systems, their links, especially in the context of management education, have not been explored. This study seeks to fill this research gap as teachers' appraisal and career growth greatly depend on SET across higher education institutes worldwide.Design/methodology/approachData were collected from 247 student participants from higher education institutes/universities using a questionnaire-based survey. Further, 15 students were interviewed to glean insights from their feelings regarding social media distractions and how it impacts their engagement and evaluation of teachers. For the quantitative and qualitative analyses, multiple regression and thematic analysis were applied, respectively. A word tree was developed to determine the association between experiences and contexts.FindingsThe results indicate that social media distraction moderates the relationship between students' cognitive engagement and their evaluation of teaching faculty. Social media distractions disrupt student engagement inside the classroom, leading to wrong/biased assessments of teachers' performance. Additionally, cognitive and emotional engagement significantly impacted student evaluations of the management college faculty.Originality/valueThe authors’ findings enable educators in higher education systems, specifically management education, to rethink the faculty feedback system and social media distractions impeding student engagement other than SET drawbacks.

The Impact of COVID: Case Study of an Academic English Reading Course at Qassim University

This paper contributes to the burgeoning field of research investigating the educational effects of COVID. A questionnaire with both open and closed questions was administered to female takers of an Academic English Reading course at a Saudi university, eliciting responses about aspects of the quality of delivery of that course before and after the onset of COVID. From this quantitative and qualitative student evaluation of traditional class teaching (TCT) versus emergency remote teaching (ERT), some findings emerged that echoed those in the few comparable studies in the West. For example, certain features like the greater autonomy of the student in ERT than TCT were liked by some students but not others, while features such as the superiority of teacher-student communication in TCT were more uniformly approved. Furthermore, predictions from the literature such as that favouring TCT would be associated with greater self-efficacy, or a greater perceived degree of transactional interaction between student and teacher were not supported in the context of this study. Analysis of suggestions for ERT improvement leads to the conclusion that, at present, it may be that students need training to exploit the existing ERT, rather than that the ERT to be altered.
 
 Received: 28 September 2022 / Accepted: 30 December 2022 / Published: 5 January 2023

Exploring students' willingness to provide feedback: A mixed methods research on end-of-term student evaluations of teaching

Student Evaluations of Teaching (SET) are one of the most consistently administered tools to assess teaching performance in higher education institutions. SET affect the careers of individuals (summative evaluation), and have potential to shape the quality of instruction (formative evaluation). Past studies have addressed several issues with SET, but few have focused on surveying and interviewing students to better understand how they navigate and complete these evaluations. Therefore, a mixed methods design was used to explore university students' willingness to provide feedback through SET as part of the teaching evaluation process. Results indicate students’ positive views about the evaluation process and their perception of usefulness of evaluations increased their willingness to provide feedback, whereas potential student biases decreased their willingness to provide feedback. More importantly, results also highlight students are still not aware of, and do not really understand, the implications of their SET responses.

A Text Mining and Statistical Approach for Assessment of Pedagogical Impact of Students’ Evaluation of Teaching and Learning Outcome in Education

Technology-enhanced learning (TEL) is now at the heart of teaching and learning process in many higher education institutions (HEIs). Today, educators are faced with the challenges of pedagogically specifying what tools, methods, and technologies are used to support the teachers and students, and to help maintain/sustain a continuous education and practices. This study shows that there is an opportunity in the use of (educational) datasets derived about the teaching and learning processes to provide insights for fostering the education process. To this effect, it analyzed the students’ evaluation of teaching (SET) dataset ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n=471968$ </tex-math></inline-formula> ) collected within a higher education setting to determine prominent factors that influences the students’ performance or the way (TEL-based) education is being delivered, including its didactical impact and implications for practice. Theoretically, the study employed a mixed methodology grounded on integration of the Data-structure approach and Descriptive decision theory to study the rationality behind the students’ evaluation of the teaching and performance. This was done through the Textual data quantification (qualitative) and Statistical (quantitative) analysis. Qualitatively, the study applied the Educational Process and Data Mining (EPDM) model (a text mining method) to extract the different sentiments and emotional valence expressed by the students in the SET, and how those characteristically differ based on the period and type of evaluation they have completed (between 2019 to 2021). For the quantitative analysis, the study used a multivariate analysis of covariance (MANCOVA) and multiple pairwise comparisons post-hoc tests to analyze the quantified information (average sentiment and emotional valence) extracted from the SET data to determine the marginal means of effect the different SET types and evaluation period have on the students’ learning outcomes/perception about the teaching-learning process. In addition, the study empirically discussed and shed light on the implications of the main findings for TEL-based Education, particularly implemented by the HEI during the analyzed periods. The scholastic indicator from the study shows that while the flexible digital models or instructional methods are effective for continuous education, innovative pedagogies, and teaching transformations. It also, on the other hand, serve as an incentive for more robust research that idiosyncratically look into their implications for the students’ learning outcomes and assessment done in this study.

Secondary school mathematics teaching evaluations by students: A report card for the mathematics teacher

This study presents findings from multiple case studies undertaken in three secondary schools in South Africa’s Limpopo Province. The goal was to collect input from students on mathematics teaching practices in their classrooms. The study included a self-selected sample of eleven grade 12 students. Unstructured individual interviews and students’ written reports were used to collect data, which was then analyzed based on emerging themes. Students expressed genuine concerns about teachers’ lesson preparation, subject and pedagogical knowledge, classroom management, attention given to slow learners, quality of classwork and homework tasks given to students, exam preparation, class attendance, and utilization of contact time, teachers’ attitudes, and their emotional intelligence, among others. Based on the study’s findings, the author suggests that student evaluation of teaching be used in secondary schools to help teachers reflect on their teaching practices in order to create learning environments that most students would enjoy.

TEACHING EVALUATIONS AS A SITE OF INSTITUTIONAL INTERVENTION TO IMPROVE CULTURE

Student evaluations of teaching are a controversial medium; there's some meaningful information and some bias encoded in the data. Many institutions have recognized this and adjusted in terms of how quantitative information is used in promotion and retention decisions, but they are (mostly) ignoring the harms that student evaluations can have on instructors. The anonymous open-ended feedback students provide is specifically a place that enables trolling. Trolling has negative effects on instructors, especially instructors of color and women in male-coded disciplines. There are some things institutions can do about it.

Automating Identification of Toxic Comments in Student Evaluations of Teaching

Automating Identification of Toxic Comments in Student Evaluations of Teaching

Student evaluation of teaching and student centeredness in the Humboldtian and Emirati tribal traditions of higher education

ABSTRACT This study is the result of interviewing administrators and academics as well as conducting content analysis of student evaluation of teaching (SET) forms in 14 universities in Germany and the United Arab Emirates (UAE). The aim was to establish whether and if so, how, this procedure differs in two highly hierarchical social and academic contexts. We found that SETs have been adopted by all the universities we visited between 2001 and 2013, for some courses in Germany and for all of those in the United Arab Emirates. National and regional higher education laws and accreditation procedures act as coercive and normative mechanisms that promote SET’s implementation in Germany and the UAE, respectively. It emerged that SETs are commonly used for monitoring in Germany and the UAE, in private Emirati and German universities for checking the continuity of contracts and, in private German ones, to report students, while Emirati students do not receive reports, despite their treatment as clients. Further, we elicited that SETs reproduce student-centeredness in all the visited universities in both countries, albeit to varying degrees. We conclude that SETs demonstrate the strength of rationalizing trends in higher education (HE). Moreover, nuances in the implementation of SET, which we relate to the protection of professors’ autonomy in Germany and the cultural norm of respecting faculty as a highly regarded position in the UAE, remain related to the Humboldtian and tribal heritage of our studied countries, respectively.

In the eye of the beholder – how course delivery affects anatomy education

BackgroundThe COVID-19 pandemic caused major shifts in students’ learning strategies as well as teaching environments that profoundly affected the delivery of anatomy courses in medical schools. The Department of Anatomy at the University of Zagreb School of Medicine had a unique experience where the anatomy course in 2019/2020 was first taught in-person before transferring to an online course delivery, while the inverse happened in 2020/2021. The core curriculum, course material and examination criteria were the same in both academic years. The aim of the study was to determine whether course delivery affected students’ perceptions of the course and whether it impacted students’ engagement and success. MethodsThe students’ perceptions of the course were assessed via an anonymous course survey (student evaluation of teaching, SET). The questions in the SET assessed the usefulness of teaching modalities rather than students’ satisfaction. Most questions were in the form of statements to which students responded with their level of agreement on a five-point Likert scale. Differences between responses in 2019/2020 and 2020/2021 were analyzed using the Mann-Whitney test. Effect size was estimated using Cliff’s delta and association between responses was assessed using Spearman’s r coefficient. ResultsStudents’ perceptions were significantly affected by changes in course delivery. Students’ success and engagement were higher in 2019/2020 when in-person teaching preceded online teaching. Furthermore, students’ views on course organization and the usefulness of continuous assessment were more positive in 2019/2020. Finally, students’ perceptions of the usefulness of online materials and activities were more positive in 2019/2020. All comparisons between the two academic years were statistically significant (P ≤ 0.0001 for all comparisons, Mann-Whitney test). ConclusionsStudents’ perceptions of the anatomy course were dependent on the teaching environment they were exposed to at the beginning of the course. A transfer from in-person to online course delivery was more successful than vice-versa. This has important implications for structuring hybrid courses in medical education in the future.

Online and active learning design in large-size classes: a phenomenological study

Abstract This paper presents a qualitative–quantitative study that investigates whether an active and collaborative design can help university students positively evaluate the course, even in a large-size online class for special ed teachers. The analysis of the collected data (a fact-finding questionnaire created on purpose; a standard questionnaire for the course evaluation; field notes, design documentation; a class climate analysis questionnaire; an individual opinion poll about the educational activities; in-chat reflections of trainees) clearly testifies that trainees greatly appreciated the active and collaborative design. Some of the students hoped to be able to repeat a similar instructional design in their own classes, even though they underlined that this type of design often requires much more commitment, for the student and the teacher, than what is foreseen by the recognised credits. The results that can be achieved, in terms of Students’ Evaluation of Teaching (SET), are relevant if the teacher is properly trained to the introduction of pedagogical innovation, as also recalled by the European Union.

Using deepfakes for experiments in the social sciences - A pilot study.

The advent of deepfakes - the manipulation of audio records, images and videos based on deep learning techniques - has important implications for science and society. Current studies focus primarily on the detection and dangers of deepfakes. In contrast, less attention is paid to the potential of this technology for substantive research - particularly as an approach for controlled experimental manipulations in the social sciences. In this paper, we aim to fill this research gap and argue that deepfakes can be a valuable tool for conducting social science experiments. To demonstrate some of the potentials and pitfalls of deepfakes, we conducted a pilot study on the effects of physical attractiveness on student evaluations of teachers. To this end, we created a deepfake video varying the physical attractiveness of the instructor as compared to the original video and asked students to rate the presentation and instructor. First, our results show that social scientists without special knowledge in computer science can successfully create a credible deepfake within reasonable time. Student ratings of the quality of the two videos were comparable and students did not detect the deepfake. Second, we use deepfakes to examine a substantive research question: whether there are differences in the ratings of a physically more and a physically less attractive instructor. Our suggestive evidence points toward a beauty penalty. Thus, our study supports the idea that deepfakes can be used to introduce systematic variations into experiments while offering a high degree of experimental control. Finally, we discuss the feasibility of deepfakes as an experimental manipulation and the ethical challenges of using deepfakes in experiments.

Biases in student evaluations of teaching: An exploratory analysis to a Chinese case study

ABSTRACT This paper examines factors influencing on student evaluations of teaching (SET) with consideration of disciplinary variations. Data were obtained from a university in Guangzhou of China and consisted of 1443 undergraduate course ratings collected from 19 schools over two semesters. The students and courses are classified into hard and soft disciplinary groupings and divided into four sub-groups for further comparison. The result of the ANOVA test showed significant difference in the mean SET scores across four subgroups; and results of linear models reflect students who major in hard discipline tend to give higher SET scores regardless of the course’s feature. In addition, there was also evidence of possible biases in the ratings according to the features of teachers, students and courses. Overall, the disciplinary differences would need to be considered when decisions are being made on the basis of the SET scores, especially for cross-school evaluation.

Educational model transition: Student evaluation of teaching amid the COVID-19 pandemic

The Education 4.0 Framework calls for Higher Education Institutions (HEIs) to innovate their curriculum for developing the competencies of the future. Tecnologico de Monterrey started a transition from an active-learning educational model to Tec21, a challenge-based learning educational model focused on competency development. After one semester of this transition, the COVID-19 pandemic disrupted education worldwide, causing most universities to adapt to online education. We found the opportunity to analyze the institutional Student Evaluation of Teaching (SET) survey at different stages of the COVID-19 pandemic, prior to the outbreak, in the transition to online learning, and after the transition to fully online course delivery. We performed this analysis separately for the two coexisting educational models and each of the schools at the university. We also compared the SET scores for the spring semester of 2021, when the two educational models had a comparable number of students. We found that SET scores were not negatively impacted by the COVID-19 pandemic hinting toward positive implications about the institutional response to the pandemic. Another finding is that the Tec21 educational model has received higher SET scores, which implies a positive perception by students. There were a few exceptions to these results, which we address explicitly; for instance, the COVID-19 pandemic might have affected SET scores in the School of Medicine and Health Sciences. Further research is necessary to evaluate the implementation of the Tec21 model comprehensively.

Nothing nice about it: Critiquing Midwest nice in teacher education

In teacher education, critical scholars have lamented how “niceness” hinders progress toward social and racial justice. A place characteristic of this “niceness” is the Midwestern region of the United States, which the dominant narrative paints as overly agreeable and free of racial inequities. This image overlooks the rampant systemic racism that is foundational to the entire country, allowing the Midwest to tout an ideological stance of Midwest nice—a race-evasive semblance of social and political politeness that is seemingly harmless. This conceptual article draws on critical race theory and critical geographies of race to analyze how Midwest nice influences Midwestern teacher education programs. By conceptualizing two teacher education sites—educator praxis (an input) and student evaluations of teaching (an output)—we consider the particular plight of Women of Color critical scholars instructing preservice teachers in the Midwest. We subsequently explore what this critique of Midwest nice means for the field, arguing that teacher education programs in the Midwest perpetuate racial inequity and violence through Midwest nice. Ultimately, we urge a teacher education program to institutionally support efforts that expose and disrupt Midwest nice’s preservation of social and racial injustice.

Setting out SET: a situational mapping of student evaluation of teaching in Australian higher education

The student evaluation of teaching (SET) in higher education has become an increasingly complex and subjectively contested area. From a singular purpose in seeking information to improve teaching in the 1920s, evaluation has now expanded to encompass administrative and regulatory purposes. Currently, evaluation impacts on personal and institutional reputation and is frequently used as a benchmark in determining and shaping individual academic careers. The value and purpose of evaluation is open to ongoing debate, as is the notion of transparency regarding who should have access to evaluation data (quantitative scores and/or free text comments). This paper presents the outcome of a situational mapping we conducted to better understand student evaluations of teaching in Australian higher education. We identified the component actors, actants and elements of the setting and derived a list of the discursive constructions which drive the relationships between them. To test the efficacy of our mapping in terms of isolating situations within the broader setting, we describe three hypothetical case studies: making student evaluations public, closing the loop and academic surveillance.

Evaluation Method of Innovative Education Model of E-Commerce Video Live Broadcast Based on Big Data Analysis Technology

In the environment of education big data, how colleges and universities make good use of these data not only affects the orderly operation of the whole education and teaching system of colleges and universities. It will also become an inexhaustible driving force to help colleges and universities promote the reform and innovation of the education and teaching system. This paper takes student evaluation data and student online learning data as research objects. Focusing on the teaching operation and students’ autonomous learning, this paper uses the improved k-mode algorithm to cluster analyze the classroom teaching operation. This paper uses neural network algorithm based on machine learning to predict and compare students’ online course learning. It is hoped that it can provide meaningful reference for the construction of teaching management system and the reform and innovation of teaching management system in colleges and universities. Two research works are mainly carried out through the preliminary analysis and transformation of the data of student evaluation of teaching in a certain university. The improved cosine dissimilarity algorithm is used to eliminate the abnormal teaching evaluation data. The normalization method was used to standardize the teaching evaluation data. The traditional k-mode algorithm is used to cluster the teaching evaluation data. Some problems of k-mode algorithm are pointed out, and the traditional k-mode algorithm is improved. Experimental results show that the improved algorithm is more reasonable and effective.

Playing the SET game: how teachers view the impact of student evaluation on the experience of teaching and learning

Student evaluation of teaching (SET) has become a ubiquitous feature of higher education. The attainment and maintenance of positive SET is essential for most teaching staff to obtain and maintain tenure. It is not uncommon for teachers to receive offensive and non-constructive commentary unrelated to teaching quality. Regular exposure to SET contributes to stress and adversely impacts mental health and well-being. We surveyed Australian teaching academics in 2021, and in this paper, we explore the perceived impacts of SET on the teaching and learning experience, academic standards and quality. Many respondents perceived that SET contributes to an erosion of standards and inflation of grades. A thematic analysis of open-ended questions revealed potential mechanisms for these impacts. These include enabling a culture of incivility, elevating stress and anxiety in teaching staff, and pressure to change approaches to teaching and assessment to achieve the highest scores. Playing the SET game involves balancing a commitment to quality and standards with concessions to ensure optimal student satisfaction. Anonymous SET is overvalued, erodes standards and contributes to incivility. The process of SET needs urgent reform.

How does grade inflation affect student evaluation of teaching?

Student evaluation of teaching (SET) is important for assessing university instructors’ performance. However, this system seems biased as students’ grade expectations result in rewards or penalties in SET. As a fair evaluation of grades became difficult during the COVID-19 pandemic, universities implemented a relaxed grade policy that expanded the distribution of high grades. This grade inflation altered students’ expected grade. Through empirical analysis, this study examined the change in the relationship between bias and SET due to grade inflation. A top-ranking South Korean university provided 125,003 cases of SET data in 2019 and 2020 for the analysis. Grade inflation diminished the biasing effect on SET, mainly in terms of reward. Furthermore, the group with the lowest grade point average (GPA) showed the highest decrease in rewards, and the group with the highest GPA showed maximum decrease in punishment. This finding implies that a change in expected grades due to factors other than lectures may alter students’ attitudes toward SET, and grade expectations may play a key role in reducing bias in SET.

Teachers Can Make a Difference in Bullying: Effects of Teacher Interventions on Students’ Adoption of Bully, Victim, Bully-Victim or Defender Roles across Time

School bullying is a serious problem worldwide, but little is known about how teacher interventions influence the adoption of bullying-related student roles. This study surveyed 750 early adolescents (50.5% female; average age: 12.9 years, SD = 0.4) from 39 classrooms in two waves, six months apart. Peer ratings of classmates were used to categorize students to five different bullying-related roles (criterion: >1 SD): bully, victim, bully-victim, defender, and non-participant. Student ratings of teachers were used to obtain class-level measures of teacher interventions: non-intervention, disciplinary sanctions, group discussion, and mediation/victim support. Controlling for student- and class-level background variables, two multilevel multinomial logistic regression analyses were computed to predict students’ bullying-related roles at wave 2. In the static model, predictors were teacher interventions at wave 1, and in the dynamic model, predictors were teacher intervention changes across time. The static model showed that disciplinary sanctions reduced the likelihood of being a bully or victim, and group discussion raised the likelihood of being a defender. Mediation/victim support raised the likelihood of being a bully. The dynamic model complemented these results by indicating that increases in group discussion across time raised the likelihood of being a defender, whereas increases in non-intervention across time raised the likelihood of being a victim and reduced the likelihood of being a defender. These results show that teacher interventions have distinct effects on students’ adoption of bullying-related roles and could help to better target intervention strategies. The findings carry practical implications for the professional training of prospective and current teachers.