Teaching Evaluations Research Articles

Nursing Student Satisfaction with the Teaching Methodology Followed during the COVID-19 Pandemic.

Background: Halfway through the 2019–2020 academic year, the entire university system was affected by an exceptional situation caused by the COVID-19 pandemic. Online learning was globally implemented for all degrees to finish the course and to meet academic objectives. This unforeseen change in teaching and subsequent evaluations meant teachers and students had to invest significant effort. Student satisfaction is used to measure the evaluation of teaching/learning processes in higher education. Our objective was to know and compare the satisfaction of nursing students taught at a Spanish public university after making changes to the teaching methodology. Methods: A descriptive observational study that measures student satisfaction. Study population: 240 students registered in academic years 2019–2020 and 2020–2021 answered the survey. The survey contained 30 items answered on a Likert-type scale. The main variables: the learning methodology (online or blended) was the independent variable; student satisfaction was the dependent variable. Descriptive and bivariate analyses were performed. Results: A response rate between 37.4% and 41.2%. Overall satisfaction was 2.75 points (SD 0.56) and 2.94 points (SD 0.49) with online learning and bimodal learning, respectively (maximum score 4 points) (p < 0.004). Conclusions: Student satisfaction was moderate–high for both learning methodologies. Students found that the b-learning methodology was the most valued.

Improving student participation in SET: effects of increased transparency on the use of student feedback in practice

Student evaluations of teaching (SET) are an influential – and often sole – tool in higher education to determine course and teacher effectiveness. It is therefore problematic that SET results are disturbed by low response rates and response quality. An important factor discussed in prior research to increase SET effectiveness and students’ motivation to participate is transparency about how their feedback is being applied in practice. The current study is the first to empirically test effects of transparency in a quasi-experimental field setting. After students filled in the SET, the intervention group was given a summary of the students’ comments and how the teacher will use these to improve the course. We examined student participation on subsequent course evaluations. In contrast to our expectations, there was no significant improvement in response rates nor response quality between the intervention and control group. Furthermore, perceptions of meaningfulness did not significantly differ between the control and intervention group. This study indicates that more empirical research is needed to define the conditions under which transparency influences student participation. Further implications and recommendations for future research are discussed.

Teaching Evaluations and Student Grades: That’s Not Fair!

University teachers and students are evaluated regularly on their performance, and when evaluations are lower than expected, the feedback may be threatening to the individual, potentially causing deviant behaviors including un-collegiality and poor performance. In this paper, we use the self-threat model of procedural justice to examine faculty responses to teaching evaluations (Study 1) and student responses to course grades (Study 2). The model proposes that group identification influences self-serving bias and self-threat, which then influences procedural justice, and helps explain why teachers and students sometimes criticize decision procedures considered fair by socially accepted standards. Results show full support of the model for faculty responses to evaluations and partial support for student responses to grades. Group identification mitigated self-threat and self-serving bias for the faculty sample but had no influence on the student sample. These findings overall suggest that it is important to reduce the level of self-threat to make negative feedback less threatening to both teachers and students. This may be done either directly via fostering group identification or indirectly by making sure that sensitive performance-based information is not shared to the public.

Applying an interactive learning approach provided by an academic coach in a graduate-level accounting course

PurposeThe paper makes a significant contribution to the accounting education literature by examining the impact of using online meetings with an academic coach on student-perceived learning outcomes, course and instructor evaluations in an online graduate-level accounting course.Design/methodology/approachA quasi-experimental design is adopted by the author to compare the students' evaluations in fall 2018 and in spring 2019 in an online graduate-level accounting course taught by the same instructor and assisted by the same academic coach.FindingsThe study finds that the use of online meetings with an academic coach helps students have a much positive view about the course and a better perception about the instructor's teaching effectiveness for the course. The offering of online meetings by an academic coach also improves students' evaluation about the academic coach in an online graduate-level accounting course.Originality/valueThe paper provides a new perspective to investigate students' perception with the use of online meetings with an academic coach.

The Teaching Evaluation Index System and Intelligent Evaluation Methods of Vocational Undergraduate Pilot Colleges

In recent years, with the popularization of higher education, quality problems have become increasingly prominent, and government documents often propose that the main task of education reform and development is to improve the quality of higher education. As a result, teaching evaluations have appeared one after another. Although various evaluation activities are carried out dynamically, expert evaluation has proven to be one of the most effective methods to ensure professional quality. This will help improve the quality of human resource development, promote regional economic development, improve the quality assurance system of higher education, and promote the formation of professional skills development mechanisms. This article studies the teaching evaluation index system and intelligent evaluation methods of vocational undergraduate pilot colleges, understands the relevant knowledge of the teaching evaluation index system on the basis of literature data, and then, constructs the teaching evaluation system of vocational undergraduate pilot colleges. The constructed system is tested, and the test results show that the error of the results of teacher self-evaluation and student evaluation is controlled within the two, which also verifies that the construction of the teaching ability evaluation index system this time is reasonable and scientific.

Abusive comments in student evaluations of courses and teaching: the attacks women and marginalised academics endure

This paper examines the volume and type of anonymous comments academics receive in student evaluations of courses and teaching (SETs) at the 16,000 higher education institutions that collect this data at the end of each teaching period. Existing research has increasingly pointed to the negative issues of student surveys, but very little research has focused on the volume, type, and impact of anonymous student comments on academics. This paper analyses the survey results of 674 academics to inform higher education leaders and the sector more widely of the amount and type of abusive comments academics are receiving. The work also demonstrates that the highest volume, most derogatory, and most threatening abuse is directed towards women academics and those academics from marginalised groups. The paper finds that previous estimates of the rate and severity of abusive comments that academics receive, and the impact to academics’ wellbeing, mental health, and career progression, have underestimated what is taking place. The paper argues that many universities are failing to protect their staff from this abuse, and the prejudice nature of SET results, which will continue to have a negative impact on the career progression of marginalised academics - a major flaw in a sector that prides itself on diversity and inclusion.

Contradicting findings of gender bias in teaching evaluations: evidence from two experiments in Denmark

A series of studies have found gender bias in student evaluations of teaching. Evidence from several observational studies show that women are evaluated lower than men. These findings are supported by experimental studies aimed at isolating the effect of a possible gender bias from other differences between male and female teachers. In this paper, we conduct two experiments in Denmark to test whether a similar gender bias is present in a national context that is generally considered among the most gender equal. Study one investigates differences in the evaluation of two similar presentations by teachers reported to be either male or female. Study two focuses on the evaluation of teaching material prepared by teachers presented as men and women respectively. The two studies arrive at similar conclusions: There is no gender bias in favor of men in the evaluations made by students. The paper discusses the implications of these findings. Supplemental data for this article is available online at https://doi.org/10.1080/02602938.2022.2048355 .

Gender-Based Linguistic Analysis of Pediatric Clinical Faculty Evaluations.

Gendered stereotypes are embedded in the culture of medicine. Women are stereotypically expected to act collaboratively and less assertively, while men are expected to act with authority and power. Whether gender-biased language is expressed in academic pediatric teaching evaluations is unknown. Determine whether stereotypic gender-based linguistic differences exist in resident evaluations of pediatric faculty. We performed a retrospective cross-sectional study of clinical faculty evaluations by pediatric residents in a single program from July 2016 to June 2019. Using Linguistic Inquiry and Word Count, responses to 2 open-ended questions were analyzed for stereotypic language. Categories were reported as a percent of total words written. Comparisons between gender groups were conducted using nonparametric Wilcoxon rank sum tests. Rates of word use within each category were analyzed using logistic regression where faculty and resident gender were included as predictor variables. A total of 6436 free-text responses from 3218 unique evaluations were included. As hypothesized, evaluations of women faculty were less likely than those of men to include certain agentic language like power (odds ratio [OR] 0.9, P < .001) and insight (OR 0.9, P < .001), and research words (OR 0.6, P=.003). As expected, evaluations of women were more likely to include grindstone words, like "hardworking" (OR 1.2, P=.012). Contrary to our hypothesis, women received fewer teaching words like "mentor" (OR 0.9, P=.048) and communal words like "friendly" (OR 0.6, P=.001). Certain stereotypic language was demonstrated in clinical teaching evaluations of pediatric faculty. These findings should be further examined to improve gender inequities in academic pediatrics.

영상 베트남어 PBL 수업의 학생 성장 과정 분석: 동료･발표 평가와 성찰 일지를 중심으로

In this study, the PBL class was applied to a Vietnamese video class. 13 learners were divided into 3 groups, and it was examined whether all learners grew toward their learning goals, and if so, through what process they grew. When group 1 announces a task, groups 2 and 3 perform peer evaluation and submit a peer evaluation sheet. For this, group 1 categorized the contents of peer evaluation into reflecting, partially reflecting, revised, and not reflecting to prepare the presentation evaluation sheet. Further, they were required to write a reason why “not reflecting.” A reflection log was also submitted. As a result of the above, the assignments were immediately revised, supplemented, and developed every week through peer and instructor evaluation. Through this process, the instructor learned in detail what kind of reflection the presenter and team members were doing each week. In particular, learners achieved ‘up-leveling’ with the activeness of immediately accepting each other’s strengths while conducting peer evaluations on each other’s presentations. This shows that the weekly assignments are improved, while the competencies of both the presentation team and peers are developed at the same time.

Gendered mundanities: gender bias in student evaluations of teaching in political science

ABSTRACT Much research has been undertaken on gender bias in student evaluations of teaching (SETs) in universities, with inconsistent findings. We undertake a qualitative analysis of the comments in four years of SETs in a school of political science and international relations in a highly regarded Australian university. We ask, can the same evaluations produce different results when analysed qualitatively rather than quantitatively? And do students evaluate male-identified and female-identified teachers differently, and if so what are the differences? We show that qualitative analysis can reveal gender bias that is invisible in quantitative analysis. We find that female-identified staff are evaluated more positively than their male counterparts for undertaking time-intensive, stereotypically feminine, emotional labour. Male-identified staff are evaluated more positively for their technical expertise and teaching style. This suggests SETs evaluate gender-stereotypical behaviour rather than only teaching quality, and has significant implications for their use in universities.

Gender Bias Interacts With Instructor Disfluency to Negatively Affect Student Evaluations of Teaching

Recent research has shown that instructor fluency can impact student judgments of learning and instructor ratings but has no real effect on actual learning. In addition, women tend to receive lower course and instructor evaluations than men. In the current study, we examined how instructor fluency and instructor gender influenced instructor evaluations and student learning. Participants watched a short lecture video. The speaker was either male or female and was either fluent (i.e., even paced in their speech) or disfluent (i.e., disorganized, made mistakes). Following the video, participants evaluated the instructor and took a quiz over the lecture. Results indicated that disfluency negatively affected quiz scores, but instructor gender did not. Participants rated the female speaker significantly lower than the male speaker, but only when the speaker was disfluent. These results are explained through the lens of attributional gender bias.

Measuring the motivational climate in an online course: A case study using an online survey tool to promote data-driven decisions

Creating a positive motivational climate in an online course can engage students in their learning. Instructors may be able to better manage their courses and create a positive motivational climate if they implement online survey tools that allow them to assess the motivational climate of their courses. Teachers and researchers have documented that five student perceptions—empowerment, usefulness, success, interest, and caring—are particularly important for creating a positive motivational climate and are associated with students’ engagement and evaluations of teaching. In this paper, we describe a case study of an instructor who used an online survey tool to assess the motivational climate in his online asynchronous course over time. He then used the feedback to consider improvements that he could make to his course in the future. In addition, we describe how this process of using the online survey tool could be used by instructors to transform education.

The influence of ethnicity and language variation on undergraduates' evaluations of Dutch-speaking instructors in Belgium: A contextualized speaker evaluation experiment

This study reports on a contextualized speaker evaluation experiment exploring the effects of language variation (standard vs. colloquial varieties), ethnic identity (Flemish vs. Maghrebi) and wearing a headscarf on students’ evaluation of a female university instructor in Flanders. 314 participants evaluated a single lecture of the instructor on professionalism, comprehensibility, authority, standardness, social attractiveness and physical attractiveness. The results suggest that when measured indirectly, colloquial Dutch is no less acceptable than Standard Dutch in higher education. No general downgrading of the Maghrebi identity was observed. When wearing a headscarf, the standard-speaking Maghrebi instructor was upgraded for professionalism.

An Improved Particle Swarm Optimisation Method for Performance Evaluation of Instructors

Performance evaluation of counselors plays a vital role for education industry (schools, colleges, universities, and vocational colleges). The problem can be stated with the phrases; how to design reasonable and appropriate performance indicators? The objective of this research is to design an effective performance evaluation. The purpose of this study is to explore a new method of performance evaluation that combines strategic goals with personal development goals. The purpose of performance evaluation is to better motivate the enthusiasm of counselors. With the methodology, a new issue faced at modern colleges and universities is being resolved. Therefore, for explaining methodology, this study has carried out the application analysis of the fusion particle swarm algorithm (FPSA) in the performance evaluation of instructors. First, on the basis of comprehensive analysis of performance evaluation, it discusses the advantages and disadvantages of university performance evaluation. Secondly, particle swarm and fuzzy comprehensive evaluation methods are used in the research of instructor performance evaluation. Pass the superiority of this assessment system. Index parameter evaluation is from 2.5 to 3.0. The range indicates an excellent value. In result this improved particle swarm can be compared with the state of the art (Liu et al., 2019). In conclusive remarks, this study is to provide state-of-the-art study for the current research on the topic of instructor performance appraisal in colleges and universities.

The persistence of neoliberal logics in faculty evaluations amidst Covid-19: Recalibrating toward equity.

In this paper, we theorize the intersectional gendered impacts of COVID‐19 on faculty labor, with a particular focus on how institutions of higher education in the United States evaluate faculty labor amidst the COVID‐19 transition and beyond. The pandemic has disrupted faculty research, teaching, and service in differential ways, having larger impacts on women faculty, faculty of color, and caregiving faculty in ways that further reflect the intersections of these groups. Universities have had to reconsider how evaluation occurs, given the impact of these disruptions on faculty careers. Through a case study of university pandemic responses in the United States, we summarize key components of how colleges and universities shifted evaluations of faculty labor in response to COVID‐19, including suspending teaching evaluations, implementing tenure delays, and allowing for impact statements in faculty reviews. While most institutional responses recenter neoliberal principles of the ideal academic worker that is both gendered and racialized, a few universities have taken more innovative approaches to better attend to equity concerns. We conclude by suggesting a recalibration of the faculty evaluation system – one that maintains systematic faculty reviews and allows for academic freedom, but requires universities to take a more contextualized approach to evaluation in ways that center equity and inclusion for women faculty and faculty of color for the long term.

Who Can Debriefing Assessment for Simulation in Healthcare? Different Perspectives on Teledebriefing Assessments.

Virtual debriefing is a cardinal element to achieve the effectiveness of telesimulation. There are different instruments to assess face-to-face debriefing to determine the degree of effectiveness of debriefing; Debriefing Assessment for Simulation in Healthcare (DASH) instrument is one of them. This study aims to describe and compare the evaluation of raters, instructors, and students during a virtual debriefing using the DASH. A cross-sectional study was performed evaluating the virtual debriefing of 30 instructors after a heart failure telesimulation scenario. The evaluation was conducted by 30 instructors, 338 undergraduate students in the seventh semester, and 7 simulation raters. The 3 versions of the DASH instrument in Spanish were applied, respectively. Two comparisons were made, student versus instructor and rater versus instructor. Cronbach α was 0.97 for each version. The averages of the results on the DASH instrument were: 6.61 (3.34-7.0), 5.95 (4.65-7.0), and 4.84 (2.68-6.02) for student, rater, and instructor versions, respectively. The size effect between student and debriefer perspectives was 0.42. In contrast, the size effect between instructor and rater was 0.72. All differences were significant. There are different rates between the persons who use the DASH. In this study, from the perspective of the instructor and rater, the difference was 1 point with a wide range, in contrast with the difference between instructor and student, which is relatively minor. It is necessary to consider the perspectives of experts in the subject to achieve a virtual debriefing of high quality and improve a debriefing by using the DASH.

Is my opinion important in evaluating lecturers? Students’ perceptions of student evaluations of teaching (SET) and their relationship to SET scores

ABSTRACT Previous studies have identified a positive relationship between students’ perceptions of student evaluations of teaching (SET) and the grades that students provide in SET, controlling for other bias factors. The research by Spooren and Christiaens in 2017 at the University of Antwerp supported this finding. In this study, the methodology used by Spooren and Christiaens was replicated at the Technological Indoamerica University in Ecuador, in a close conceptual replication. In the replicated study, 967 undergraduate participants answered the questionnaires used by the original authors. The replication study sample was very similar in size, seniority, and gender to the original study but not in academic disciplines studied. Most of the students agreed that the evaluation was relevant and could improve teaching practices. Results show a statistically significant but small positive relation among perceptions of SET and SET scores (0.20 for the Belgian university and 0.27 for the Ecuadorian university).

Impact of Bedside Rounding on Attending Teaching Evaluations.

To examine associations between bedside rounding (BSR) and other rounding strategies (ORS) with resident evaluations of teaching attendings and self-reported attending characteristics. Faculty from three academic medical centers who attended resident teaching services for ≥4 weeks during the 2018-2019 academic year were invited to complete a survey about personal and rounding characteristics. The survey instrument was iteratively developed to assess rounding strategy as well as factors that could affect choosing one rounding strategy over another. Survey results and teaching evaluation scores were linked, then deidentified and analyzed in aggregate. Included evaluation items assessed resident perceptions of autonomy, time management, professionalism, and teaching effectiveness, as well as a composite score (the numeric average of each attending's scores for all of the items at his or her institution). BSR was defined as spending >50% of rounding time in patients' rooms with the team. Hallway rounding and conference room rounding were combined into the ORS category and defined as >50% of rounding time in these settings. All of the scores were normalized to a 10-point scale to allow aggregation across sites. A total of 105 attendings were invited to participate, and 65 (62%) completed the survey. None of the resident evaluation scores significantly differed based on rounding strategy. Composite scores were similar for BSR and ORS (difference of <0.1 on a 10-point scale). Spearman correlation coefficients identified no statistically significant correlation between rounding strategy and evaluation scores. An exploratory analysis of variance model identified no single factor that was significantly associated with composite teaching scores (P > 0.45 for all) or the domains of teaching efficacy, professionalism, or autonomy (P > 0.13 for all). Having a formal educational role was significantly associated with better evaluation scores for time management, and the number of lectures delivered per year approached statistical significance for the same domain. Conducting BSR did not significantly affect resident evaluations of teaching attendings. Resident perception of teaching effectiveness based on rounding strategy should be neither a motivator nor a barrier to widespread institution of BSR.

We can work it out: faculty interpretation of student evaluations of teaching

ABSTRACT Despite previous literature strongly criticising the use of student evaluations of teaching (SETs), it is unlikely that the frequency of SETs will be decreased for university instructors. However, little attention has been given to how we can make SETs work as a tool for improving teaching effectiveness. This research explores how faculty members of a large professional master’s programme in Brazil interpret the results of their SETs. The findings show that qualitative questions combined with quantitative scores are effective in improving classes. It also shows that faculty critically assess students’ feedback by disregarding emotional positive and negative comments and looking for suggestions in comments. Instructors with lower scores have difficulty interpreting feedback and tend to attribute their performance to nonresponse bias. This research contributes to the SET literature by showing that a combination of qualitative and quantitative questions can be effective in improving teaching quality but that some instructors may need support in interpreting SET results to change their teaching.

The biasing impact of positive instructor reputation on student evaluations of teaching

A naturally-occurring intervention in a longitudinal field setting (4 months) was used to examine the presence and biasing impact of a positive reputation on subsequent ratings of work performance (student evaluations of teaching). During pre-semester interactions, first-year MBA students received information from second-year MBAs about their upcoming professors and classes. Favorable information about the two professors and course examined in the present study caused a positive reputation. Results indicated that despite four months of experiencing actual performance, the positive reputation hindered students’ decision-making process resulting in biasedly inflated ratings of instructor performance and halo error judgments of course materials, grading, and amount learned. The problematic implications of using biased student evaluations of teaching to measure faculty performance is discussed, along with suggestions of ways to mitigate against overreliance on this evaluation method and to possibly minimize reputational effects.