Student Evaluations Of Teaching Research Articles

A national student survey for the Italian higher education system

Despite the strong criticisms, mass student surveys play an important role in the quality assurance process of national higher education systems. While some national higher education systems have a long history of using student surveys to inform teaching and course quality improvement (e.g. the UK, Australia), in Italy, where the quality assurance system has been put in place at both state and institutional levels, a national student survey has never been implemented. As a consequence, higher education institutions have used, and still use, only student evaluations of teaching. If, on the one hand, these local surveys tend to overlap with the rationale and structure of a national survey, on the other hand, they prevent universities from comparing quality indicators. Given the strong drive to use student surveys as a quality assurance mechanism, the present article reports a study aimed to design and probe a new questionnaire to be used at the national level. A total of 572 final year students enrolled in a public university participated in the study. Data were examined using a principal component analysis. Study results could set the groundwork for a critical debate on changes and improvements in the quality assurance process.

STUDENTS’ PERCEPTIONS TOWARDS THE IMPLEMENTATION OF ROBERT FROST’ POETRY IN WRITING CLASS AT MUSAMUS UNIVERSITY

This study investigated the students’ perceptions concerning the use of Robert Frost’s poetry in writing class at English Literature Department of Musamus University which was obtained through a survey. A total of 17 undergraduate students taking writing class participated in this study. The participants responded to a questionnaire and an open-ended questions concerning the two focal points: (1) how the students perceived the use of Robert Frost’s poetry in teaching writing, (2) the implementation of Frost’s poetry in improving students’ writing short story. The results of the study revealed that the implementation of Frost’s poetry helped them to be easier in writing short story. Most of the participants gave positive response to the use of Frost’s poetry in teaching them to write a short story. In addition, they seemed to prefer learning writing short story by using English poetry in writing classes. The concluding discussion addresses suggestion about the need to consider students’ wants and needs by gauging their perceptions as the student evaluation of teaching in order to keep up the better improvement to the teaching writing the texts and the using of authentic material or media in English Literature Department at Musamus University.

The Effect of Gender Inequities in the Classroom and Beyond in U.S. Business Schools

When women teach management in U.S. business schools they are likely to experience more gender inequities than men. In this essay we examine three dimensions of management teaching where gender inequities are likely to occur: (1) student-faculty interactions; (2) student evaluations of teaching; and (3) interactions between faculty peers. The types of inequities experienced by women when they teach include feeling social pressure to submit to ad hoc student demands for personal favors and emotional support that infringe on a professor’s time; having their teaching performance judged from student evaluations of teaching that are subject to gender bias; and experiencing lower levels of organizational inclusion compared to their male colleagues. We utilize theoretical logic from social role theory, relational practice, and perceived organizational inclusion frameworks supported by the research literature to provide greater insight as to why women are likely to experience more adversity when they teach management in business schools. After we explain the basis for gender inequities we prescribe action steps that should improve equity for all faculty who teach management. Finally, we provide a call-to-action for business school administrators to implement action steps to make an equitable teaching culture a reality.

Teaching about vulnerability and love in social work: lessons and reflections from two academics

ABSTRACT Teaching vulnerably, and doing so in a way that ‘talks back’ to the dominant ‘banking model’ and the neoliberal forces that have entered the classroom where everything is evaluated through flawed metrics such as Student Evaluations of Teaching (SETs), is to take an enormous personal and professional risk. This paper is written as a critical collaborative autoethnographic reflective dialogue between two social work academics who embody diametrically opposed academic and life experiences. Contextualised within the space and place of the social work classroom, we offer stories and dialogue of our theoretical reflections of engaging with the concepts of love and vulnerability in our classrooms. Our narratives weave together an embodied vulnerability pedagogy, our processes of learning and our arrival at this place where we interrogate how vulnerability is taught, understood and experienced in social work education as part of our commitment to produce decolonised ways of teaching in social work.

Contribution of open-ended questions in student evaluation of teaching

ABSTRACT Student evaluation of teaching (SET) is applied in the vast majority of universities and higher education institutions. They are used to design professor training programs, evaluate teaching performance, and show evidence of performance to different stakeholders. SET surveys typically include an open-ended question which is not always considered in the analysis of the results. This study aims to show the contribution of analyzing the students’ comments by means of the Latent Dirichlet Allocation methodology to factor them into the analysis of the quantitative part of the survey. For this purpose, a sample of 737 courses taught during 2017 and 2018 in an undergraduate program at a Chilean university is used. The results show that both the number of comments and the topics that can be extracted from them contribute significantly to the analysis of the professors’ teaching performance. The topics extracted are more specific than the quantitative dimensions of the survey, which allows obtaining very concrete feedback for professors and for designing training programs. Around half of the topics extracted are actionable and do not depend on the intrinsic characteristics of the professors, which allows for effective improvements in teaching. Additionally, the extracted topics can be grouped into dimensions that have a correspondence with the quantitative dimensions of the survey, although they only cover a subset of the latter. This result provides insights to improve the survey design and adjust the weighting of its different dimensions.

COVID-19 Effects on Communication Course and Faculty Evaluations

Student evaluations of teaching (SETs) are utilized by universities as one component in assessing course effectiveness, despite evidence in the research regarding their validity. With the global COVID-19 pandemic, many universities rapidly transitioned teaching modalities from face-to-face to online learning, regardless of the faculty experience. This study investigates the effects on SETs of the rapid transition in teaching modalities for all sections of courses occurring during COVID-19 compared with all sections of courses taught within a Communication department at a large public research university over the past 8 years. The results indicate moderate effects from the rapid transition to online learning.

Does Course Specific Nudging Enhance Student Engagement, Experience and Success?: A Data-Driven Longitudinal Tale

Low levels of online student engagement impact negatively on student success and adversely affect attrition. Course learning analytics data (CLAD), combined with nudging initiatives, have emerged as strategies for engaging online students. This paper presents a mixed method case study involving a staged intervention strategy focussing on the employment of timely, strategic communication interventions conducted across 19 courses and six disciplines. The research methodology utilised CLAD, online surveys, student interviews and student evaluations of teaching. The findings substantiate benefits for both academics and students. Academics benefitted from the provision of a relatively simple, accessible and proactive intervention for increasing students’ capacities to be more in control and engaged in their learning. Students benefitted as the intervention accentuated critical resources to assist them to better address assessment requirements, align their expectations more realistically with those of the course, and more readily demonstrate their learning obligations and responsibilities.

The wolf in sheep's clothing.

Many universities and business schools have introduced a specific form of performance feedback: Student Evaluation of Teaching (SET). SET increasingly take place in a digitized and anonymous form, thereby reversing the power relations between professors and students in a form that would never exist in face-to-face interactions. SET is a double-edged sword. While SET are relied on for administrative decisions, researchers question their value, validity and reliability. Yet despite the extensive research on the student perspective of SET, we know little about professors´ mental reactions to the receipt of negative SET. In the present work, we identify different categories of negative SET, test their frequency and investigate their influence on professors´ perceived levels of stress. We also explore the mediating effect of interpersonal justice. Our theoretical and empirical findings highlight that negative SET can be operationalized in a 2x2 matrix comprising active disrespect regarding the person/course and matter-of-fact feedback concerning person/course. All SET categories targeting the person lead to increased stress in professors, which is significantly positively mediated by the perception of fairness. By examining this relationship, we offer important limitations for the use of SET at universities as tool to motivate professors to improve their teaching performance.

We Can Work it Out: Faculty Interpretation of Student Evaluations of Teaching

Despite previous literature strongly criticizing the use of student evaluations of teaching (SETs), it is unlikely that the frequency of SETs will be decreased for university instructors. However, little attention has been given to how we can make SETs work as a tool for improving teaching effectiveness. This research explores how faculty members of a large professional master’s program in Brazil interpret the summative and formative results of their SETs. The findings show that formative comments combined with summative scores are effective in improving classes. It also shows that faculty critically assess students’ feedback by disregarding emotional positive and negative comments and looking for suggestions in comments. Less effective instructors have difficulty interpreting feedback and tend to attribute their performance to nonresponse bias. This research contributes to the SET literature by showing that a combination of formative and summative questions can be effective in improving teaching quality but that some instructors may need support in interpreting SET results to change their teaching.

Enhancing the Teaching Environment: 3-Year Follow-Up of a Resident-Led Residents-as-Teachers Program.

Previous faculty-driven residents-as-teachers (RAT) models have had limited efficacy and sustainability. To evaluate the acceptability and effects of a resident-led RAT program on resident teaching. In October 2016, obstetrics and gynecology (OB/GYN) residents at a large academic institution implemented a resident-led RAT program, consisting of a steering committee of peer-selected residents with 2 faculty mentors who planned education-focused resident didactics and journal clubs, organized resident involvement in clerkship activities, and recognized residents who excelled in teaching as Distinguished Educators (DEs). From July 2016 through June 2019, using the Kirkpatrick Model, we evaluated the program with annual resident surveys assessing self-perception of 13 teaching skills (5-point Likert scale) and value of RAT program, institutional end-of-clerkship student evaluations of resident teaching, and resident participation in DE award. Annual resident survey response rates ranged from 63% to 88%. Residents' self-reported teaching skills improved significantly in 11 of 13 domains from 2016 to 2018 (improvements ranging from 0.87-1.42; 5-point Likert scale; P < .05). Of the 2018 respondents, 80% agreed that the resident-led RAT program added value to the residency. For 2017-2018 and 2018-2019 academic years, 47% and 48% of medical students (100% response rate) strongly agreed that residents provided effective teaching compared to 30% in 2016-2017 (P < .05). Ten residents have graduated as DEs during this time period. A resident-led RAT program increased residents' self-reported teaching skills, improved medical student perceptions of teaching quality, and was sustainable and acceptable over a 3-year period.

Improving the Assessment of Teaching Effectiveness With the Nonequivalent Dependent Variables Approach

Introduction: Assessing teaching effectiveness is relevant for improving one’s teaching and for moving through the tenure process; however, the validity of assessment methods, such as Student Evaluations of Teaching (SET), have been heavily criticized. Statement of the Problem: Using a one–group pretest–posttest design and assessing learning over the semester has several advantages over SET; however, one drawback is in making conclusions about the cause of changes in the post-test. A change could be due to learning in the semester, maturation, history, or even a testing effect. Literature Review: To improve the inferential quality of teaching assessment, a nonequivalent dependent variable (DV) design is highly advantageous. A nonequivalent DV is an outcome that is not the target of the intervention yet responds to the same contextually relevant factors. Teaching Implications: By using a nonequivalent DV design, there might be an increase from the beginning of the semester to the end of the semester in the main DV, but no increase in the nonequivalent DV, which provides a stronger argument that the change in the main DV is due to a true learning effect. Conclusion: Using nonequivalent DV methodology improves inferential quality and is easily implemented.

Higher Education Instructors' Usage of and Learning From Student Evaluations of Teaching - Do Achievement Goals Matter?

Identifying what motivates and hinders higher education instructors in their self-regulated learning from student evaluations of teaching (SETs) is important for improving future teaching and facilitating student learning. According to models of self-regulated learning, we propose a model for the usage of SETs as a learning situation. In a longitudinal study, we investigate the associations between achievement goals and the usage of and learning from SETs in the context of higher education. In total, 407 higher education instructors (46.4% female; 38.60 years on average) with teaching commitments in Germany or Austria reported their achievement goals in an online survey. Out of these participants, 152 instructors voluntarily conducted SET(s) and subsequently reported their intentions to act on the feedback and improve future teaching in a short survey. Using structural equation modeling, we found, in line with our hypotheses, that learning avoidance, appearance approach, and appearance avoidance goals predicted whether instructors voluntarily conducted SET(s). As expected, learning approach and (avoidance) goals were positively associated with intentions to act on received SET-results and improve future teaching. These findings support our hypotheses, are in line with assumptions of self-regulated learning models, and highlight the importance of achievement goals for instructors’ voluntary usage of and intended learning from SET(s). To facilitate instructors’ learning from SET-results, our study constitutes a first step for future intervention studies to build on. Future researchers and practitioners might support instructors’ professional learning by encouraging them to reflect on their SET-results.

Validation of a Short Scale for Student Evaluation of Teaching Ratings in a Polytechnic Higher Education Institution.

The general purpose of this work is 2-fold, to validate scales and to present the methodological procedure to reduce these scales to validate a rating scale for the student evaluation of teaching in the context of a Polytechnic Higher Education Institution. We explored the relationship between the long and short versions of the scale; examine their invariance in relation to relevant variables such as gender. Data were obtained from a sample of 6,110 students enrolled in a polytechnic higher education institution, most of whom were male. Data analysis included descriptive analysis, intraclass correlation, exploratory structural equation modeling (ESEM), confirmatory factorial analysis, correlations between the short and long form corrected for the shared error variance, gender measurement invariance, reliability using congeneric correlated factors, and correlations with academic achievement for the class as unit with an analysis following a multisection design. Results showed four highly correlated factors that do not exclude a general factor, with an excellent fit to data; configural, metric, and scalar gender measurement invariance; high reliability for both the long and short scale and subscales; high short and long-form scale correlations; and moderate but significant correlations between the long and short versions of the scales with academic performance, with individual and aggregate data collected from classes or sections. To conclude, this work shows the possibility of developing student evaluation of teaching scales with a short form scale, which maintains the same high reliability and validity indexes as the longer scale.

Psychometric Properties of Student Evaluation of Teachers’ Performance Scale: Evidence from Debre Markos University Students’ Evaluation Dataset

Introduction. Student evaluation of teachers’ effectiveness is one of the most common tools used as a measure of teaching performance and accountability by various universities across the globe. The major purpose of this study was to evaluate the validity and underlying structure of students’ evaluation of the higher education teaching effectiveness scale used by all public universities in Ethiopia. Methodology. Data collected from 1397 students at Debe Markos University were used for this analysis. Cronbach’s alpha values and average interitem correlation were used to study the internal consistency reliability of the scale. Composite reliability, average variance extracted, hetero trait-mono-trait ratio, maximum shared variance, average shared variance, and interconstruct correlations were used to assess the construct validity of the scale, and exploratory factor analysis and confirmatory factor analysis were performed with 20 items to test the hypothesis which introduced a four-dimensional construct for teachers’ evaluation scale. We used different goodness-of-fit indices to measure the fit of the models. Results. The scale was shown to have good internal consistency and convergent validity but lacked discriminant validity. Furthermore, confirmatory factor analysis indicated that the four-factor model produced inadequate fit indices, revealing that the original factor structure of the scale changed. Conclusions. The results showed that Student Evaluation of Teaching Effectiveness did not measure what it was supposed to be measuring. Moreover, the exploratory factor analysis and confirmatory factor analysis results indicate that a two-dimensional model is better than the four-dimensional model to explain the data structure, which places limitations on its use.

Comparison of team-based learning vs. lecture-based teaching with small group discussion in a master's degree in nursing education course

BackgroundThe high student-to-faculty ratio is an obstacle in the reform of teaching methods in master's programmes of nursing education in China. Though Kim's study showed that team-based learning (TBL) can be implemented in high student-to-faculty ratio classes, it is still a challenge for students accustomed to the traditional didactic lecture (TDL) teaching method, and the lack of an inclusive design model has restricted the effective application of TBL. ObjectiveTo compare the effect of TBL on postgraduate nursing students with lecture-based teaching with small group discussion (LBTWSGD) and to clarify key factors to promote the effective application of TBL. MethodsA quasi-experimental study design. We selected 27 unbiased participants in the applied LBTWSGD group and 48 participants in the applied TBL group and conducted questionnaire surveys at the last day of the course which lasted for 10 weeks. Students' self-directed learning ability was assessed by the Self-Directed Learning Instrument (SDLI), classroom engagement was assessed by the Self-Report of Engagement Measure (SREM), and students' evaluation of the teaching experience was assessed by the Student Evaluation of Teaching and Units (SETU). A difference analysis between the two groups was performed by the SETU. The mean and standard deviation of each tool were calculated and compared between the two groups. ResultsThe study determined that compared to LBTWSGD, TBL significantly improved students' self-directed learning (p < 0.01) and classroom engagement (p < 0.05). The scores for 6 of 9 items in the SETU were significantly higher in the TBL group than in the LBTWSGD group. ConclusionsThe TBL teaching method can effectively improve nursing students' self-learning ability and classroom engagement through effective teacher feedback, discussion, and communication. The same clear and definite reading assignments in TBL can help students obtain effective teacher feedback and form effective discussions and communication.

Can the Cognitive Processes of Reading Student Evaluations Influence Their Utility?

The end of the semester has any number of regular milestones for teachers, among them the receipt of student evaluations of teaching. Particularly for those who teach large course sections, the anticipated experience of reading evaluations is often bemusement. Frequently, there are significant numbers of student comments on both “sides” of a component of a course, from homework to tests to perceptions of instructor flexibility. Encountering conflicting information induces challenges to the cognition one uses to digest the material. Because evaluations tend to contain such conflicts, the potential for ambivalent reactions to the information is heightened. How such ambivalence arises, and what it might mean for discerning actionable ideas from student evaluations, is a timely concept to ponder as an unusual semester/quarter comes to a close.

Supporting STEM Faculty of Large Enrollment Undergraduate Courses: A Mixed Methods Study of Impact

A plethora of literature exists identifying the importance of engaging students in learning within Science, Technology, Engineering, and Math (STEM) undergraduate courses, making professional development (PD) for faculty teaching these courses vital. Yet few studies of PD programs focus on STEM faculty, incorporate direct measures of faculty practice, and seek to understand the student experience in courses taught by these faculty. In this study, I examine the impact of a program for faculty teaching large enrollment STEM courses on their perceptions, instructional practices, and student perceptions of learning. The program included a week-long course design institute (35 hrs) and a year-long STEM Faculty Learning Community (STEM-FLC) that met monthly (14 hrs). Data included faculty surveys, course syllabi, observations of teaching, and Student Evaluations of Teaching (SETs). Results suggest the program helped instructors create more learning-focused courses and implement student-centered instructional practices to differing degrees. Despite the STEM-FLC, faculty still perceived barriers to implementing their course design. Students’ perceptions of course instruction and self-reported learning in these courses highlighted the importance of in-class activities. Finally, when the course design and instructional practice were aligned, students had more positive perceptions and reported higher learning gains compared to students in courses with misalignments in design and practice. Based on the findings of the study and connection with PD literature, I provide a set of essential PD features that may enable STEM faculty to make meaningful and lasting changes to practice.

The metrification of teaching: student evaluation of teaching in the United States, Germany and Colombia

ABSTRACT By means of a systematic comparison of three countries we investigate the origins and conditions for the diffusion of student evaluation of teaching (SET). We found that (a) SET is present cross-nationally; (b) SET was initiated in the US during the 1950s by students and adopted in the 1970s by universities, while German and Colombian universities started implementing SET during the 2000s; and that (c) common cultural elements for adopting SET include increasing accountability, competition and student empowerment. Concerns for academic freedom in Germany and faculty diversity in the US influence SET’s diffusion and implementation. Our findings test the neo-institutional proposition about the diffusion of similar evaluation practices into university teaching. We also extend theoretical interpretations of the transfer of practices by positioning SET in a global culture that has recently been aiming to balance technocratic measurements of teaching with local concerns about academic freedom and increasing concerns about diversity.

How Faculty Status Impacts Student Evaluations of Teaching: A Study of Full- Versus Part-Time Marketing Faculty

The present multiyear study sheds light on the effects of faculty status on student evaluations of teaching (SETs). By comparing actual SETs of full- versus part-time marketing faculty, this study fills a void in the marketing education literature. Collecting first-hand institutionally administered SETs ( N = 6,123) over a seven-year period, the extended data collection phase includes 21 semesters and 240 undergraduate courses. Study findings reveal that students perceive full-time faculty as more knowledgeable, enthusiastic, and better prepared for class than part-time faculty. Full-timers are also better able to communicate the subject matter and develop assignments focused on student learning than part-timers. In contrast, students perceive part-time faculty as better able to relate the course material to the real world, develop exam questions that reflect lectures and assignments, and return graded material more quickly than full-time faculty. Compared with part-time faculty, students perceive full-time faculty as being more rigorous and tougher in terms of grading. Given the differences regarding instructor knowledge, pedagogical skill, rigor, and grading, the discussion of the findings rests on how faculty status affects the overall quality of higher education.

THE STUDENT EVALUATION OF TEACHING: LET’S BE HONEST – WHO IS TELLING THE TRUTH?

ABSTRACT The student evaluation of teaching is a common practice in the business schools of most colleges and universities. Currently, over 99% of business schools use student evaluations to measure teaching performance, but the honesty of student responses has been questioned. Prior research has indicated that student evaluations can contain answers the students knew were not deserved or purposely false. Given the importance of evaluations in decisions for promotion and faculty tenure, the honesty of student evaluation responses is paramount. Understanding there may be certain characteristics which could predispose students to be less than truthful, this research investigated the influence of consumerism on truthfulness in student evaluations. A survey of 178 undergraduates revealed that about half think of themselves as customers, and that those who think of themselves as customers may be more likely to provide false information on student evaluations.