Student Evaluations Of Teaching Research Articles

Gender bias and statistical discrimination against female instructors in student evaluations of teaching

This paper uses administrative data from a public university in Taiwan to examine gender bias in teaching evaluations. We test for statistical discrimination against female instructors using the employer learning model in which the instructor value-added to the grades of the current course and follow-on course is used to measure teaching effectiveness. The results show that statistical discrimination is a significant source of gender bias in teaching evaluations, especially among male students and in STEM departments where female faculty is underrepresented. Gender bias in teaching evaluations was reduced by nearly 50% after ten years of teaching. The results also suggest that the gender gap in teaching evaluations changes over time as male and female students evaluate male and female instructors differentially. Statistical discrimination is closely related to the underrepresentation of women in academia. For female students, the gender gap in evaluation scores narrows when the share of female faculty in the department rises. By contrast, male students are less sensitive to the percentage of female faculty in the department.

Improving the quality of teaching by utilising written student feedback: A streamlined process

Currently student feedback is mainly evaluated with quantitative methods since qualitative analysis has been highly effort intensive. In this article, we present a process for tapping into the resource of responses to open-ended feedback questions by using a topic-modelling approach that goes beyond listing modelling outcomes. The objective of this study is to present a streamlined, yet rigorous, process for analysing large amounts of written feedback that connects qualitative findings to existing literature, theories and quantitative feedback. The topic models are created using the Latent Dirichlet Allocation (LDA) method, after which qualitative and quantitative evaluation methods are used to validate the topic outcomes. The proposed process can help educators analyse teaching quality on programme- or institution-wide level, or on single courses with a very large number of students. The process systematizes and combines existing processes, is repeatable, and can serve as a basis for richer analysis for educators. In student evaluation of teaching (SET) research, it advances the state of the art in applied topic modelling by demonstrating how to validate the topics via thematic analysis and by connecting them to theoretical frameworks and quantitative data. Previous topic modelling studies in this field follow mainly descriptive approaches. We demonstrate the process with feedback data collected from 6087 student evaluations of university courses and confirm that quantitative feedback variables can be used to validate qualitative feedback topic-modelling outcomes and thematic analysis provides a more in-depth explanation of the topics. We additionally find that the proposed topic modelling approach discovers new constructs of SET that cannot be distinguished from quantitative SET measures. The main limitation of the study is that the proposed process is novel and requires further evaluation to establish its full validity.

“Walkabouts” Integrated Physical Activities from Preschool to Second Grade: Feasibility and Effect on Classroom Engagement

An evolving literature demonstrates that physical activity in the classroom represents a promising avenue not only for increasing the physical activity of youth but also for facilitating academic achievement and classroom engagement. However, structured physical activity programs that make clear connections with the academic standards across different grades are limited. The purpose of this study was to examine the feasibility and effectiveness of the “Walkabouts”, a web-based physically active learning program, on attention and behavioral control of children. Twelve preK to 2nd grade classes (N = 245 students) were assigned to the intervention (integrating “Walkabouts” with academics) or control (no added movement) group. Teacher ratings of students’ attention and behavioral control in the classroom were collected before and after the 7-week intervention in both groups. Treatment fidelity was monitored through observations and daily teacher logs. Teachers perceived the Walkabouts to be feasible, physically active, of appropriate difficulty, and enjoyable for the students. MANOVAs showed a significant time by group interaction, meaning that the intervention group improved significantly more, compared to the control group, in both attention and behavioral control whereas the control group declined over time. A significant time by group by grade interaction showed that the benefit of Walkabouts was larger for the Kindergarten students. Integrating movement with academic subjects with programs such as the Walkabouts, may facilitate learning and academic achievement by increasing cognitive and behavioral control in the classroom. Schools can focus on academic achievement goals without sacrificing physical activity throughout the school day.

Distance Learning – the Current Status and Directions for Further Research

The article aims to discuss current research trends in the field of Distance Learning (DL). It consists of the following parts: (1) the definitions and characteristics of DL; (2) current directions of research in the field of DL, among others: measuring transactional distance in Web-Based Learning Environments, Problem-Based Learning (PBL) in DL, development of learning content and the task for MOOCs, effects of online course efficiency perceptions on Student Evaluation of Teaching (SET) measures, new technologies for the DL, the use of metacognitive and affective model Self-Regulated Learning (MASRL), questionnaire (Constructivist Learning on a Scale Higher Settings of Education [CLHES]) as a method of research of constructivist learning environment in higher education, Transactional Distance (TD) theory in DL and online community culture; (3) summary and conclusions with a particular focus on the use of DL methods in higher education. The article is based on the study of literature registered in the databases ERIC, LISA, and LISTA.

Engaging expectations: Measuring helpfulness as an alternative to student evaluations of teaching

Abstract We propose an alternative to student evaluations of teachers (SETs), arguing that writing programs can use the SET moment to share responsibility for students’ expectations and course experiences. We argue studying students’ perceptions can help writing programs generate research for localizing engagement and aiding professional development. We study the perceived helpfulness of first-year writing (FYW) program experiences for students at a large, doctoral-granting research institution in the Midwest. We engage the expectation disconfirmation paradigm, which posits both expectations and experiences contribute to satisfactory outcomes. We analyze two semesters of survey data (n = 977) using structural equation modeling and find that disconfirmation – a confrontation between students’ expectations and experiences – related to students’ ratings of helpfulness. Helpfulness ratings varied directly with disconfirmation, suggesting students found courses more helpful when their experiences exceeded locally defined expectations, and less helpful when experiences did not meet expectations. Program and instructor variables may have also influenced student attitudes. Male instructor gender identity associated with greater helpfulness ratings. Additionally, disconfirmation varied with enrollment level, and had an indirect effect on helpfulness ratings. Given our findings, we describe directions for professional development. We also identify opportunities for future writing assessment research related to the perception of helpfulness.

Examining strategies to increase student evaluation of teaching completion rates

Student evaluations of teaching (SETs) can provide valuable assessment information. However, online administration of SETs has corresponded with lower SET completion rates. We examined perceptions of strategies for increasing SET completion rates and the effect of offering psychological prompts (completion goals, reminders, progress feedback) and micro-incentives (-extra credit that has a negligible effect on student grades) on SET completion rates, grades, and student evaluations. Study one (N = 469 instructors) revealed that instructors who used incentive-based strategies (compared with instructors who did not) rated such strategies as more ethical and more effective, and reported greater likelihood of using incentive-based strategies in the future. Examination of student records revealed higher SET completion rates among instructors who offered micro-incentives than among instructors who did not. In Study two (N = 36 instructors), we randomly assigned instructors to one of three conditions: psychological prompts, micro-incentives plus prompts, or a no-information standard-practice control condition. SET completion rates were lowest in the control condition (53.9%), followed by the prompts (64.5%) and micro-incentives plus prompts (79.7%) conditions. The two experimental conditions did not differ in completion rates, but only micro-incentives differed significantly from the control. These results demonstrate that prompts and incentives are promising methods for increasing SET completion rates.

Student Evaluation of Teaching: Exploring Instructor and Student Perspectives with Course Redesign

Student Evaluation of Teaching (SET), or instructor evaluations, is used as a significant instrument across the world to measure instructors teaching methods and course evaluations. With the lack of standardized SET across universities and institutions, this study gains insight into how instructors use and improve student evaluations and students’ views of how their feedback is utilized by posing questions to university students and faculty through focus groups and interviews. Data was gathered and recorded to interpret students’ perceptions with how instructors utilize the students’ evaluations and instructors’ perceptions of student evaluations and how the instructors use the students’ feedback. Results indicate that students and instructors have different values for student feedback and curriculum improvement. Implications of different values for student feedback and curriculum improvement include instructors not attempting to improve their teaching and course, students poorly rating their instructors, and students that may not be challenged due to possibly receiving a negative evaluation.

Teaming up to Teach Teamwork in an LIS Master’s Degree Program

Abstract
 Objective – Collaboration and working in teams are key aspects of all types of librarianship, but library and information studies (LIS) students often perceive teamwork and group work negatively. LIS schools have a responsibility to prepare graduates with the skills and experiences to be successful working in teams in the field. Through a grant from the university office of assessment, the assessment committee at the University of Rhode Island Graduate School of Library and Information Studies explored their department’s programmatic approach to teaching teamwork in the MLIS curriculum.
 Methods – This research followed a multi-method design including content analysis of syllabi, secondary analysis of student evaluation of teaching (SET) data, and interviews with alumni. Syllabi were analyzed for all semesters from fall 2010 to spring 2016 (n = 210), with 81 syllabi further analyzed for details about their team assignments. Some data was missing from the dataset of SETs purchased from the vendor, resulting in a dataset of 39 courses with SET data available. Interviews were conducted with a convenience sample of alumni about their experiences with teamwork in the LIS program and their view of how well the LIS curriculum prepared them for teamwork in their careers (n = 22).
 Results – Findings indicate that, although alumni remembered teamwork happening too often, it was required in just over one-third of courses in the sample period (fall 2010 to spring 2016), and teamwork accounted for about one-fifth of assignments in each of these courses. Alumni reported mostly positive experiences with teamwork, reflecting that teamwork assignments are necessary for the MLIS program because teamwork is a critical skill for librarianship. Three themes emerged from the findings: alumni perceived teamwork to be important for librarians and therefore for the MLIS program, despite this perception there is also a perception that the program has teamwork in too many courses, and questions remain about whether faculty perceive teaching teamwork as important and how to teach teamwork skills in the MLIS curriculum.
 Conclusions – Librarians need to be able to collaborate internally and externally, but assigning team projects does not guarantee students will develop the teamwork skills they need. An LIS program should be proactive in teaching skills in scheduling, time management, personal accountability, and peer evaluation to prepare students to be effective collaborators in their careers.

The Street Lamp Paradox: Analysing Students’ Evaluation of Teaching through Qualitative and Quantitative Approaches

This paper presents a study about Students Evaluation of Teaching (SET). Student surveys delivered at the end of the academic course is an approach currently applied in all Italian university courses. However, the quality of teaching is a multi-layered phenomenon whose evaluation might require additional methods. In this study, concerning a course offered during the academic year 2016/2017, we propose the qualitative analysis of students’ final essays, which includes the students’ opinion about teaching quality. The qualitative evaluation of teaching is then compared with the results of the standard SET survey. 48 students filled in the survey, while 47 delivered their final essay (a corpus of about 650 text pages). Our study shows that while the standard survey provides an overall picture useful at institutional level for Quality Assurance (QA), the qualitative approach captures an accurate account of students’ reactions, sensitive to the pedagogical approach adopted. This provides additional information on the students’ perspectives regarding the specific features of the course. Our case study suggests that the integration of traditional SET survey with qualitative teaching evaluation approaches, at least for innovative courses based on socio-constructivist learning, might provide information, overlooked in the Italian SET survey, that is useful for Quality Enhancement (QE) of teaching.

Exploring the personal and professional factors associated with student evaluations of tenure-track faculty.

Tenure-track faculty members in the United States are evaluated on their performance in both research and teaching. In spite of accusations of bias and invalidity, student evaluations of teaching have dominated teaching evaluation at U.S. universities. However, studies on the topic have tended to be limited to particular institutional and disciplinary contexts. Moreover, in spite of the idealistic assumption that research and teaching are mutually beneficial, few studies have examined the link between research performance and student evaluations of teaching. In this study, we conduct a large scale exploratory analysis of the factors associated with student evaluations of teachers, controlling for heterogeneous institutional and disciplinary contexts. We source public student evaluations of teaching from RateMyProfessor.com and information regarding career and contemporary research performance indicators from the company Academic Analytics. The factors most associated with higher student ratings were the attractiveness of the faculty and the student’s interest in the class; the factors most associated with lower student ratings were course difficulty and whether student comments mentioned an accent or a teaching assistant. Moreover, faculty tended to be rated more highly when they were young, male, White, in the Humanities, and held a rank of full professor. We observed little to no evidence of any relationship, positive or negative, between student evaluations of teaching and research performance. These results shed light on what factors relate to student evaluations of teaching across diverse contexts and contribute to the continuing discussion teaching evaluation and faculty assessment.

Use of Student Evaluation of Teaching (SET) Survey to Evaluate Effectiveness of Teaching in a Leadership Course among Dental Students over Three Years.

Leadership courses are being increasingly integrated into dental curricula. The study aimed to assess the validity and reliability of student evaluation of teaching (SET) instrument among dental students and to evaluate the effectiveness of teaching in a new leadership course over a period of three years. This cross-sectional study was conducted on fourth-year undergraduate dental students (N = 260) who took a practice management course over three consecutive years from 2014 to 2016. A 29-item SET questionnaire was administered among students who were willing to participate in the study. Out of 260 students, 185 returned completed surveys and the response rate was 71.15%. Factor analysis (principal component analysis) showed the validity of four dimensions of the SET instrument. Total variance explained by four dimensions was 62.80%. Cronbach's alpha for the instrument was 0.95 and each dimension had fairly high internal consistency (>0.80). Treating students with respect (94%), accepting different viewpoints of students (94.1%), being flexible/open-minded (92.5%), and preparedness in the course (91.9%) were the most common effective teaching traits. Over the period of three years, 16 items showed improvement in teaching and there was a significant improvement in four items (P < 0.05). In conclusion, it was found that SET is a valid instrument to evaluate the effectiveness of teaching in nonclinical courses in dentistry. This instrument should be used longitudinally to compare the effectiveness of teaching.

Why and how you should read student evaluations of teaching

Why and how you should read student evaluations of teaching

Team-based learning in social work law education: a practitioner enquiry

ABSTRACT Team-based learning (TBL) as a model of teaching boasts a robust and growing evidence base, indicating that it is both preferred by students and leads to better educational outcomes. Despite this evidence base, there is limited research into the use of TBL in social work education, and no research into the use of TBL in social work law. This article presents the results of a practitioner enquiry undertaken into the use of TBL with a first-year Master’s cohort of students undertaking a social work law module. Results are based on anonymous questionnaires, student evaluations of teaching, and the observations and reflections of the researcher. Student responses are shown to be overwhelmingly positive, recognising TBL to be a more effective model of teaching content, as well as enhancing team working skills, student confidence and promoting friendships. Students also raised some concerns related to group tensions and dynamics. These findings have implications for teaching the practical application of social work law, an area where social workers and social work students have been shown to struggle, as well as having implications for social work education more broadly.

Student evaluations of teaching (SET): implications for medical education in psychiatry and an approach to evaluating SET and student performance

We present reflections on student evaluation of teaching (SET) in the context of recent higher educational research that assesses SET, as well as concurrent and/or subsequent student performance. In a sense, there is in-built cynicism in SET, with more favourable SET for easier assessment. There is emerging evidence that SET is inversely proportional to the performance of students in subsequent courses, i.e. the higher the ratings, the poorer the students perform in subsequent studies. It is proposed that SET should be combined with contemporaneous formative and summative assessments of student performance in medical school settings, especially in psychiatry education.

Teacher-student ratings of social, emotional, behavioural needs among high school freshmen students

ABSTRACT Students’ social, emotional, and behavioural (SEB) needs are associated with their academics and school behaviours, and teachers are in proximal roles to address students’ needs. However, it is unclear how teacher-student understanding of SEB needs relate to students’ academic achievement and behaviours. Based on 264 ninth-grade student-teacher dyads from one American semi-urban high school, teacher-student levels of SEB agreement were assessed and compared with students’ grade point averages, school attendance, and number of disciplinary referrals. Findings indicate that the degree of common agreement about SEB needs ranges from 48% (engagement) to 87% (bullying). Further, teacher-student agreement about externalising and bullying behaviours relates to students’ GPA and attendance. Agreement about an array of social skills also predicts students’ office disciplinary referrals. Understanding the level of agreement between teachers and students is critical in informing teacher professional development and SEB-related classroom instruction.

Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis

Student Evaluations of Teaching (SETs) do not measure teaching effectiveness, and their widespread use by university administrators in decisions about faculty hiring, promotions, and merit increases encourages poor teaching and causes grade inflation. Students need to get good grades, and faculty members need to get good SETs. Therefore, SETs empower students to shape faculty behavior. This power can be used to reward lenient-grading instructors who require little work and to punish strict-grading instructors. This article reviews research that shows that students (a) reward teachers who grade leniently with positive SETs, (b) reward easy courses with positive SETs, and (c) choose courses that promise good grades. The study also shows that instructors want (and need) good SETs.

Gender Bias in Student Evaluations of Teaching: Students’ Self-Affirmation Reduces the Bias by Lowering Evaluations of Male Professors

Students evaluate male professors higher than female professors. In a study that we presented to participants as a test of a new form for student evaluations of teaching (SETs), we examined if self-affirmation (contemplating elements that positively contribute to one’s self-image) reduced the gender bias. Belgian students (n = 568), who were randomly assigned to self-affirm (through either a value-affirmation task or self-superiority priming) or not, read a vignette prompting them to imagine that they had received a good or a bad grade from a male or a female professor. They evaluated the course, the professor, and the form. Non-self-affirmed participants showed a gender bias after a bad grade, disadvantaging the female professor. Self-affirmation eradicated the gender bias by lowering evaluations for the male professor, suggesting that the gender bias involves overvaluing male rather than derogating female professors. Without self-affirmation, the positivity of the SETs was correlated with participants’ evaluation of the SET form itself. Self-affirmation inflated the correlation for the male professor and eradicated it for the female professor. Having students self-affirm before SETs may be useful when SETs are obligatory only. An even better approach is asking SETs before students learn their grades or simply abolish SETs as a factor in hiring and promotion decisions.

Students as Evaluators of their Learning Experience

Student evaluations of teaching (SET) have been an evergreen topic of passionate debates in higher education. Concerns have been raised about use student feedback in the form of SETs for administrative purposes. Nevertheless, they remain a reality at most institutions and are frequently the sole or key metric on which teaching excellence is evaluated.This session will provide an overview of national trends with the struggle to define teaching excellence in context and the current discussion on what aspects of the teaching and learning can be evaluated with different SET formats. Viewed through the lens of academic development, participants will examine why SETs are so pervasive, how to mitigate the emotional charge that is a barrier to individual growth of instructors within their professional setting, and how students can act as productive partners in institutional learning.

Current Use of Student Evaluations of Teaching in Anatomy Education

Anatomy, histology, and embryology (or anatomical sciences) provide a framework for understanding normal function as well as disease and its underlying mechanisms. Thus, they are often taught (by basic scientists) in the context of their clinical relevance. Delivery formats and settings vary widely: anatomy, histology, and embryology can be combined in a single course or integrated into other courses at different points in a curriculum. Anatomy labs are taught with the expectation (by students) that faculty know every detail, be it taught in the form of dissection, prosection, or virtually. The uniqueness of teaching contexts in anatomical sciences education can make it challenging to document effective teaching and to obtain feedback from students that provide growth opportunities to develop as an educator. Educators in anatomical sciences are charged with facilitating learning of skills and knowledge that are foundational and can be applied to many health sciences fields. Because anatomical sciences can be taught virtually as well as practically, additional professional development needs arise. But carving out time for development activities is challenging. Some educators teach several different levels of learners, while managing donor programs, and coordinating laboratory use, all competing demands on their time. Simultaneously, curricular changes have reduced anatomical sciences curricular hours to an all‐time low, and education is shifting towards active learning modalities with increased digital learning and reduced classroom and laboratory teaching. These modalities, if not applied expertly, can increase student pushback. Topics explored in this presentation include if and how our methods of evaluating teaching effectiveness have evolved with the changed landscape in which anatomical sciences faculty teach, and how to support faculty in their development as educators by constructive use of student evaluations of teaching.

Graduate Teaching Assistant Training Program at the University of Arizona

The art of effective teaching requires guidance, practice and feedback. The 2000+ undergraduates who enroll in our introductory Human Anatomy and Physiology (PSIO 201/202) courses each year deserve an optimal educational experience; similarly, the Graduate Teaching Assistants (GTAs) who teach the associated labs deserve the professional training to optimize their teaching and presenting skills to further their careers. At University of Arizona, we have developed a three‐pronged approach to enable success in both. 1) The Physiological Sciences Teaching Workshop (PS 697A&amp;B) is a year‐long course designed to help new GTA’s develop and fine tune their teaching techniques by focusing on ‘best practices’ through guest lectures, peer observations and group discussion. This prong emphasizes the development of a teaching philosophy, provides helpful ideas for engaging students and creating an active learning environment in the lab, and provides training in professional skills related to presenting and communicating. 2) Weekly content‐based sessions led by the 201/202 course instructors and lab coordinators focusing on the course‐specific material and experiments to be covered in the upcoming week’s lab, with additional insights and handy hints provided from seasoned TA’s. During these 2–3 hour meetings, GTAs receive lab outlines (prep notes), template slides, perform dissections and experiments, and review slides and anatomical models. They also have the opportunity to give feedback on the previous week’s lab and ask questions about the upcoming lab, enabling continuous fine‐tuning of the labs used each year. Both the Teaching Workshop and the 201 or 202‐Specific Preparation begins each fall with an intensive orientation the week prior to the semester to insure the teaching team is well‐versed with university and department policies and resources, course specific policies and problem solving and are certified in CPR and First Aid. The TW and weekly preparation meetings complement each other to provide a well‐rounded training program for the GTAs fully preparing them to meet the demands of teaching 2 labs of 25–30 students each about the intricacies of the human body. 3) The 3rd prong is the Physiology Forum, a bimonthly seminar series, now in its 10th year, which hosts research presentations required of all our Physiological Sciences graduate students (MS and PhD candidates). Each student presentation is provided with post‐hoc critiques from the full cohort of graduate students and 2 faculty selected on a 2 year rotating basis, resulting in remarkably high quality presentations across the board. Each of these 3 venues provide preparation with respect to content and format, presentation skills and formative feedback. Using student teaching evaluations for the labs, analyses by TA’s on the benefits of the Prep and Workshop sessions, and comments from intra‐ and extramural faculty where our students have presented their work suggests that this combination of hands‐on practice paired with constructive feedback has served our graduate students well as they proceed onto the next steps in their professional careers. Here, we describe the implementation of the specific activities, goals, and learning outcomes that drove the design of our GTA training program.