Student Evaluations Of Teaching Research Articles

Veterinary Student Evaluations of Teaching: Scores and Response Rate When Administered before or after Final Exams.

In higher education, instructors and administrators use student evaluations of teaching (SETs) as formative and summative assessments of instruction; thus, they need adequate response rates for optimal validity and reliability. Veterinary students are often requested to complete dozens of SETs each semester, and response rate is shown to decline as the number of SETs increases. Allowing students to complete SETs after final examinations has been suggested to help increase response; however, students' knowledge of their final course grade has been previously shown to negatively influence SET scores. This case study explored how making SETs available to veterinary students after final exams affected quantitative item scores and response rate when compared to SETs administered during the final weeks of the semester, prior to final examinations. Participants (n = 262) were randomly assigned to before finals or after finals groups, and 171 students completed 2,926 SETs. Students were more likely to complete evaluations before finals (vs. after), and first-year students completed more SETs than third-year students. Compared to the prior year, in which SETs were administered before finals, students in the study year completed 31% fewer SETs. Timing of SET delivery did not significantly affect SET item scores, but third-year students rated instructors higher than first-year students on five of 10 items. Students' self-reported expected grade was positively correlated with all 10 SET items for both groups. In this study, timing of completion had no statistical effect on SET item score. However, when students completed SETs after final exams, response rates decreased.

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching

Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions.

Engaging in student evaluations of teaching through intrinsic motivation: an exploratory study of competence, perceived choice, value/usefulness, and relatedness

ABSTRACT Student evaluations of teaching (SET) elicit feedback to enhance the course, teaching, and learning. However, hospitality and tourism curriculum research lacks an understanding of the dyadic relationship between the instructor and motivation and other logistics to complete SET. This remains important to uncover student competence, perceived choice, value/usefulness, and relatedness to complete SET to increase SET feedback from instructors. Pearson product-moment correlation coefficient analysis was employed to test the hypotheses using data collected from 69 undergraduate hospitality and tourism students recruited through U.S. educational institutions in the northeast and southeast. Results reveal that intrinsic motivation relates positively to SET completion, showing a strong, positive correlation between all variables. The study provides contributions to instructors to offer extrinsic motivators to encourage students to recognize the value of the SET. The study is among the first to explore the intrinsic motivators and completion of SET with hospitality and tourism undergraduate students.

Making Sense of Today’s Use of Student Evaluations of Teaching (SET)

Making Sense of Today’s Use of Student Evaluations of Teaching (SET)

Qualitative Review of Monitoring and Evaluation Practices in Higher Institutions in Uganda

Purpose: The purpose of the study was to investigate how the practices employed in the Monitoring and Evaluation processes contribute to the overall goal of Higher Education Institutions in Uganda. It was carried out using a qualitative document review method. The study was guided by five objectives: to examine the legal framework of monitoring and evaluation in Uganda’s higher education institutions, to formulate monitoring and evaluation policy in higher education institutions, to assess the establishment of constituent colleges of Makerere university, to examine how monitoring and evaluation policy affects university teaching quality, and to assess how the M & E policies are performed at the university senates and council committees.
 Findings: The findings of the study show that the Uganda Ministry of Education and Sports (MoES) has prioritized the concept of quality assurance in the sector at all education levels. Each college at Makerere University is administered as a semi-autonomous entity within the larger university and the overall supervision of the college is the responsibility of the university council and it is the responsibility of College leadership to conduct regular M&E of services offered in their units to ensure that quality is not compromised. Student evaluation of teaching (SET) is used in Makerere university where questionnaires are given to students to evaluate the quality of instruction by their instructors at the end of every lecture, course or semester. There is also peer observation of teaching where colleagues from the same department, subject or discipline help in giving and receiving feedback from one another on the quality of teaching and effectiveness.
 Recommendations: However, the study recommends that monitoring and evaluation should be implemented even in the secondary and primary education, institutions should have both internal and external evaluations for consistency and transparency, more research should be done on the challenges facing the established policies in higher education institutions of learning in Uganda.

The Themes Present in the Qualitative Portion of the Student Evaluation of Teachers with Emphasis on the Differences in Themes among the Sex and Academic Rank of Faculty Members in a State University in the Philippines

This paper identified the themes present in the comments of the students of a state university in the Philippines in the qualitative portion of the student evaluation of teachers (SET) for two semesters, one before the pandemic and the other during the pandemic. Thematic analysis revealed the following themes present: (a) general positive comments, (b) general negative comments, (c) expressions of gratitude, (d) wishes and greetings to the teacher, (e) positive comments on teaching method, (f) negative comments on teaching method, (g) positive comments on teachers’ demeanor, and (h) negative comments on teachers’ demeanor. There were no differences in the themes between male and female teachers and among instructors, assistant professors, associate professors, and professors. Similarly, there were no differences found in the themes identified before and during the pandemic. The thematic analysis showed that many students made only general comments that were not helpful at all in improving teaching effectiveness. Thus, it is recommended that in the SET instrument, instruction should be given to students to make their comments specific so that teachers will know what exact things they are doing well or not. Knowing these will help improve their teaching. It was noted that there were numerous negative comments on the teachers' teaching methods and demeanor. Hence, it is recommended that the university craft intervention measures that will acquaint teachers with the various methods of teaching, considering that at the tertiary level, many teachers are not education majors trained in pedagogy. In addition, seminars and workshops regarding teacher demeanor are suggested.

Using Measurement Invariance to Explore the Source of Variation in Basic Medical Science Students’ Evaluation of Teaching Effectiveness

Introduction: Many research studies have shown that students' evaluations of teaching(SET) are affected by different variables without testing the requirement of fair comparisons. These studies have not tested the measurement equivalency of SET surveys according to these variables. Measurement equivalency of SET refers to whether a SET survey is interpreted similarly across different groups of individuals (Variable Levels). Without evidence of measurement invariance across different variables under investigation, the SET ratings should not be compared across these variables and this is the goal of this study.
 Methods: Measurement Invariance analysis of SET survey was investigated using 1649 responses to SET of four different medical core courses offered by the College of Science and College of Medicine and from different levels.
 Results: The results showed the existence of teaching practices in the SET survey that are not equivalently loaded on its factor across the levels of targeted variables, and the college offered medical courses were a source of variation in basic medical science students’ evaluation of teaching effectiveness. On the other hand, teaching practices in the SET survey are equivalently loaded on its factor across course levels.
 Discussion: The study results showed that the SET of medical courses is comparable to the courses only taught by the College of Medicine. These results provide evidence that medical courses are different from other courses offered by other colleges. This means that comparing SET of the College of Medicine with other colleges and colleges of medicine needs to compare SET results at the college level only.

Effectiveness of the use of open-ended questions in student evaluation of teaching in an engineering degree

Purpose: The purpose of our research is to show the point of view of the members of the Board concerning advantages, disadvantages and effectiveness of open ended questions used as a complement to closed response questionnaires.Design/methodology/approach: In this paper, we will describe a pilot experience carried out at a Spanish public university where a short questionnaire with open ended questions was launched, and students were invited to comment on their perception of the teaching received.Findings: The response return rate (about 35%) was relatively high compared to other online closed response questionnaires delivered. Moreover, the students’ comments provided valuable information which made the members of the Board of the Engineering Degree chosen reflect. Their reflection was evidence based and led to initiatives and actions to improve the quality of teaching, as well as to get an extensive view of the Degree.Practical implications: Findings reveal that the information retrieved can also be used in multiple ways such as formative feedback or even for improvement of courses and instruction. Originality/value: Student evaluation of teaching is a powerful tool for continuous teaching improvement but the information provided by conventional closed response questionnaires may not be sufficient.

Student and teacher performance during COVID-19 lockdown: An investigation of associated features and complex interactions using multiple data sources.

Due to the COVID-19 pandemic, testing what is required to support teachers and students while subject to forced online teaching and learning is relevant in terms of similar situations in the future. To understand the complex relationships of numerous factors with teaching during the lockdown, we used administrative data and survey data from a large Danish university. The analysis employed scores from student evaluations of teaching and the students' final grades during the first wave of the COVID-19 lockdown in the spring of 2020 as dependent targets in a linear regression model and a random forest model. This led to the identification of linear and non-linear relationships, as well as feature importance and interactions for the two targets. In particular, we found that many factors, such as the age of teachers and their time use, were associated with the scores in student evaluations of teaching and student grades, and that other features, including peer interaction among teachers and student gender, also exerted influence, especially on grades. Finally, we found that for non-linear features, in terms of the age of teachers and students, the average values led to the highest response values for scores in student evaluations of teaching and grades.

The impact of multimedia design and the accent of the instructor on student learning and evaluations of teaching

AbstractIn college classrooms, the language background of instructors is rarely considered when designing instructional materials. Previous work suggests that redundant text on slides during multimedia presentations can hinder learning (i.e., the redundancy principle). The current study examines whether additional text support can facilitate learning when the instructor speaks with a non‐native accent. In two experiments conducted in the United States, participants were randomly assigned to watch a lecture with more or less text on the slides narrated by either a Mandarin‐ or an American‐accented instructor. Then, participants completed a multiple‐choice assessment and evaluation of the instructor's effectiveness. The amount of text on the slides and the accent of the instructor did not differentially impact performance on the assessment. However, students rated the instructor who spoke with a Mandarin accent less favorably, suggesting that accent‐based biases influence students' perceptions of teaching even when there are no observable differences in learning.

Student Evaluations of Teaching as a Predictor of Teaching Effectiveness in a Selective Liberal-Arts College

Previous research has shown mixed results regarding the relationship between Student Evaluations of Teaching (SETs) and teaching effectiveness. Although the literature on this topic is extensive, it appears that none of the scholarship was conducted at small, selective, undergraduate institutions in which teaching is prioritized. In the current research, we explored the association between student evaluations and teaching effectiveness at such an institution. Data were collected at a selective liberal-arts institution of approximately 2200 students in the Northeastern United States. Participants were 139 students who had enrolled in both Calculus I and Calculus II. Seven different professors taught Calculus I, while seven different professors taught Calculus II. Linear regression analyses indicated that, for one of the three subscales of the College’s SET, there was a positive association between SET scores of the faculty member in Calculus I and student grades in Calculus II. Machine-learning analyses confirmed this association and indicated that the relationship was strongest for students with overall grade-point averages near the cohort’s mean. Our research thus adds to the scholarship broadly but also suggests that assessment of SETs in smaller, selective teaching-oriented institutions is an important direction for future scholarship.

Rethinking student evaluation of teaching: A policy analysis

Institutions of higher education with tenure and promotion policies that rely heavily on student evaluations of teaching may create academic progression barriers for Black, Indigenous, and People of Color (BIPOC) faculty. The purpose of this policy analysis is to explore how current practice of utilizing Student Evaluation of Teaching (SETs) as the major source to evaluate teaching effectiveness, impacts tenure for BIPOC faculty, using Bardach and Patashnik's (2019) eight steps of policy analysis. Context provided includes cultural history, policy and evidence regarding the use of SETs. Evaluation criteria focuses on two national nursing reports and nursing accreditation standards, along with utilization of the 2020 Teaching Quality Framework for Evaluation of Teaching. The status quo utilizes SETs as the major evaluation tool for overall teaching effectiveness. SETs are shown to produce an unacceptably high error rate and fail to identify the best teachers. Teaching evaluations that utilize both formative and summative methods provide a global, less biased view of instructional pedagogy. Making a change in the way teaching is evaluated is a critical component in the retention of BIPOC Faculty and their ability to attain tenure.

Missing the forest for the trees: investigating factors influencing student evaluations of teaching

Student evaluations of teaching (SETs) feature prominently in higher education and can impact an academic’s career. As a result, they have attracted considerable research attention in order to identify evidence of bias and the influence of factors beyond an educator’s control. This study investigates the influence of seven factors on a large dataset of student evaluations (N = 376,805) of academics teaching at an Australian university. Students were invited to rate their experience at the end of each teaching period using an online survey instrument. The following factors are analysed comparing means between relevant groups to verify if: i) SET is dominated by students with strong feelings; ii) revenge reviews are given by angry students; iii) larger units are rated lower than smaller units; iv) different expectations/ratings are given by students of different gender and backgrounds; v) reticence of international students lowers overall ratings; vi) bigoted students skew results for some staff; and, vii) SET surveys during examinations disadvantaging academics teaching units with examinations. Overall, while statistically significant differences were found, they represented only small or trivial effects, with medium effects in only two limited cases. The results highlight the importance of explicitly reporting effect size of statistically significant results, and the benefits of representing differences visually in ways that avoid over-emphasising differences.

Student Evaluation of Teaching: More Harm than Good?

Student evaluation of teaching (SET) has more than forty years of history as a tool of feedback in North American, Australian and European universities. In Russian universities it is not widespread, although increasing its implementation could solve the problem of Russian students feeling that they are not involved in the management of the educational process and improve the quality of courses and teaching. The barriers to the implementation of SET in the Russian higher education system, among other things, are concerns and prejudices of teachers and students, who see SET more as a problem than an effective tool. In this article, authors analyze the most common faculty concerns about student evaluations of teaching, obtained from ten focus groups in five Russian universities. Authors put collected opinions in the context of the recent methodological discussions about the validity of Student Evaluations of Teaching. The main idea is that SET is an effective tool of getting feedback in higher education and involvement of students and teachers in a dialogue and cooperation in the educational process, but highly questionable as a data source for administrative decisions.

How do Students and Faculty Consider Numerical Ratings and Comments About Daily Quizzing when Interpreting Student Evaluations of Teaching?

Background: Student evaluations of teaching (SETs) are used to assess faculty performance, but prior research has identified sources of bias in the completion and interpretation of SETs. Objective: We investigated how SET ratings and comments about quizzes are interpreted by faculty and undergraduates. Method: Participants made judgments about teaching effectiveness after reviewing a fictional professor's SETs. SETs varied in whether the professor was rated lower or higher than the departmental average using quantitative measures, and whether qualitative comments about the professor mentioned daily quizzes. Results: In Experiment 1, more positive evaluations were provided for SETs with higher ratings, whereas comments about quizzing minimally influenced evaluations; this pattern was similar for student and faculty responses. In Experiment 2, qualitative comments were presented in isolation. Student participants provided more positive evaluations for some measures of teaching effectiveness when regular quizzing was mentioned in student comments. Conclusion: Quantitative SET ratings may overshadow qualitative comments about effective teaching practices yet qualitative ratings presented alone reveal some association between quizzing and evaluations of teaching. Teaching Implications: SET interpretation should focus on effective pedagogy such as quizzing, which can aid student learning. We found no evidence that comments about quizzing lowered perceptions of teaching effectiveness.

The Economic Statistics Skills Assessment (ESSA)

In this paper, we document the design, development, and validation of a new multiple-choice assessment created to test student knowledge and understanding of probability and statistics concepts used in economics. High quality measures of student skills and understanding are useful for course and program evaluation, analysis of pedagogical innovation, and identifying misconceptions and gaps in students’ prerequisite knowledge in more advanced courses. Alternatives such as final examinations often do not provide the desired coverage and are difficult to compare across terms and institutions, and student evaluations of teaching (SETs) may reflect student sentiment rather than the quality of instruction and overall student learning. Until now, the only standard assessment appropriate for undergraduate economics courses was the Test of Understanding College Economics (TUCE), which tests introductory economics knowledge.

Student Evaluation of Teaching (SET): Why the Emperor Has No Clothes and What We Should Do About It

Student Evaluation of Teaching (SET): Why the Emperor Has No Clothes and What We Should Do About It

Student evaluations of teaching during the COVID-19 pandemic.

Student evaluations of teaching during the COVID-19 pandemic.

Cross-Classified Item Response Theory Modeling With an Application to Student Evaluation of Teaching

The cross-classified data structure is ubiquitous in education, psychology, and health outcome sciences. In these areas, assessment instruments that are made up of multiple items are frequently used to measure latent constructs. The presence of both the cross-classified structure and multivariate categorical outcomes leads to the so-called item-level data with cross-classified structure. An example of such data structure is the routinely collected student evaluation of teaching (SET) data. Motivated by the lack of research on multilevel IRT modeling with crossed random effects and the need of an approach that can properly handle SET data, this study proposed a cross-classified IRT model, which takes into account both the cross-classified data structure and properties of multiple items in an assessment instrument. A new variant of the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm was introduced to address the computational complexities in estimating the proposed model. A preliminary simulation study was conducted to evaluate the performance of the algorithm for fitting the proposed model to data. The results indicated that model parameters were well recovered. The proposed model was also applied to SET data collected at a large public university to answer empirical research questions. Limitations and future research directions were discussed.

Using topic modeling to understand comments in student evaluations of teaching

Written comments in student evaluations of teaching offer a rich source of data for understanding instructors’ teaching and students’ learning experiences. However, most previous studies on student evaluations of teaching have focused on numeric ratings of close-ended questions, while few studies have tried to analyze the content of students’ written comments on open-ended questions, which normally involves a labor-intensive manual process of coding and categorizing. Such qualitative efforts prevent solutions on a large scale since it is almost impossible to go through all the textual data manually. Therefore, an innovative quantitative method that can analyze a large corpus of data holds great promise. This paper proposes the latent Dirichlet allocation (LDA) method of topic modeling to discover important themes that emerge in students’ written comments. We compare our results with findings in previous qualitative studies. We also investigate how these themes vary by course grade level and course subject. Our results provide evidence that topic modeling can be an effective and efficient alternative for understanding teaching and learning experiences through students’ written comments on a large scale.