Student Evaluation Of Teaching Surveys Research Articles

When to conduct student evaluation of teaching surveys: before or after the final examination?

Student evaluation of teaching (SET) involves numerous conceptual and methodological problems. This paper focuses on a specific methodological dilemma: whether to conduct SET surveys before or after the final examination. This decision is a critical administrative and practical issue that can affect the level and quality of SET results. To address this problem, a systematic literature search was initially performed. However, it found only 17 studies and inconclusive results. In the empirical component, a replication survey (n = 5,077) was conducted. It posed seven course-related questions to the same students before and after their final examination. A slightly higher but statistically significant (p < 0.001) average score was obtained in the after-examination survey (a 0.06 difference on a 1–5 scale). This increase was partly attributed to a positive examination experience. The overall test–retest correlation (r = 0.61) indicated relatively low reliability, suggesting that the before-examination survey better reflected students’ attitudes. The data quality was also higher in the before-examination survey, although the before-examination survey could not properly assess the examination-related aspects of the course. From a methodological perspective, the most comprehensive insight seems to be derived from conducting SET surveys before the examination and following up with a brief after-examination survey.

Major increases in teachers’ performance evaluations: Evidence from student evaluation of teaching surveys

Purpose: This exploratory study examined major increases in teachers’ performance evaluations and their immediate impact on next year’s score for those instructors that taught the same subject for at least two years in a row. The purpose was twofold. Firstly, to identify those Student Evaluation of Teaching (SET) survey items associated with major increases in teacher evaluations. Secondly, to examine if there is evidence of the use of these SET results by instructors to improve their teaching.Design: The sample comprised SET survey ratings from one university over a five consecutive year period, for a total sample of 13,052 teacher evaluations and 3,893 teachers-subject observations under analysis. Frequency tables and Student’s t-test were used for analysis.Findings: The results highlighted the three SET survey items captured by the dimension of teaching methodology as those most closely related to major increases in teacher evaluations. Regarding the second objective, the results show no generalised response from teachers who experience major increases in SET ratings. This suggests that the use of SET results is either limited or does not have an immediate measurable effect on student satisfaction.Originality/Value: To the best of our knowledge, this was the first study to specifically examine major increases in teachers’ performance evaluations and their immediate impact on next year’s score based on evidence from SET surveys.&#x0D; Received: 22 December 2021Accepted: 26 February 2023

Is it possible to redress noninstructional biases in student evaluation of teaching surveys? Quantitative analysis in accounting and finance courses

Several studies have reported that student evaluation of teaching (SET) presents important problems. First, depending on the area, there are significant differences in the evaluations. Second, numerous noninstructional biases exist, such as when those teachers who award better grades obtain better SETs. Correcting the rankings by considering these biases (e.g., adjusting SETs according to the class grade) has been proposed. In this paper, we analyse a third problem: it is impossible to correct the biases because they are specific to each area, level, and even class. On a sample of 15,439 SETs, we compared the biases present in two very close areas (accounting and finance) and at two levels (undergraduate and postgraduate). Then, we used a procedure based on the analysis of residuals in OLS models to eliminate area- and level-specific biases. However, there are still latent biases apparently linked to each specific group of students.

Student motivations, perceptions and opinions of participating in student evaluation of teaching surveys: a scoping review

Several times each year the teaching performance of academics at higher education institutions are evaluated through anonymous, online student evaluation of teaching (SET) surveys. Universities use SETs to inform decisions about staff promotion and tenure, but low student participation levels make the surveys impractical for this use. This scoping review aims to explore student motivations, perceptions and opinions of SET survey completion. Five EBSCO® databases were searched using key words. Thematic analysis of a meta-synthesis of qualitative findings derived from 21 papers identified five themes: (i) the value students’ place on SET, (ii) the knowledge that SET responses are acted upon to improve teaching, (iii) assurance of survey confidentiality and anonymity, (iv) incentives for completing SET, and (v) survey design and timing of survey release. Perceptions, knowledge and attitudes about the value of SET are essential factors in motivating students to engage and complete SETs, particularly if surveys are easy to interpret, time for completion is incentivised and responses are valued.

First, do no harm: automated detection of abusive comments in student evaluation of teaching surveys

ABSTRACT Student evaluation of teaching (SET) surveys are the most widely used tool for collecting higher education student feedback to inform academic quality improvement, promotion and recruitment processes. Malicious and abusive student comments in SET surveys have the potential to harm the wellbeing and career prospects of academics. Despite much literature highlighting abusive feedback in SET surveys, little research attention has been given to methods for screening student comments to identify and remove those that may cause harm to academics. This project applied innovative machine learning techniques, along with a dictionary of keywords to screen more than 100,000 student comments made via a university SET during 2021. The study concluded that these methods, when used in conjunction with a final stage of human checking, are an effective and practicable means of screening student comments. Higher education institutions have an obligation to balance the rights of students to provide feedback on their learning experience with a duty to protect academics from harm by pre-screening student comments before releasing SET results to academics.

Measuring Teaching Effectiveness—or Not

Faculty in the present-day academic medicine environment are expected to perform multiple functions, notably, the provision of high-quality teaching to the medical professionals of tomorrow. However, evaluating the effectiveness of this teaching is particularly difficult. Student evaluations of teaching, despite their many flaws, are widely used as a convenient tool to measure teaching effectiveness. Administrators continue to routinely use student evaluation of teaching surveys in faculty retention/promotion and merit pay decisions. This practice should be reevaluated since it may have unintended consequences, such as grade inflation and content debasement, and may contribute to faculty leaving the institution and even the profession. A more valid, reliable, and formative protocol for the evaluation of genuine teaching effectiveness needs to be developed as a matter of some urgency. In this review, alternatives to the student evaluation of teaching are explored to better measure true teaching effectiveness.

Creating confusion or creative evaluation? The use of student evaluation of teaching surveys in Japanese tertiary education

A new reform movement in Japanese tertiary education has emerged which is more economy centered, more market sensitive, and more influenced by a government shift towards decentralization. With the dramatic decline in the 18 year old population, a buyer’s market has led to the introduction of student evaluation of teaching surveys (SETs) partly as a measure student satisfaction. This is not without debate, and this study seeks to understand the perceptions of 22 local and expatriate English language teachers who participated in interviews. They suggest that using SETs as the sole criterion for evaluating teachers is flawed, unsystematic, and does not lead to improvement. Participants suggest the need for a model of “creative evaluation” over the present “creation of confusion.” Teachers are unaware of the purpose of the evaluation which is not explained and often are just expected to administer without any consultation or input into the questions. Evaluation should draw distinction between prescriptive, acontextual summative evaluation and collaborative approaches that show the richness and diversity while giving learners as well as faculty more voice.

Does the use of end of semester evaluation forms represent teachers’ views of teaching in a tertiary education context in Japan?

This paper considers both how the mandatory introduction of student evaluation of teaching surveys (SETs) fits into the social and political context of tertiary education in Japan, and the assumptions about teaching underlying SETs. Through interviews with English teachers, a technical–rational perspective whereby teaching is a profession only to the extent that it involves applying theoretical knowledge to the pursuit of educational ends is discussed. Teachers’ capacities to make complex judgments are not recognized so that teaching becomes indistinguishable from technical expertise. Quantitative evaluation as a measurement of effectiveness and efficiency leads to teacher exclusion from dialogue on how teaching might be improved.