Student Evaluations Of Teaching Effectiveness Research Articles

How has teaching effectiveness been conceptualized? Questioning the consistency between definition and measure

Teaching effectiveness (TE) research impacts educators and their careers, learners and the quality of education they receive, and administrators and the organizations they safeguard. However, there is a lack of consistency in how TE has been conceptualized: many papers used inaccurate or implied definitions of TE, or despite discussing and often measuring TE, did not define TE—other papers defined TE without proposed measures or provided measures of the concept without defining it. We found two dimensions of TE, student-focused (outcome) and educator-focused (input), and an existing TE definition evaluated as the strongest for both dimensions. Further, TE measurements may be summarized in five categories: student evaluation of teaching effectiveness (SETE), objective measures, peer review, administrative evaluation, and self-reflection. To conceptualize TE, our findings suggest pairing the TE student-focused construct with SETE and objective measures, while the educator-focused dimensions of TE should be measured with peer review, self-assessment, and administrator evaluation. By consistently conceptualizing TE, researchers may contribute to rigorous research and work together to consistently add to the body of knowledge, thus furthering the quality of TE research.

Gender penalties and solidarity — Teaching evaluation differentials in and out of STEM

Employing a comprehensive, ten-year data set of class-level observations and institutional data from a large US university, we explore gender disparities in STEM fields by isolating differences in student evaluations of teaching effectiveness (SETEs). Faculty teaching face-to-face or hybrid classes receive significantly higher scores on average when compared to themselves online — a premium that is independent of gender. In STEM fields, the face-to-face premium for males is double that for females. The gender composition of the class matters. As the percentage of female students increases, SETE scores increase on average for females teaching in STEM fields but not for males. These results have implications for the ongoing problem of retention of women in STEM fields.

Refocusing on the Traditional and Effective Teaching Evaluation: Rational Thoughts About SETEs in Higher Education

Some higher educational institutions often use a student evaluation of teaching effectiveness (SETE) as the only way to evaluate teaching. Unfortunately, this instrument often fails to serve as a tool for improving instruction. It often serves as a disincentive to introducing rigor. Studies have found that student feedback is not enough to be the basis for evaluating teaching. This paper performs a literature review of student evaluations to measure teaching effectiveness. Problems are highlighted, and suggestions are offered to improve SETEs and refocus teaching effectiveness on outcome-based academic standards.

STUDENT EVALUATION OF TEACHING EFFECTIVENESS: AN INVESTIGATION INTO TEACHERS' AND STUDENTS' PERCEPTIONS

STUDENT EVALUATION OF TEACHING EFFECTIVENESS: AN INVESTIGATION INTO TEACHERS' AND STUDENTS' PERCEPTIONS

Are Grades Inflated for Good Teaching Evaluations? Evidence From Bangladesh

The hazard of excessive reliance on using student evaluations of teaching effectiveness, namely, teaching evaluations, has been found to be a contributing factor towards grade inflation. As performance evaluation of faculty members are largely dependent on these teaching evaluations, there is a significant incentive for course instructors to give higher, potentially inflated, grades to students in return for favorable teaching evaluations as positive reciprocity. The likelihood of grade inflation is further exacerbated for part-time and adjunct faculty members, who rely more on good teaching evaluations than their full-time compatriots. This study investigated the question of whether the use of teaching evaluations has been a contributing factor towards grade inflation at a certain private university in Bangladesh. Analyzing the data for 98,362 student evaluations of more than 3,653 course sections, taught by more than 350 faculty members, this study examined the relationship between teaching evaluation and average class grade for courses offered during Spring 2012, Summer 2012, and Fall 2012 semesters at this university. As a novel econometric analysis in such research, instrumental variable estimation is undertaken, whereby statistically significant results indicate a positive relationship between teaching evaluation and average class grade for courses, after controlling for latent variables, such as teaching quality and other independent variables. The conclusion that higher grades beget high evaluation scores regardless of teaching quality indicates grade inflation for ensuring good teaching evaluation.

Evaluating the validity of Student Evaluation of Teaching Effectiveness (SET) in India

Purpose – There is a debate in literature about the generalizability of the structure and the validity of the measures of Student Evaluation of Teaching Effectiveness (SET). This debate spans the dimensionality and validity of the construct, and the use of the measure for summative and formative purposes of teachers valuation and feedback. The purpose of this paper is to contribute to the debate on the aforementioned issues. Specifically the paper tests the relationship of teacher’s “charisma” trait with a measure of SET consisting of the two dimensions of “lecturer ability” and “module attributes.” The market characteristics of the paper are those of an emerging market and cross-cultural context with a specific reference to India. Design/methodology/approach – In this study, a two-dimensional scale of SET, which was originally developed by Shevlin et al. (2000) in their study in the UK, was empirically tested with Indian students and modified. Empirical data were collected from Indian students pursuing their MBA program in a north Indian university and statistical testing using exploratory and confirmatory factor analyses was undertaken. The proposed relationship of a teacher’s “charisma” trait was tested as a reflective construct comprising of the two dimensions of SET with the help of the software package Amos ver 4.0. Findings – The results indicate that the measure of SET is influenced by the teacher’s “Charisma” (trait), thus providing evidence of a halo effect. This raises the issue of validity of SET as an instrument for measuring teaching effectiveness (TE). The results provide support to the hypothesis that structure of SET is multidimensional along with the need for adapting the instrument in diverse cultural and market contexts. Originality/value – This study contributes to the debate on the validity, structure and use of SET as an instrument for measuring TE in a developing market with cross-cultural implications such as India.

Faculty Research Productivity and Standardized Student Learning Outcomes in a University Teaching Environment: A Bayesian Analysis of Relationships

This study examines whether faculty research productivity is associated with student learning. Unlike previous studies that define learning by student evaluations of teaching effectiveness, the authors specifically measure teaching effectiveness by a standardized student learning outcome measure developed by a School of Business at a US university. Using a Bayesian data reduction algorithm and regression techniques, it was found that faculty research activity is positively and significantly related to teaching effectiveness.

Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses

In this study we investigate the underlying relational structure between student evaluations of teaching effectiveness (SETEs) and achievement of student learning outcomes in 116 business related courses. Utilizing traditional statistical techniques, a neural network analysis and a Bayesian data reduction and classification algorithm, we find little or no support for the validity of SETEs as a general indicator of teaching effectiveness or student learning. In fact, the underlying structure appears to be non-linear and possibly negatively bimodal where the most effective instructors are within the middle percentiles of student course ratings, while instructors receiving ratings in the top quintile or the bottom quintile are associated with significantly lower levels of student achievement.

The Ethics Of Setting Course Expectations To Manipulate Student Evaluations Of Teaching Effectiveness In Higher Education: An Examination Of The Ethical Dilemmas Created By The Use Of SETEs And A Proposal For Further Study And Analysis

Because student evaluations of teaching effectiveness (SETEs) are an important and widely used tool used in the evaluation and reward systems for faculty members in higher education, a discussion and analysis of the ethical problems that may arise as a result of the conflict created by expectations of performance is provided. This discussion specifically focuses on ethical issues related to setting course expectations and attendance policies to manipulate students’ perceptions of course rigor and the overall evaluation of the course and the instructor.

Comparing the Effectiveness of Intensive and Traditional Courses

The present study investigated differences in the effectiveness of instructors from a variety of departments who taught the same course in both intensive and traditional formats within the same year, while controlling for many confounding variables. Results indicated that intensive courses did not significantly differ from traditional courses in overall instructor ratings on student evaluations of teaching effectiveness when confounding variables were taken into account. Conversely, intensive courses received significantly higher overall course ratings on student evaluations than did traditional courses, even after controlling for class size and probable grade in course. These findings provide further evidence that negative beliefs concerning intensive courses may be unjustified, and intensive courses may be as or more effective than those presented in traditional formats.

Do examinations influence student evaluations?

This paper measures the impact of timing on student evaluations of teaching effectiveness, using a dataset of close to 3000 observations from Erasmus School of Economics. A special feature of the data is that students were able to complete on-line questionnaires during a time window ranging from one week before to one week after the final examination. This allows for the isolation of the effect of the examination on student evaluations. Among students who subsequently pass the exam, we find little difference between pre- and post-exam ratings. Among students who fail, evaluation scores are significantly lower after the exam on a number of items. Our evidence is compatible with a self-serving bias in student evaluations, but does not indicate that students seek revenge on instructors through lower ratings.

Counseling Students' Decision Making Regarding Teaching Effectiveness: A Conjoint Analysis

The authors examined the relative importance of 4 attributes of decision making for student evaluation of teaching effectiveness: perceived knowledge base of the professor, professor's delivery style, course organization, and course workload. Participants were 234 counseling graduate students from 6 midwestern universities in the United States. Data were analyzed with classic conjoint analysis using an additive model with full‐profile method. Results indicated that decisions about teaching effectiveness are complex and based on multiple attributes. There was sufficient variability in the decision‐making process to warrant caution in overgeneralizing results. Implications for faculty development and faculty hiring are considered.

An improvement in instructional quality: can evaluation of teaching effectiveness make a difference?

PurposeTo report study findings on teaching effectiveness and feedback mechanisms in Kenyan universities, which can guide management in developing a comprehensive quality control policy.Design/methodology/approachThe study adopted an exploratory descriptive design. Three public and two private universities were randomly selected to participate in the study. A random sampling procedure was also used to select 79 respondents to participate in the research. A questionnaire administered in all participating universities was the main instrument for data collection.FindingsThere was no clear university policy on the evaluation of teaching effectiveness, despite its importance in quality control. Student evaluation of teaching effectiveness (SETE) was found to be unreliable, although widely used where evaluation existed, without other evaluation support systems. Feedback from the evaluation, though crucial in professional improvement, was not made available to the respondents.Research limitations/implicationsThe study examined the evaluation of teaching effectiveness from the lecturers' perspectives. Further research may provide insights into the contribution of SETE to teaching effectiveness from the students' standpoint.Practical implicationsUse of a variety of evaluation tools (e.g. self, peer) rather than relying solely on SETE is necessary. Comprehensive and usable information may be provided for effective teaching. Universities should provide clear policy guidelines on quality control for faculties to develop multiple teaching effectiveness evaluation instruments.Originality/valueTeaching evaluation is important in order to bring about an improvement in areas such as student achievement, and use of public funds or educational materials. The findings provide critical information for management decision making to assist universities to translate the resources at their disposal into learning outcomes.

The Relative Merits of Different Types of Overall Evaluations of Teaching Effectiveness

An unresolved issue in student evaluations of teaching effectiveness (SETE) literature is what type of overall evaluation of teaching effectiveness should be used in personnel decisions. The objective of this study is to compare the merits of: (a) an overall evaluation made by students, (b) a weighted average overall evaluation with the weights determined by students, (c) a weighted average overall evaluation with the weights being determined by the individual instructors teaching their respective classes, (d) an unweighted average overall evaluation, and (e) a second-order factor that proxies for an overall evaluation. Our results indicate that: (a) all of these overall evaluations were very highly intercorrelated, (b) the unweighted and weighted average overall evaluations measured virtually the same thing, and (c) the second-order factor that served as an overall evaluation was most highly correlated with the other overall evaluations and had the advantage of being most understandable to the faculty.

Return to academic standards: a critique of student evaluations of teaching effectiveness

A student evaluation of teaching effectiveness (SETE) is often the most influential information in promotion and tenure decision at colleges and universities focused on teaching. Unfortunately, this instrument often fails to capture the lecturer’s ability to foster the creation of learning and to serve as a tool for improving instruction. In fact, it often serves as a disincentive to introducing rigour. This paper performs a qualitative (e.g. case studies) and quantitative (e.g. empirical research) literature review of student evaluations as a measure of teaching effectiveness. Problems are highlighted and suggestions offered to improve SETEs and to refocus teaching effectiveness on outcome‐based academic standards.

Student Evaluation of Teaching Effectiveness: An assessment of student perception and motivation

Over the past century, student ratings have steadily continued to take precedence in faculty evaluation systems in North America and Australia, are increasingly reported in Asia and Europe and are attracting considerable attention in the Far East. Since student ratings are the most, if not the only, influential measure of teaching effectiveness, active participation by and meaningful input from students can be critical in the success of such teaching evaluation systems. Nevertheless, very few studies have looked into students' perception of the teaching evaluation system and their motivation to participate. This study employs expectancy theory to evaluate some key factors that motivate students to participate in the teaching evaluation process. The results show that students generally consider an improvement in teaching to be the most attractive outcome of a teaching evaluation system. The second most attractive outcome was using teaching evaluations to improve course content and format. Using teaching evaluations for a professor's tenure, promotion and salary rise decisions and making the results of evaluations available for students' decisions on course and instructor selection were less important from the students' standpoint. Students' motivation to participate in teaching evaluations is also impacted significantly by their expectation that they will be able to provide meaningful feedback. Since quality student input is an essential antecedent of meaningful student evaluations of teaching effectiveness, the results of this study should be considered thoughtfully as the evaluation system is designed, implemented and operated.

COLLEGE STUDENTS' EVALUATIONS OF TEACHING AND GRADE INFLATION

This study investigated the question: Has the use of student evaluations of teaching effectiveness been a contributing factor to a trend of grade inflation in a mid-sized, public university in the upper midwest? Aggregating the data for 983,491 student evaluations of more than 37,000 course sections, this study examined the trends in the percentage of students expecting the grades of A or A− and students' composite ratings of teaching effectiveness in courses offered between 1980 and 1999. Statistically significant second order polynomial trends over 40 semesters were found for the percentage of A/A− grades and for students ratings of teaching effectiveness. The percentage of students expecting A/A− grades increased steadily by a total of more than 10 percentage points during the 1990s after remaining stable during the 1980s. Student ratings of teaching gradually, but steadily, increased by more than one-tenth of a point after remaining relatively stable during the first half of the 1980s. The predictive relationship between student ratings of teaching and expected grades was significant even after variables related to alternative explanations were statistically controlled. The conclusion that, although generally valid as measures of teaching effectiveness, college students' ratings of instruction may be used in ways that raise questions of consequential validity, specifically by encouraging grade inflation, was supported.

Students’ Evaluation of Teaching Effectiveness: Some Implications of Stability

Previous research has indicated that the mean scores of student evaluation of teaching effectiveness do not change with experience. If this is true, whatever aspects of effective instruction the instruments are measuring do not change in the students’ perception. University students were asked to indicate what instructional characteristics changed or stayed the same as an instructor gained more and more experience. Instructor structural-related factors such as organization, knowledge, fairness, and even the ability of the students to learn were perceived as improving over time. Personalityrelated factors were perceived as stable. Implications of these findings are discussed.

Uses and misuses of student evaluations of teaching effectiveness

Uses and misuses of student evaluations of teaching effectiveness

Midterm Evaluation of Teaching Provides Helpful Feedback to Instructors

Advantages of using student evaluation of teaching effectiveness at midterm are discussed. They include receiving information on microbehaviors of teaching instead of global ratings, getting feedback that can be used immediately, and bypassing some of the controversies associated with end-of-course departmental evaluations. Procedures for constructing questionnaires for a particular teaching situation are described, as are tips for reporting the data to the class. In addition to giving the instructor useful, non-threatening feedback, midterm evaluations may favorably alter the attitudes of the students toward the instructor and the teaching process.