Online Student Evaluations Of Teaching Research Articles

Student motivations, perceptions and opinions of participating in student evaluation of teaching surveys: a scoping review

Several times each year the teaching performance of academics at higher education institutions are evaluated through anonymous, online student evaluation of teaching (SET) surveys. Universities use SETs to inform decisions about staff promotion and tenure, but low student participation levels make the surveys impractical for this use. This scoping review aims to explore student motivations, perceptions and opinions of SET survey completion. Five EBSCO® databases were searched using key words. Thematic analysis of a meta-synthesis of qualitative findings derived from 21 papers identified five themes: (i) the value students’ place on SET, (ii) the knowledge that SET responses are acted upon to improve teaching, (iii) assurance of survey confidentiality and anonymity, (iv) incentives for completing SET, and (v) survey design and timing of survey release. Perceptions, knowledge and attitudes about the value of SET are essential factors in motivating students to engage and complete SETs, particularly if surveys are easy to interpret, time for completion is incentivised and responses are valued.

Online Parameter Estimation for Student Evaluation of Teaching.

Student evaluation of teaching (SET) assesses students' experiences in a class to evaluate teachers' performance in class. SET essentially comprises three facets: teaching proficiency, student rating harshness, and item properties. The computerized adaptive testing form of SET with an established item pool has been used in educational environments. However, conventional scoring methods ignore the harshness of students toward teachers and, therefore, are unable to provide a valid assessment. In addition, simultaneously estimating teachers' teaching proficiency and students' harshness remains an unaddressed issue in the context of online SET. In the current study, we develop and compare three novel methods-marginal, iterative once, and hybrid approaches-to improve the precision of parameter estimations. A simulation study is conducted to demonstrate that the hybrid method is a promising technique that can substantially outperform traditional methods.

The effect of insufficient effort responding on the validity of student evaluation of teaching

PurposeThe purpose of this paper is to investigate the effect of insufficient effort responding (IER) on construct validity of student evaluations of teaching (SET) in higher education.Design/methodology/approachA total of 13,340 SET surveys collected by a major Jordanian university to assess teaching effectiveness were analyzed in this study. The detection method was used to detect IER, and the construct (factorial) validity was assessed using confirmatory factor analysis (CFA) and principal component analysis (PCA) before and after removing detected IER.FindingsThe results of this study show that 2,160 SET surveys were flagged as insufficient effort responses out of 13,340 surveys. This figure represents 16.2 percent of the sample. Moreover, the results of CFA and PCA show that removing detected IER statistically enhanced the construct (factorial) validity of the SET survey.Research limitations/implicationsSince IER responses are often ignored by researchers and practitioners in industrial and organizational psychology (Liu et al., 2013), the results of this study strongly suggest that higher education administrations should give the necessary attention to IER responses, as SET results are used in making critical decisionsPractical implicationsThe results of the current study recommend universities to carefully design online SET surveys, and provide the students with clear instructions in order to minimize students’ engagement in IER. Moreover, since SET results are used in making critical decisions, higher education administrations should give the necessary attention to IER by examining the IERs rate in their data sets and its consequences on the data quality.Originality/valueReviewing the related literature shows that this is the first study that investigates the effect of IER on construct validity of SET in higher education using an IRT-based detection method.

Why bother? Student motivation to participate in student evaluations of teaching

When response rates on student evaluations of teaching (SETs) are low, inability to properly interpret and use responses from the students who do participate is a big problem. Where does the motivation to participate break down, and where and how does it make sense to invest efforts in rectifying that? In this study, we examined 641 university students’ reported behaviours and motivation related to their SET participation. In terms of behaviour, students who seldom or never participate in online SET tools reported a willingness to invest, at most, five minutes in the process, though the majority never even open the online evaluation links when they receive them. In terms of motivation, they significantly differed from students who always participate with distinctly lower levels of: (1) willingness to participate at all, (2) perception of autonomy and competence, (3) meaningfulness, (4) personal value, (5) engagement in others’ participation, and (6) understanding of the value of their own participation for others’ benefit. Based on these findings, we propose a strategy for increasing future response rates, particularly among recalcitrant students, in order to be able to gather sufficient and reliable results for the work of improving teaching.

Student evaluations of teaching: the impact of faculty procedures on response rates

In the last 10–15 years, many institutions of higher education have switched from paper-and-pencil methods to online methods of administering student evaluations of teaching (SETs). One consequence has been a significant reduction in the response rates to such instruments. The current study was conducted to identify whether offering in-class time to students to complete online SETs would increase response rates. A quasi-experiment (nonequivalent group design) was conducted in which one group of tenured faculty instructed students to bring electronic devices with internet capabilities on a specified day and offered in-class time to students to complete online SETs. A communication protocol for faculty members’ use was developed and implemented. A comparison group of tenured faculty who did not offer in-class time for SET completion was identified and the difference-in-differences method was used to compare the previous year’s response rates for the same instructor teaching the same course across the two groups. Response rates were substantially higher when faculty provided in-class time to students to complete SETs. These results indicate that high response rates can be obtained for online SETs submitted by students in face-to-face classes if faculty communicate the importance of SETs in both their words and actions.

Improving SET Response Rates: Synchronous Online Administration as a Tool to Improve Evaluation Quality

Institutions of higher education continue to migrate student evaluations of teaching (SET) from traditional, in-class paper forms to online SETs. Online SETs would favorably compare to paper-and-pencil evaluations were it not for widely reported response rate decreases that cause SET validity concerns stemming from possible nonresponse bias. To combat low response rates, one institution introduced a SET application for mobile devices and piloted formal synchronous classroom time for SET completion. This paper uses the Leverage Salience Theory to estimate the impact of these SET process changes on overall response rates, open-ended question response rates, and open end response word counts. Synchronous class time best improves SET responses when faculty encourage completion on keyboarded devices and provide students SET completion time in the first 15 min of a class meeting. Full support from administrators requires sufficient wireless signal strength, IT infrastructure, and assuring student access to devices for responses clustering around meeting times.

Honesty on student evaluations of teaching: effectiveness, purpose, and timing matter!

Student evaluations of teaching (SETs) are an important point of assessment for faculty in curriculum development, tenure and promotion decisions, and merit raises. Faculty members utilise SETs to gain feedback on their classes and, hopefully, improve them. The question of the validity of student responses on SETs is a continuing debate in higher education. The current study uses data from two universities (n = 596) to determine whether and under what conditions students are honest on in-class and online SETs, while also assessing their knowledge and attitudes about SETs. Findings reveal that, while students report a high level of honesty on SETs, they are more likely to be honest when they believe that evaluations effectively measure the quality of the course, the results improve teaching and benefit students rather than the administration, and when they are given at the end of the term. Honesty on evaluations is not associated with socio-demographic characteristics.

The Causal Effect of Survey Mode on Students’ Evaluations of Teaching: Empirical Evidence from Three Field Experiments

In recent years many universities switched from paper- to online-based student evaluation of teaching (SET) without knowing the consequences for data quality. Based on a series of three consecutive field experiments - a split-half design, twin courses, and pre–post-measurements - this paper examines the effects of survey mode on SET. First, all three studies reveal marked differences in non-response between online- and paper-based SET and systematic, but small differences in the overall course ratings. On average, online SET reveal a slightly less optimistic picture of teaching quality in students’ perception. Similarly, a web survey mode does not impair the reliability of student ratings. Second, we highlight the importance of taking selection and class absenteeism into account when studying survey mode effects and also show that it is necessary and informative to survey the subgroup of no-shows when evaluating teaching. Third, we empirically demonstrate the need to account for contextual setting of the survey (in class vs. after class) and the specific type of the online survey mode (TAN vs. email). Previous research either confounded contextual setting with variation in survey mode or generalized results for a specific online mode to web surveys in general. Our findings suggest that higher response rates in email surveys can be achieved if students are given the opportunity and time to evaluate directly in class.

Student and faculty perceptions of student evaluations of teaching in a Canadian pharmacy school

ObjectiveThis qualitative study investigated motivators and barriers to student and faculty engagement with an online student evaluation of teaching (SET) process. MethodsSemi-structured interviews were conducted with 13 students, who self-identified as either “completers” or “non-completers” of SETs, and 12 faculty members. Interview transcripts were coded and analyzed thematically. ResultsStudents were motivated to complete SETs when they perceived that results would be used and/or considered by instructors. Timing and number of surveys presented barriers to student completion. Faculty members were motivated to engage with SETs when the response rate was high and when senior administrators acknowledged survey results. ConclusionsImplementing processes whereby students are assured that instructors have read and considered their feedback may improve student engagement. Faculty members’ engagement may be augmented when they better understand what an adequate response rate is for their given class size. Senior administrators must regularly acknowledge and discuss SETs with faculty members to confirm their importance in academic careers.

Transition from Paper to Online Course Evaluation: Preliminary Trends in Student Response Rate and Overall Professor Evaluation

Student evaluation of teaching (SET) hasbeen used as a metric to arguably evaluate instructoreffectiveness and quality of instruction since the 1920s.SET is used in decisions regarding annual evaluation (offaculty) and is one of the most researched topics inevaluation of instructor effectiveness. Central researchquestions associated with SET include whether SET is anappropriate measure of effectiveness and whether it leadsto improved teaching and quality of graduates.In the fall 2013, the Faculty of Engineering at theUniversity of Alberta for the first time administered SETonline. The transition from paper-based and in class SETto online and out of class SET provides a uniqueopportunity to investigate changes in SET response rateand ratings of overall instructor effectiveness that couldbe attributed to change in protocol and that wouldsuggest protocol-related bias. Our preliminary resultsshow lower response rates for online SET andeffectiveness scores that were outside one standarddeviation of the previous 5-year mean. These findingsshow the importance of continuing to monitor web-basedSET results and point to directions of further research. Atthis time, this will not be possible, as the Faculty ofEngineering has discontinued online SET testing after asingle term.

Expectancy theory outcomes and student evaluations of teaching

As student evaluation of teaching (SET) instruments are increasingly administered online, research has found that the response rates have dropped significantly. Validity concerns have necessitated research that explores student motivation for completing SETs. This study uses Vroom's [(1964). Work and motivation (3rd ed.). New York, NY: John Wiley & Sons] expectancy theory to frame student focus group responses regarding their motivations for completing and not completing paper and online SETs. Results show that students consider the following outcomes when deciding whether to complete SETs: (a) course improvement, (b) appropriate instructor tenure and promotion, (c) accurate instructor ratings are available to students, (d) spending reasonable amount of time on SETs, (e) retaining anonymity, (f) avoiding social scrutiny, (g) earning points and releasing grades, and (h) being a good university citizen. Results show that the lower online response rate is largely due to students’ differing feelings of obligation in the 2 formats. Students also noted that in certain situations, students often answer SETs insincerely.

Non-response bias in student evaluations of teaching

For as long as colleges and universities have been conducting student evaluations of teaching (SET), faculty have questioned the validity of the information collected. Substantial effort has been expended to improve the SET process, and researchers have communicated a consistent understanding of why students evaluate teachers as they do. Most of these conclusions have been based on an analysis of SET data gathered at the end of the semester by a sample of students who may not represent all students enrolled in the class. This clearly creates the potential for a sample selection bias that puts into question much of what we have learned about why students evaluate their instructors as they do.In this paper, we gather a unique data set from individual students who provide either online SET forms or in-class SET forms. We use these responses to address the concerns regarding the validity of SET data which results from the low response rates often associated with online SET evaluations. The Heckman (1979) sample selection bias correction estimator is used to quantify the bias resulting from the low online response rate and finds this bias to be negligibly small.

On the Validity of Student Evaluation of Teaching

This article provides an extensive overview of the recent literature on student evaluation of teaching (SET) in higher education. The review is based on the SET meta-validation model, drawing upon research reports published in peer-reviewed journals since 2000. Through the lens of validity, we consider both the more traditional research themes in the field of SET (i.e., the dimensionality debate, the ‘bias’ question, and questionnaire design) and some recent trends in SET research, such as online SET and bias investigations into additional teacher personal characteristics. The review provides a clear idea of the state of the art with regard to research on SET, thus allowing researchers to formulate suggestions for future research. It is argued that SET remains a current yet delicate topic in higher education, as well as in education research. Many stakeholders are not convinced of the usefulness and validity of SET for both formative and summative purposes. Research on SET has thus far failed to provide clear answers to several critical questions concerning the validity of SET.

Nonresponse and Online Student Evaluations of Teaching: Understanding the Influence of Salience, Fatigue, and Academic Environments

Technological advances have enabled institutions of higher education to administer course evaluations online, forgoing the traditional paper-and-pencil methods. Consequently, many of these institutions suffer from low response rates, but little research is available on this topic. To increase understanding about course evaluation participation in the online environment, this study examined over 22,000 undergraduates to whom the university administered about 135,000 evaluations. Multilevel models were constructed to analyze the data, and several variables emerged as significant predictors of participation. The results were mostly consistent with previous research and aligned with theories of survey nonresponse. However, the integration of uncommon variables provided new perspectives about course evaluations in particular. Implications for research and practical applications for institutions are also addressed, including ways to combat survey fatigue, increase the salience of the survey, and increase participation in online course evaluations.

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In‐depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface‐level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.

Teacher Evaluation: Teachers’ Reflections and Actions on online Student Evaluations of Teaching

Teacher Evaluation: Teachers’ Reflections and Actions on online Student Evaluations of Teaching

Online Student Evaluation of Teaching for Distance Education: A Perfect Match?

Given the limited number of currently available resources, one public mid-western university is working to develop and implement an effective and appropriate means for online student evaluation of teaching in distance education courses that is useful and beneficial for all stakeholders — student, faculty, and administration. Because traditional methods have proven insufficient in addressing the breadth of instructional delivery and course design methods, the university began a pilot project supporting online evaluation for distance education courses in spring 2003. This paper outlines the issues considered and challenges faced in the search to employ reliable and valid online student evaluation for distance education courses, to maximize the benefits associated with electronic data gathering and reporting, and to meet the organizational and logistical challenges inherent in such an endeavor.