Student Evaluations Of Teaching Research Articles

Evaluating student satisfaction - restricting lecturer professionalism: outcomes of using the UK national student survey questionnaire for internal student evaluation of teaching

In the neo-liberal context of a UK university, responding to student feedback in order to raise student satisfaction levels is important in improving National Student Survey (NSS) scores. This article focuses on the impact of a UK university’s new student feedback questionnaire - for individual modules - which used the NSS questions. The research draws on survey data (N = 101) to identify lecturers’ views and three student focus groups. The outcomes raised issues relating to performativity, professionalism and ‘provision’, the latter defined as the university’s contract with each student, including the aspects that affect the student learning experience but are beyond the lecturers’ control, for example, class sizes and timetables. The results indicate that by recognising the impact of provision university managers may be better able to develop systemic improvements to student experience and (in the UK) a corresponding uplift in NSS and Teaching Excellence Framework (TEF) results. The article puts forward a model linking performativity, professionalism and provision to the relationships between university managers, academics and students. This model could enrich understandings of professionalism and performativity, extend the range of issues affecting student experience in SETs and support data analysis in future research studies.

What Do Teachers Think About Their Students' Inclusion? Consistency of Students' Self-Reports and Teacher Ratings.

The aim of this study was to investigate the consistency between the self-reports and teacher ratings of students’ emotional and social inclusion at school as well as for their academic self-concept. The German version of the Perceptions of Inclusion Questionnaire (PIQ) was administered to 329 grade 8 students (50.8% female, Mage = 14.5 years, SDage = 0.5 years) and their teachers. First, the three-dimensional structure of both PIQ versions was confirmed by confirmatory item factor analysis. The α and ω coefficients demonstrated good reliability for all scales. Second, a correlated trait-correlated method minus one model provided evidence that the method-specificity of teacher ratings was larger than the consistency between the self-reports and teacher ratings. Third, the results of a latent difference model indicated that general method effects can partly be explained by a student’s gender or special educational needs. Finally, the low consistency between self-reports and teacher rating is discussed.

Technical Adequacy of Procedural and Conceptual Algebra Screening Measures in High School Algebra

Although algebra often functions as a gatekeeper to higher-level mathematics courses and higher education admissions, few quality measures exist for assessing conceptual understanding. This study explored the technical adequacy of three procedural and two conceptual algebra screening measures. We administered three rounds of assessments throughout an algebra course to 2,021 secondary students of 31 teachers in three states. Assessments included procedural and conceptual screening measures and additional criterion measures: teacher ratings of students’ algebra proficiency, course grades, results of two project-specific algebra proficiency exams, and state test scores. Descriptive and correlation analyses were used to investigate measure scores, alternate-form and test–retest reliability, concurrent validity, and predictive validity. Procedural measure results indicated high levels of reliability ( r = .72–.99), and moderate concurrent and predictive validity ( r = .36–.64; .36–.58). The conceptual measures produced moderate to low levels of validity ( r = .10–.44). The procedural measure results suggest they may be suitable for use as screening measures, pending further revision and diagnostic testing, while the conceptual measures did not produce acceptable results for current implementation. The findings contributed to measure redesigns to bolster their use as mathematics proficiency assessments with algebra students.

Student Evaluations of Teaching: Is There a Relationship between Student Feedback on Teaching and the Student Final Grade?

The use of Student Evaluations of Teaching (SET) has become widespread practice in higher education despite inconclusive evidence reported in the literature around its validity. Not surprisingly, the question of the validity of SET continues to be a current debate in higher education, pointing to more research to be conducted in this area. The current study contributes to broadening knowledge and understanding on the validity of SET by drawing on an online unit evaluation completed by students (n=2430 out of total student enrolment of N=7757) in one university across three postgraduate education programs over a two-year period, to determine whether there is a relationship between student feedback on teaching and student final unit grade. Findings revealed that students who achieved very high or very low final unit grades did not participate in the SET, while students who achieved Pass or Credit grades partook in the SET, thus providing feedback. This indicates that teaching and evaluating staff need to be aware that a large subset of their students that are not providing feedback to staff to improve the quality of their courses.

Twelve Years a Servant: Race and the Student Evaluation of Teaching

The end of year student course evaluations (SETs), the dreaded final act of a semester of teaching and learning that serves the “supposed” purpose of evaluating a course and its instructor. Various studies have already shown how SETs are ineffective in serving such a purpose, yet SETs are still used for major decisions such as tenure and promotion, and annual merit pay increases. Further, the fact that SETs are rife with racial, gender, and ethnic bias towards instructors is equally troubling. The aim of this reflexive essay is to highlight a form of racialized intellectual violence that is heaped upon faculty of color based on a content analysis of the author’s negative and potentially racially motivated SETs over the span of a 12-year career. The outcomes of such a reflexive essay is to bring awareness and highlight the professional and psychological impacts that make teaching in/and of color in academia a precarious endeavor, as well as problematizing SETs use as a true evaluation of teaching effectiveness.

Conflict of interest explains the size of student evaluation of teaching and learning correlations in multisection studies: a meta-analysis.

We examined the associations between the size of student evaluation of teaching and learning (SET/learning) correlations and presence of several conflicts of interest (COIs) including corporate, administrative, evaluation unit, SET author, and funder interests. Our meta-analyses of SET/learning correlations reported by multisection studies show that researchers with a vested interest in finding large positive SET/learning correlations found, on average, large positive SET/learning correlations. In contrast, researchers with no identifiable COIs found that SET/learning correlations were zero or nearly zero. The largest SET/learning correlations were reported by authors with ties to SET selling corporations. Smaller but still substantial SET/learning correlations were reported by researchers with administrative assignments and by researchers in evaluation units/departments responsible for the administration of SET. Moreover, authors with the most significant COIs were publishing their studies primarily prior to 1981 whereas authors with no or less significant COIs were publishing their studies in 1981 or afterwards. Studies published prior to 1981 reported small but significant (r = .31) SET/learning correlations whereas studies published in 1981 and after reported near zero, non-significant SET/learning correlations (r = .06). The presence of COIs was associated with earlier publication date but also with smaller samples. Finally, whereas corporate, administrative, and evaluation unit authors nearly ceased publishing multisection studies on SET/learning correlations, authors from business and economics departments are now responsible for the substantial portion of newer, larger, and higher quality studies published in 1981 and after.

Converting Likert Scales Into Behavioral Anchored Rating Scales(Bars) For The Evaluation of Teaching Effectiveness For Formative Purposes

Likert scales traditionally used in student evaluations of teaching (SET) suffer from several shortcomings, including psychometric deficiencies or ambiguity problems in the interpretation of the results. Assessment instruments with Behavioral Anchored Rating Scales (BARS) offer an alternative to Likert-type questionnaires. This paper describes the construction of an appraisal tool with BARS generated with the participation of 974 students and 15 teachers. The resulting instrument eliminates ambiguity in the interpretation of results and gives objectivity to the evaluation due to the use of unequivocal behavioral examples in the final scale. However, BARS methodology presents the problem of losing behavioral information during scale construction. The BARS methodology presented by the authors introduces an additional step to the traditional procedure, which significantly reduces the loss of information during the scale construction. The authors conclude that the qualitative approach of the proposed instrument facilitates the application of the formative function of the evaluation.

Does the Flipped Classroom Improve Student Learning and Satisfaction? A Systematic Review and Meta-Analysis

We searched and meta-analyzed studies comparing flipped classroom teaching with traditional, lecture-based teaching to evaluate the evidence for the flipped classroom’s influence on continuous-learning measures, pass/fail rates, and student evaluations of teaching. Eight electronic reference databases were searched to retrieve relevant studies. Our results indicate a small effect in favor of the flipped classroom on learning (Hedges’ g = 0.35, 95% confidence interval [CI] [0.31, 0.40], k = 272). However, analyses restricted to studies with sufficient power resulted in an estimate of 0.24 (95% CI [0.18, 0.31], k = 90). Effects on pass rates (odds ratio = 1.55, 95% CI [1.34, 1.78], k = 45) and student satisfaction (Hedges’ g = 0.16, 95% CI [0.06, 0.26], k = 69) were small and also likely influenced by publication bias. There is some support for the notion that the positive impact on learning may increase slightly if testing student preparation is part of the implementation.

Adapting Law Lectures to Maximise Student Engagement: Is It Time to 'Transform'?

Technological advances have undoubtedly brought significant benefits to students in higher education. Classes can be recorded for students who are unable to attend, communication between students and staff can be facilitated more easily online, and web-based tools and resources improve learning and university efficiency. However, this has not come without a cost. Class attendance has long been considered a significant issue in higher education, and has dramatically declined in the past decade. This is especially prominent in lectures, which have traditionally focused on information transfer from an instructor to their students without requiring students to think critically or respond. This is in contrast to tutorials and other workshop-style classes that are heavily problem solving-based. As Huff notes, this traditional style of lecturing often leads only to undeveloped information transfer from the lecturer to the student ‘without passing through the brains of either’. In such circumstances, lecture recordings capture most important information that students require, with significant resulting impact on attendance rates. In a study of a psychology class, Grabe, Christopherson, and Douglas found that 61 per cent of voluntary absences from class can be attributed to the availability of online lecture notes. A University of Western Australia (UWA) study conducted by Mascher and Skead found that 80 per cent of students attend classes because they do not want to miss something not picked up on a recording or because a class is not recorded. Dwindling student attendance is a major teaching and learning concern of higher education institutions worldwide. For many years, studies have shown that low attendance rates result in a significantly less effective teaching and learning environment. Higher student attendance directly corresponds with more positive student evaluation of teachers, and with higher grades and deeper learning. Collett, Gyles, and Hrasky found that a class taught fully online had significantly lower performance on multiple-choice tests when compared to a class taught through a combination of online and in- class instruction. Furthermore, if lectures fail to engage students or even draw them to the university campus, then this will have significantly adverse implications for traditional campus-based universities in an increasingly globalised, online, and competitive higher education market. Law schools face their own particular challenges, too. Law students are expected to develop the skills to critically interpret, apply, analyse, and comment on legal issues and principles. However the traditional, ‘one-way’ information transfer style of lecturing is far more adept to rote learning than to developing skills in critical analysis. How, then, can universities respond to this problem? One solution is shifting to ‘transformative’ teaching, whereby students’ learning and experience is ‘transformed’ and dynamic relationships between teachers, students, and a shared body of knowledge are created in order to promote student learning and personal growth. Transformative teaching and learning has its roots in the late-twentieth century, but has achieved significant traction in higher education literature in the past 10 years. Transformative teaching methods are designed to substantially increase student engagement and interest through transforming, or altering, students’ outlook or learning styles. As lectures, the most fundamental educational tool in legal education, ordinarily contain few transformative aspects, this style of class is ripe for such transformation. This article outlines an empirical study of students, staff, and executives at the UWA Law School to determine preferences for transformative teaching techniques and how they would best improve engagement and enjoyment in a lecture environment. Through a mix of student-centred learning, problem-based learning and active learning, an adaption of lectures to include more transformative techniques will be proposed — a formative resuscitation that would not go as far as King’s suggestion in the 1970s of a ‘radical departure from the normal type of classroom teaching’. This exploratory process will assist law schools, and indeed other university faculties, in improving attendance, engagement, and learning outcomes in lectures, with the aim of contemporising and helping transform student experiences at university. An emphasis on transformative teaching will also provide support for the continued use of ‘small seminar-style’ classes that are already utilised in many Australian law schools. Following an examination of relevant transformative teaching literature in Part II and a brief methodological outline in Part III, results of the study are analysed in Part IV and a set of key recommendations for law lectures proposed in Part V.

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Panel: Peer-Based Faculty Evaluation v. Student Evaluation of Teaching

Flipped pedagogy and student evaluations of teaching

A common concern of faculty members is student evaluations of teaching scores. Many factors influence student evaluations of teaching including how the course is designed. This study investigated student evaluations of teaching across three pedagogical techniques: the traditional lecture (where lectures were not recorded), recorded lectures, and a flipped classroom model. A between-subject design was used over three semesters. Student evaluations of teaching showed that students rated the instructor significantly higher in a flipped classroom setting. Students also reported liking the course more using the flipped pedagogical technique. Since the flipped format improved both teaching evaluations and student perceptions of the class, this could be useful for instructors when they are deciding how to set up the format of their class.

Advancing text-analysis to tap into the student voice: a proof-of-concept study

Student evaluations of teaching and courses (SETs) are part of the fabric of tertiary education and quantitative ratings derived from SETs are highly valued by tertiary institutions. However, many staff do not engage meaningfully with SETs, especially if the process of analysing student feedback is cumbersome or time-consuming. To address this issue, we describe a proof-of-concept study to automate aspects of analysing student free text responses to questions. Using Quantext text analysis software, we summarise and categorise student free text responses to two questions posed as part of a larger research project which explored student perceptions of SETs. We compare human analysis of student responses with automated methods and identify some key reasons why students do not complete SETs. We conclude that the text analytic tools in Quantext have an important role in assisting teaching staff with the rigorous analysis and interpretation of SETs and that keeping teachers and students at the centre of the evaluation process is key.

“Somebody has to teach the ‘broccoli’ course”: Administrators Navigating Student Evaluations of Teaching (SET)

Student Evaluations of Teaching (SET) have been the most consistently administered tool, and they are still extensively used in higher education institutions to assess teaching effectiveness. The purpose of this study was to explore how SET are used by administrators in the teaching evaluation process at a large, research-intensive Canadian university. A basic qualitative research design was used in this project, and semi-structured interviews were used to obtain administrators’ experiences. The research question that guided this study was: How are SET (and other tools) used in the evaluation of teaching at this university? Findings showed that although participants mostly utilized a couple of SET statements as indicators of effective teaching, they were certainly aware of the intrinsic issues concerning these tools, and that they are continually seeking to obtain more evidence if SET results are below their benchmarks.

An exploration of the ‘African (Union Commission’s) perspective’ of quality and quality assurance in higher education: Latent voices in the African Quality Rating Mechanism (AQRM)

Quality assurance of African higher education is at the top of the region’s development agenda. Prompted by the imperative to enhance the quality of higher education, the Africa Union Commission is implementing the African Quality Rating Mechanism (AQRM). The AQRM is a continental tool that affords higher education institutions an opportunity to conduct self-assessment and compare their performance against similar institutions based on a set of common criteria. The mechanism is envisaged to engender institutional cultures of quality and enhance the quality of African higher education. However, a dearth of knowledge exists about the latent notions of quality in higher education that the AQRM aims to assure and the implicit institutional-level quality assurance practices in the AQRM. Therefore, this interpretivist article, based on a review of the AQRM survey questionnaire, answered the following research question: What notions of quality in higher education and the institutional-level quality assurance practices are inherent in the quality standards of the AQRM survey questionnaire? The findings revealed that quality as fitness for purpose and exceptional are the notions of quality in higher education in the AQRM. Nevertheless, fitness for purpose is the dominant notion of quality and this symbolises an imperative to re-direct higher education to serve social and economic ends. Distinguished (excellent) teacher awards, applied research excellence awards, staff professional development, tracer studies, external examination, and the involvement of key external stakeholders in programme development are the latent institutional-level quality assurance practices in the AQRM. These quality assurance practices are in sync with the notions of quality and aim at bridging the gap between the academy and the labour market. Methodologically, the AQRM survey questionnaire is devoid of benchmarks to inform the rating, and quality assurance practices such as student evaluation of teaching, peer observation of teaching and moderation of examination items are unnoticeable in the survey questionnaire.Received: 28 December 2018Accepted: 19 April 2019Published online: 29 May 2019

Mitigating gender bias in student evaluations of teaching.

Student evaluations of teaching are widely believed to contain gender bias. In this study, we conduct a randomized experiment with the student evaluations of teaching in four classes with large enrollments, two taught by male instructors and two taught by female instructors. In each of the courses, students were randomly assigned to either receive the standard evaluation instrument or the same instrument with language intended to reduce gender bias. Students in the anti-bias language condition had significantly higher rankings of female instructors than students in the standard treatment. There were no differences between treatment groups for male instructors. These results indicate that a relatively simple intervention in language can potentially mitigate gender bias in student evaluation of teaching.

Lessons Learned from the Development of Open Educational Resources at Post-Secondary Level in the Field of Environmental Modelling: An Exploratory Study

Open educational resources (OER) have become increasingly popular in recent times. Here, the aim was to synthesise the lessons learned through the development of OER materials for a university-level course called “environmental modelling”. Consequently, the topics of discussion included: (i) how to choose an appropriate creative commons license; (ii) ways of incorporating materials from other sources, such as publicly available sources, other open access materials, and an author’s own published materials if not published under a creative commons license; (iii) the impact of the developed OER in the field of environmental modelling; and (iv) the challenges in developing OER material. Upon developing the materials, we observed the following: (i) students enrolled in the course did not purchase textbooks; (ii) our OER materials ranked as one of the most accessed (i.e., number 7) materials according to the usage data that summed the number of file downloads and item views from PRISM (i.e., the hosting platform maintained by the University of Calgary); (iii) the students learned relatively better as per the data acquired by the University of Calgary’s universal student ratings of instruction (USRI) instrument; and (iv) other universities expressed interest in adopting the materials.

Factors Influencing Undergraduate Nursing Students’ Perception of Educational Quality

Context: Quality education is a contemporary concern. Students' evaluations of teaching (SET) are universal measures and applied in almost every higher educational system in the world.
 Aim: The present study aims to explore student perception of factors that influence educational quality among undergraduate nursing students.
 Methods: A survey research design was utilized to achieve the aim of this study. The study conducted at obstetrics and gynecology classroom. Faculty of Nursing, El-Minia University. The subjects of the study consisted of all 3rd year nursing students (male & female) who registered in Obstetrics & Gynecological course (total number was 125 students) during the academic year 2017- 2018. Three tools were used to achieve the aim of this study; they are student achievement record, student’s evaluation of educational quality scale, and factor affecting educational quality scale (FAEQS).
 Results: The results of this study revealed that students’ perception to the components of educational quality named learning experience, instructor rapport, enthusiasm, and tests grading, organization and clarity, theoretical breadth coverage, group interaction, and course texts and reading materials. The students also ranked course organization, examination system, student communication and group dynamics, availability of course materials, instructor performance, and academic values as top factors that could affect their educational quality.
 Conclusions: Obstetrics and gynecology students perceive factors which affected the provision of quality education were explored incorporating four categories named the organization, student, instructor, and course-related factors. The study recommended that all institutions and departments should use self-evaluation as an essential tool in striving to educational quality, and also should consider the students’ perception of factors that might affect their educational quality. Further researches are expected to shed light in the number of higher-order factors that could affect educational quality

Using Learning Analytics to Evaluate Course Design and Student Behavior in an Online Wine Business Course

This case study describes the use of learning analytics to evaluate the transition of a postgraduate wine business course from face-to-face to online delivery using e-learning course design principles. Traditionally, Foundations of Wine Science lectures were delivered face-to-face, however the decision to transition the course from semester to trimester format presented an opportunity for online delivery of lectures. This was initially achieved through audio recordings, then video lectures, supported by a range of digital learning resources intended to engage, support and enhance student learning and the student experience. Descriptive analysis of learning analytics, comprising assessment results, student evaluations of learning and teaching, and data sourced from the Learning Management System, was performed to evaluate the impact of online delivery of course content on student performance, satisfaction and engagement. The use of audio lecture recordings negatively impacted students’ perception of the overall quality of the course (including course organisation, learning strategies and learning resources). The subsequent implementation of e-learning designed video lectures was considered superior to audio recordings, albeit final grades were not significantly different between the delivery modes. However, student engagement was equal to, or better than face-to-face delivery, when content was designed specifically for an e-learning environment.