Student Evaluations Of Course Research Articles

A pedagogical evaluation of an institution's digital assessment platform (DAP): Integrating pedagogical, technical and contextual factors

An assessment for learning approach is foundational for student learning. The necessity to shift teaching and learning online as a response to COVID-19 has propelled digital assessment into the mainstream within higher education institutions. User experience is a common indicator of effectiveness of technologically enhanced initiatives; however, measuring the pedagogical impact of digital assessment initiatives has not been straightforward. This paper presents a pedagogical evaluation, incorporating staff and student perspectives, of a digital assessment platform (DAP) at a large Australian metropolitan university. Data were collected using surveys, interviews, focus groups and case studies and incorporated findings from student evaluations of courses and a student-staff partnership project. Results highlight strong pedagogical reasons for continuing the digital assessment initiative, while also identifying constraints and opportunities for improvement. The insights generated through this evaluation emphasise the importance of considering technical, pedagogical and contextual factors and the interplay between the factors. In a quality assurance climate, where the demand for this type of institution-wide evaluation is increasing, this study contributes an approach that addresses some of the complexities and challenges of evaluating digital assessment initiatives in higher education. Future research is needed to understand more fully the relationships between pedagogical and contextual factors when undertaking an evaluation. Implications for practice or policy: The DAP enabled the (re)design of authentic, inclusive, engaging and secure assessment tasks. Educator support is critical in learning the system and creating assessment with high pedagogical merit. Student familiarisation opportunities with using the DAP are beneficial. The DAP’s longevity within the institution is crucial considering the significant effort required of educators and students. Contextual influences blur understanding of pedagogical factors.

Purposes and tensions in organising knowledge: trajectories of student evaluations in two research universities

Student evaluations of courses and teaching (SETs) are used for various purposes, are omnipresent in academic teaching, and cause tensions within universities. Their analytical power is contested, and specific uses are problematised for negative effects on lecturers and academic relationships. This interpretive interview study addresses how academic actors navigate tensions and purposes, and how they use, shape and deliberate on student evaluations in practice and policy processes. It reconstructs the trajectories of student evaluations in two Dutch research universities. Twenty-one interviews were conducted with actively involved students, support staff and academics in various positions including management. Elias’ processual approach to human figurations was combined with framing analysis to understand how interviewees deal with tensions and create space to change evaluation processes. The two universities differed in their articulation of problems and policy trajectories. In both universities, programme committees and management smoothed tensions by guiding lecturers and students in adjusting behaviours towards each other. Issues persisted, and multifaceted questions like performance evaluations of lecturers were addressed only indirectly. Student evaluations operated as boundary objects, bounding specific perspectives and problems while leaving others untouched. The study invites more articulate deliberation and concerted action, especially concerning persistent negative structural effects.

Exploring Online Evaluation Development : A Bibliometric Approach

This study aims to explore the growth of online evaluation to give a comprehensive overview and identify research in the trending field and recommendations for further research that can be developed. This study used a bibliometric approach with descriptive methods from documents in the last ten years of the Scopus database. Quantitative and qualitative analyses were performed using data collected from numerous scientific reference sources to determine development trends, the most popular subjects, and the contributions of eminent scholars. The analysis’s findings indicated that student evaluations of courses and teachers were the most commonly discussed subjects. However, in subsequent years, interest had changed to e-learning, higher education, assessment, and evaluation. Recent scientific articles further emphasized the significant impact Covid-19 had had on altering the evaluation process, placing particular attention on online learning. With the advancement of research on artificial intelligence in education, this study also outlined prospects and challenges in online evaluation research. We found thirteen publications covering Chat GPT for educational purposes by obtaining data from articles published in the most recent year in the Scopus database. This study offered a comprehensive view of the evolution of online evaluation through a bibliometric method, covered pertinent topics, and provided a basis for academics to carry out the most recent research in this area.

Student Ratings and Course Modalities

This article examines the impact of course modality on student evaluation of courses and professors. Data were collected through the Student Perception of Instruction end of course rating form at the University of Central Florida (UCF), which contains nine items and maintains student anonymity. The findings indicate that while course modality accounts for only 1% of the variance in student evaluations, there is strong internal consistency and reliability in the rating scale. The distribution of ratings showed a concentration of scores at the high end, resulting in a high variability coefficient. However, when the long tail of low ratings was removed, the mean increased and the distribution became more symmetric, affecting various psychometric indices. The correlation matrices among the items revealed a single factor solution for each modality, suggesting that students tend to rely on a general impression when rating their courses. The multidimensional scaling process identified underlying categories such as structure, course climate, engagement, and consideration, even though students did not explicitly differentiate these elements in their responses. The study concludes that course modality has minimal influence on overall student ratings, a finding consistent across different time periods, including the COVID-19 pandemic. Although a single factor captures students' general evaluations, underlying categories shape their responses. The article also presents a classification model that predicts student ratings based on the scale items. This research addresses the complex dynamics of student evaluations, highlighting the nuanced relationship between course modality, student perceptions, and the underlying factors influencing their ratings.

Discrimination against academics and career implications of student evaluations: university policy versus legal compliance

Across the international higher education sector, existing studies highlight that student evaluations of courses and teaching are biased and prejudiced towards academics and can cause mental distress. Yet student evaluation data is often used as part of faculty hiring, firing, promotion, award and grant decisions. That a data source known to be prejudiced and biased is used for employment and career decisions raises questions around whether these university policies are discriminatory towards university staff. This paper investigates these questions via an analysis of: a) what are the common university policies relating to evaluation data collection and its use, b) are these policies leaving academics exposed to discrimination, and c) what types of policies may be leaving universities liable to legal ramifications due to non-compliance with anti-discrimination and workplace health and safety laws? The work demonstrates why most institutions are operating outside the bounds of the law, highlights to academics what types of policies may fail to meet discrimination and workplace laws, and informs university leaders of the actions that may be exposing their universities to legal implications for failing to protect their staff.

Abusive comments in student evaluations of courses and teaching: the attacks women and marginalised academics endure

This paper examines the volume and type of anonymous comments academics receive in student evaluations of courses and teaching (SETs) at the 16,000 higher education institutions that collect this data at the end of each teaching period. Existing research has increasingly pointed to the negative issues of student surveys, but very little research has focused on the volume, type, and impact of anonymous student comments on academics. This paper analyses the survey results of 674 academics to inform higher education leaders and the sector more widely of the amount and type of abusive comments academics are receiving. The work also demonstrates that the highest volume, most derogatory, and most threatening abuse is directed towards women academics and those academics from marginalised groups. The paper finds that previous estimates of the rate and severity of abusive comments that academics receive, and the impact to academics’ wellbeing, mental health, and career progression, have underestimated what is taking place. The paper argues that many universities are failing to protect their staff from this abuse, and the prejudice nature of SET results, which will continue to have a negative impact on the career progression of marginalised academics - a major flaw in a sector that prides itself on diversity and inclusion.

Innovation in Digital Education: Lessons Learned from the Multiple Sclerosis Management Master's Program.

Since 2020, the master’s program “Multiple Sclerosis Management” has been running at Dresden International University, offering structured training to become a multiple sclerosis specialist. Due to the COVID-19 pandemic, many planned teaching formats had to be changed to online teaching. The subject of this paper was the investigation of a cloud-based digital hub and student evaluation of the program. Authors analyzed use cases of computer-supported collaborative learning and student evaluation of courses and modules using the Gioia method and descriptive statistics. The use of a cloud-based digital hub as a central data platform proved to be highly successful for learning and teaching, as well as for close interaction between lecturers and students. Students rated the courses very positively in terms of content, knowledge transfer and interaction. The implementation of the master’s program was successful despite the challenges of the COVID-19 pandemic. The resulting extensive use of digital tools demonstrates the “new normal” of future learning, with even more emphasis on successful online formats that also increase interaction between lecturers and students in particular. At the same time, there will continue to be tailored face-to-face events to specifically increase learning success.

Applying team-based learning in a transnational post registration bachelor of nursing program in Singapore

BackgroundTeam-Based Learning (TBL) is growing in popularity as a method to create active learning within larger group lectures. It is facilitated through phases of individual work, teamwork with immediate feedback and an application exercise, to develop students’ understanding and assessment of conceptual knowledge. A single facilitator can manage many groups within larger lectures. The study aim was to evaluate the impact of TBL on the engagement, learning and satisfaction of students enrolled in a transnational post registration Bachelor of Nursing (BN) program in Singapore.MethodsA cross-sectional design was employed. The TBL approach was delivered during lecture sessions within a post registration BN program delivered in a Higher Education Institution in Singapore. A sample of BN students was drawn from 305 students undertaking the program. An online anonymous university-delivered student evaluation of course (SEC) survey and an online anonymous survey using the Student Assessment Instrument, were used to collect quantitative and qualitative data. Survey participation was voluntary with informed consent protocols followed. Student performance in the course was also reviewed.ResultsEighty-two students (27%) completed the SEC scoring a median of 4/5 for satisfaction, and 68 (22%) completed the online survey. As 93 was the neutral score for the survey, there was a moderately positive evaluation with an overall score of 108.5/155 for TBL in accountability for learning, TBL preference and satisfaction with TBL compared to traditional lecture approaches.ConclusionsImplementation of TBL with this cohort demonstrated evidence of moderately positive engagement, learning and satisfaction when compared to traditional didactic lectures.

Assessing and Assisting Arab Undergraduates’ Skills in Producing Research Papers in English

This paper discusses the results of a study conducted on a group of undergraduates at the Arab International University. The study has assessed the students’ research skills via a qualitative analysis of their earlier research papers submissions and a questionnaire that determined the weaknesses and difficulties they face while writing a paper. Results showed that undergraduates find it most difficult to use and understand English references, paraphrase paragraphs and use in-text citations. Based on the quantitative and qualitative analysis of the results, the researcher devised a new course that aimed to enhance students’ practice of the main research skills. This course was based on developing student’s information literacy and assigning multiple focused tasks to practice the main research skills. The post course evaluation of students’ achievement showed a notable improvement in their research practices that aided the production of above average quality English research papers.

Sexism, racism, prejudice, and bias: a literature review and synthesis of research surrounding student evaluations of courses and teaching

This paper analyses the current research regarding student evaluations of courses and teaching. The article argues that student evaluations are influenced by racist, sexist and homophobic prejudices, and are biased against discipline and subject area. This paper’s findings are relevant to policymakers and academics as student evaluations are undertaken in over 16,000 higher education institutions at the end of each teaching period. The article’s purpose is to demonstrate to the higher education sector that the data informing student surveys is flawed and prejudiced against those being assessed. Evaluations have been shown to be heavily influenced by student demographics, the teaching academic’s culture and identity, and other aspects not associated with course quality or teaching effectiveness. Evaluations also include increasingly abusive comments which are mostly directed towards women and those from marginalised groups, and subsequently make student surveys a growing cause of stress and anxiety for these academics. Yet, student evaluations are used as a measure of performance and play a role in hiring, firing and promotional decisions. Student evaluations are openly prejudiced against the sector’s most underrepresented academics and they contribute to further marginalising the same groups universities declare to protect, value and are aiming to increase in their workforces.

온라인 학습카페 활용 수업에 대한 학생평가 및 학습활동 유형 분석 - S대학교 WIU 강좌 사례를 중심으로

최근 교육환경의 변화 등으로 다양한 형태의 온라인 기반 학습이 이루어지고 있다. 본 연구는 포털사이트 기반 학습카페를 활용한 수업에 대한 학생평가 및 구체적인 학습활동 유형 분석을 통해 온라인 기반 수업의 개선방안을 탐색하고자 한다. 이를 위해 2019년 S대학교에서 운영된 65개 온라인 학습카페 활용 강좌를 대상으로 수업효과에 대한 학생평가 설문을 실시하였다. 더불어 수업별 운영된 학습활동을 학습형태, 활동수준, 상호작용 방식을 중심으로 유형화하여 분석하였다. 분석 결과, 학생들의 수업에 대한 만족도는 3.73점으로 나타났고, 인지적 차원의 수업효과에 대한 인식이 3.84점으로 가장 높게 나타났다. 한편, 학습활동의 유형은 개인기반/기억 및 이해 수준의 활동/간접적 상호작용 방식이 38.46%로 가장 높은 비율을 보였고, 개인기반/적용 및 분석 수준의 활동/간접적 상호작용 방식이 26.15%로 높게 나타났다. 이는 수업에서 온라인 학습카페가 ‘개인의 학습활동 결과를 공유하는 장’으로 주로 활용되고 있음을 의미한다. 이러한 분석결과를 토대로 보다 나은 수업효과를 위한 온라인 학습카페 활용을 위한 논의점을 제시하였다.

Unravelling student evaluations of courses and teachers

There is debate over the functional basis of student evaluations of academics, and fresh potential for looking at the data in new ways. Student evaluation data was collated over a three year period...

Potential bias factors that affect the course evaluation of students in preclinical courses.

PurposeWe aim to identify what potential bias factors affected students’ overall course evaluation, and to observe what factors should be considered in the curriculum evaluation system of medical schools.MethodsThis study analyzed students’ ratings of preclinical instructions at the Ajou University School of Medicine. The ratings of instructions involved 41 first-year and 45 second-year medical students.ResultsThere was a statistically significant difference between years of study and ratings’ scoring. Learning difficulty, learning amount, student assessment, and teacher preparation from second-year students were significantly higher than first-year students (p<0.05). The analysis results revealed that student assessment was the predictor of ratings from first-year students, while teacher preparation was the predictor of ratings from second-year students.ConclusionWe found significant interactions between year of study and the students’ rating results. We were able to confirm that satisfaction of instructions factors perceived by medical students were different for the characteristics of courses. Our results may be an important resource for evaluating preclinical curriculums.

What determines students’ perceptions in course evaluation rating in higher education? An econometric exploration

While student evaluation of courses (SEC) in higher education is an intensely researched area, the existing literature has not paid due attention to rigorous econometric analysis of the SEC data. Using the four-year (2010–2013) evaluation results for economics courses on offer at a leading Australian university, this study employed a random effects ordered probit model with Mundlak correction to identify factors influencing student ratings of courses. This represents an innovative application to educational data.Findings show that class-level, course-level, class-size, instructors’ course-specific experience and their linguistic background influence student ratings of courses. Lecturers’ prior teaching experience in a course and their English language background attracted higher rating while second and third-level courses relative to postgraduate classes, 2010 and 2012 student cohorts relative to 2013, and larger classes attracted lower ratings.Implications include specific training for instructors of non-English speaking background (NESB), teaching larger classes, and intermediate and upper undergraduate courses.This study underscores the critical importance of student-specific responses capturing student heterogeneity in preference to class-average data including students’ academic performance, discipline destination, linguistic background, age and indicators of effort-level. It raises survey instrument implications e.g., sub-scales, data on course contents providing intellectual challenges, real world applications, and problem-solving skills.

An Eight-year Retrospective Study in “Flipped” Pharmacokinetics Courses

To assess the impact on student performance of increased active learning strategies in a foundational pharmacokinetics course and a clinical pharmacokinetics course over an 8-year period. A foundational pharmacokinetics course with a lecture-with-active-learning (LAL) format was redesigned to a recitation-format (REC) using smaller groups of students (ie, the class divided into thirds) and eventually to a team-based learning (TBL) format. The lecture-based clinical pharmacokinetics course was redesigned to a case-based learning (CBL) format to encourage preclass preparation with class time used for application; this course format underwent minor redesigns over an 8-year period. An analysis of covariance (ANCOVA) was performed on examination scores in the clinical course based on foundational course format changes. End-of-semester student evaluations of the course were used as a secondary measure of impact. The highest grades in the clinical course were associated with the TBL format within the foundational course compared to LAL format (effect size 0.78). The REC format in the foundational course compared to LAL was associated with higher performance in the clinical course (effect size 0.50). Examination performance in the clinical course had a small increase when the foundational course was transitioned from the REC format to the TBL format (effect size 0.27). There was a trend within the foundational course that overall student ratings of the course decreased with enhanced self-directed learning; there was no change in overall ratings of the clinical course. Increasing the amount of active learning within the foundational pharmacokinetics course increases performance in the clinical course but this increase in performance may be associated with decreases in student evaluations of the foundational course.

Students Prefer Audience Response System for Lecture Evaluation

Objectives: Student evaluation of courses is an important component of overall course evaluation. The extent of student participation in the evaluation may be related to the ease of the evaluation process. The standard evaluation format is a paper form. This study examines medical students preference of utilizing Audience Response System compared to a paper method.&#x0D; Methods: Following several medical school lectures, students were queried if they preferred Audience Response System versus a paper method, and if they would prefer using Audience Response System more for future course evaluations.&#x0D; Results: 391 students were queried. Overall response rate was 94%. Using a five point Likert scale, 299 out of 361 (82%) responded they agreed, or strongly agreed with the statement â??We should use ARS more. . .

26. The Validity of Student Course Evaluations: An Eternal Debate?

Student evaluation of courses and teaching at universities remains a highly contentious and divisive topic. Emotions and anecdotal evidence can overrule conclusions drawn from research on the validity and design of course evaluations. However, even amongst researchers, there is significant disagreement on the efficacy of course and teaching evaluations. This paper explores this ongoing dialogue through the medium of a parliamentary debate drawing from the breadth of current research on course evaluations.

University of Washington School of Medicine

University of Washington School of Medicine

Student evaluation of courses: what predicts satisfaction?

The main goals of course evaluations are to obtain student feedback regarding courses and teaching for improvement purposes and to provide a defined and practical process to ensure that actions are taken to improve courses and teaching. Of the items on course evaluation forms, the one that receives the most attention and consequently the most weight is the question, ‘Overall, I was satisfied with the quality of this course.’ However, no attention has been placed on examining the predictors of students being ‘satisfied with the quality of this course’ overall. This study attempts to address this gap. The findings show that while student characteristics and reasons for enrolling in a course are predictors of overall satisfaction, it is the evaluation questions that predict the majority of the variation in course satisfaction. The findings also reveal that faculty‐selected optional questions are stronger predictors of overall satisfaction than compulsory questions.

Internal and External Comments on Course Evaluations and Their Relationship to Course Grades

The validity of student evaluations of courses and the relationship between evaluations and course outcomes has frequently been examined. Since many course evaluations give students an opportunity to provide answers to open-ended questions in addition to giving Likert scale ratings, it is important to understand the relationship between these responses and course outcomes. This study examined the relationship between student responses to open ended questions (specifically whether they attributed their achievement to factors within their control or factors not within their control) and their outcomes in the course. The results of the study indicate that students that identified external factors (e.g. the professor) as responsible for their success in the course had significantly higher course outcomes than students that identified external factors as responsible for their failure in the course. No differences were detected in course outcomes for students that made external comments on their course evaluations versus those that made internal comments.