Peer Assessment Research Articles

Assessment of Attitudes, Ethics and Communication Skills in a Competency-Based Undergraduate Medical Curriculum

Affective learning outcomes are difficult to objectively analyse and assess as they refer to feelings and internal processes of the mind and heart. These cannot be quantified by traditional testing methods. The available methods rely on qualitative self-reflection and the observation of simple to complex internally consistent qualities of character. Just as in the other domains of learning, the validity and reliability of these assessments need to be ensured. The assessment of the affective domain requires careful observation of specified behaviours by multiple observers. Miller’s pyramid, Krathwohl’s levels and Epstein’s conceptual frameworks can be used to devise suitable assessment plans for the complex and varied competencies in this domain. Many techniques including paper-based tests, self-administered rating scale, faculty and peer assessment, simulations, reflections, portfolios, observed clinical encounters, collated views of co-workers, record of incidents of unprofessionalism, critical incident reports, patient surveys and global view of supervisor have been used for assessment of affective domain. The effective utilization of these assessment tools needs to factor in the ground realities that exist in different medical colleges. The acceptability and feasibility of using resource intensive assessment methods needs to be carefully balanced against their contributions towards increasing validity and reliability

Peer assessment: An effective tool to modify professional behaviour of undergraduate medical students

Peer assessment: An effective tool to modify professional behaviour of undergraduate medical students

Enhancing students’ engagement and learning through peer assessment in group projects

Aiming at enhancing students’ active engagement and deep learning in a large-class setting, this study introduced group projects along with peer assessment among tertiary-level students in different cohorts. Following the designated pedagogical approach and assessment criteria illustrated by the course instructor, students’ learning experiences related to project preparation and peer assessment in a compulsory course were observed for three consecutive years. Students expressed a high degree of agreement that the flexibility allowed in selecting group members and project contents promoted their interests in conducting projects on diverse environmental issues beyond the lecture materials and helped them develop teamwork and interpersonal skills. Regarding peer assessment, the observed resemblance among teacher-assessed and peer-assessed grading in most cases indicated that students could successfully develop critical evaluation skills required for the peer assessment. Moreover, students provided positive feedback on the peer assessment method and responded that this exercise triggered reflection on their own work, which could be useful for enhancing their deep learning and understanding of the subject. However, unsurprisingly, some students requested more guidance and directions given by the course instructor regarding project selection and grading criteria ahead of the peer assessment. It is also interesting to note that students’ comments varied regarding the weighted fractions of project marks allocated to the teacher (50%) and peer (50%). Some students underscored that the teacher’s grading should have a higher weighting than those of peer students, which may improve the fairness in marking. These various comments could assist us to better implement peer assessment in other courses. In all, given the students' positive learning attitudes and improved evaluation skills, peer assessment of group projects can be considered as a comprehensive approach for engaging students in deep learning experiences and higher-level understanding of broader knowledge. Keywords: Tertiary Education, summative assessment, teaching Pedagogy, collaborative learning, reflection/evaluation, deep Learning.

Assessment of the Implementation of the International Health Regulations during the COVID-19 Pandemic: Portugal as a Case Study.

The International Health Regulations (IHR) were developed to prepare countries to deal with public health emergencies. The spread of SARS-CoV-2 underlined the need for international coordination, although few attempts were made to evaluate the integrated implementation of the IHR's core capacities in response to the COVID-19 pandemic. The aim of this study was to evaluate whether IHR shortcomings stem from non-compliance or regulatory issues, using Portugal as a European case study due to its size, organization, and previous discrepancies between self-reporting and peer assessment of the IHR's core capacities. Fifteen public health medical residents involved in contact tracing in mainland Portugal interpreted the effectiveness of the IHR's core capabilities by reviewing the publicly available evidence and reflecting on their own field experience, then grading each core capability according to the IHR Monitoring Framework. The assessment of IHR enforcement considered efforts made before and after the onset of the pandemic, covering the period up to July 2021. Four out of nine core IHR capacities (surveillance; response; risk communication; and human resource capacity) were classified as level 1, the lowest. Only two were graded level 3 (preparedness; and laboratory), the highest. The remaining three) (national legislation, policy & financing; coordination and national focal point communication; and points of entry) were classified as level 2. Portugal exemplifies the extent to which implementation of the IHR was not fully achieved, which has resulted in the underperformance of several core capacities. There is a need to improve preparedness and international cooperation in order to harmonize and strengthen the global response to public health emergencies, with better political, institutional, and financial support.

WebAVALIA: analysis of students’ opinions and experiences using an e-assessment tool in collaborative learning

The students’ assessment regarding collaborative learning and workgroups is being reported as one of the main concerns in higher education. The increased technological evolution leads to adapting novel self and peer assessment methods to e-assessment. This paper reports the results of a study to evaluate students’ opinion about their experience with an e-assessment tool, WebAVALIA, and its assessment criteria. The results indicate that students (N = 359) consider the tool fair, and it increases the productivity regarding work development. Kruskal-Wallis tests show that the students recently considered WebAVALIA fairer and more straightforward. Quickness and anonymity are also identified as tool advantages.

Hybrid Train-The-Trainer course for the Fundamental Use of Surgical Energy certificates improves self-confidence in knowledge of surgical energy and develops teaching skills: a feasibility study.

Several well-trained expert instructors who completed the "Train-The-Trainer (TTT)" course are required to disseminate the safe use of surgical energy devices, which can be learned through the Fundamental Use of Surgical Energy (FUSE) program. This study aimed to explore whether the hybrid FUSE TTT course is feasible and effective, which can improve teaching skills of surgical energy. The hybrid TTT course, which was designed to train FUSE-certified personnel as instructors, comprised three virtual sessions spread over 5h in total and a 1-day in-person training, followed by a 100-min FUSE electrosurgery hands-on workshop in practice as an instructor. The participants reported on self-confidence regarding knowledge of various energy devices or adverse events before, immediately after, and 6months after the course. Participants and experienced FUSE instructors assessed the trainees' presentation skills at the beginning of the in-person training and after the hands-on workshop. The primary outcomes were the feasibility and completion rate of the entire course. Seventeen participants completed the entire couse; most (94%) were satisfied with the course. Self-confidence in knowledge about various contents improved significantly: the fundamentals of electrosurgery (post, p < 0.001; 6months, p = 0.01), mechanism and prevention of adverse events (post, p = 0.001; 6months, p = 0.04), monopolar instruments (post, p = 0.002; 6months, p = 0.01), bipolar instruments (post, p = 0.01; 6months, p = 0.06), and integration with other medical devices (post, p = 0.006; 6months, p = 0.02). The presentation skill index scores of self- and peer assessments improved after the in-person training (self-assessment [pre 44 vs. post 56, p < 0.001], peer assessment [pre 39 vs. post 68, p < 0.001]). The hybrid TTT course can provide FUSE-certified personnel with an improved self-confidence concerning knowledge of surgical energy and improve their presentation skills with midterm retention. This can help build trainees' self-confidence as instructors.

Role of Peer Assessment in Modifying Professional Behavior among Phase 1 Medical Undergraduates: A Prospective Interventional Study

Abstract Background: India’s competency-based medical education curriculum has now included professionalism as a core competency for medical graduates. However, assessing professionalism and interpersonal skills remains challenging. This study explores the impact of peer assessment (PA) on MBBS first-phase students’ professional behavior and its association with gender and academic performance. Methodology: We conducted a prospective interventional study. Over three iterations which were done at the interval of 1 month, each student evaluated six peers using the “Rochester PA tool” comprising 15 standardized items. Statistical analysis employed t-tests, Chi-square tests, analysis of variance, and correlation tests. Results: Hundred and forty students were enrolled and their responses were analyzed. “Unable to answer” responses significantly decreased over time (P &lt; 0.0001), particularly when students did not consider peers as close friends (P &lt; 0.001). PA scores remained stable over three timelines, but in the first and second iterations, females received higher scores for all the factors except for intrapersonal attributes. Academic grades correlated positively with PA scores for work habits, global items, and overall performance but not for interpersonal attributes. Conclusions: This study highlights PA as a valuable training tool for inculcating an understanding of professionalism among medical students in early years of MBBS course. We did not find any improvement in the professional behavior after repeated implementation of PA probably because the gap between the two evaluations was insufficient to bring about noticeable behavioral changes. Further research is required to explore if early introduction of such assessments in medical school can nurture the professional development leading to improvement in the accuracy of later peer evaluations.

Combining Building Block Process With Computational Thinking Improves Learning Outcomes of Python Programming With Peer Assessment

The capability of computer programming language logic is one of the basics of technical education. How to improve students “interest in program logic design and help overcome students” fears of coding has become vital for educators. Cultivating practical talents with information technology application and basic programming development will become one of the important topics in the department of information related science. The objective of this research is to improve the ability of learning basic programming courses by using Zuvio interactive software. Zuvio employs the mathematical logic of computational thinking to analyze problems and enhance learners’ interest in learning programming skills through a graphical interface tool with building blocks. It uses innovative interactive teaching to use peer and self-assessment to study the content of the course. Zuvio improves the design ability of different groups of class learning Python programming. In line with the innovative teaching policy of the schools and the current stage of the learner’s learning model, learning effectiveness can be achieved. The research results were analyzed by midterm and final experimental group scores, and the progress of the experimental group’s scores was examined through descriptive statistics. The average and standard deviation of the assessment were used to analyze the progress of the experimental group students in the programming course. In the classroom, assessment criteria were set up as the basis for peer assessment scoring. After the midterm and final exams, the teacher assessment and peer assessment scores were analyzed for cognitive differences, and possible learning differences were analyzed. The students’ professional ability was examined to see if it met the professional standards required by the course, and whether innovative teaching methods could improve the learning outcomes of learners with different professional backgrounds in Python programming.

Exploring the Implementation of Assessment for Learning in the Asian EFL Context: A Systematic Research Synthesis

While acknowledging the significant impact of Assessment for Learning (hereinafter referred to as AfL) on student learning, the specific factors that either support or impede its integration into daily classroom practices remain unclear in Asian EFL context. To address this gap, a comprehensive review was undertaken for the purpose of determining the AfL strategies applied in existing literature and the key factors that influence the effective implementation of AfL within particular contexts. This research synthesis summarized clarification and instruction of learning intentions and goals, teachers’ feedback, peer assessment and self-assessment as main AfL strategies applied in Asian EFL classrooms and detected several key elements concerning teachers, students, assessments, and the overall context that influence the effect of the implementation of AfL. By shedding light on these various aspects, this systematic research synthesis contributes to a more comprehensive understanding of the theoretical and practical considerations involved in the implementation of AfL.

Augmented Reality With Authentic GeometryGo App to Help Geometry Learning and Assessments

We developed an Authentic GeometryGo app to help students' application of geometry in authentic contexts through authentic measurements and peer assessments. The study included an experimental group using an Authentic GeometryGo app and a control group using paper-based methods. The results showed that the experimental group outperformed the control group in terms of estimation. Measuring authentic geometry shapes using the app could foster students' estimation. Furthermore, meaningful peer assessments with visual representations in authentic contexts can facilitate the evaluation of peers' work and improve students' geometry understanding. Therefore, we encourage educators to use augmented reality with appropriate learning activities to sharpen students' measurement quality and visual representations by measuring real geometry shapes and applying mathematics knowledge in authentic contexts.

How Learning Analytics Can Help Orchestration of Formative Assessment? Data-Driven Recommendations for Technology- Enhanced Learning

Formative assessment provides teachers with feedback to help them adapt their behavior. To manage the increasing number of students in higher education, technology-enhanced formative assessment tools can be used to maintain and hopefully improve teaching and learning quality thanks to the high amount of data that are generated by their usage. Based on literature and on authentic usages of the formative assessment system called Elaastic, we use learning analytics to provide evidence-based knowledge about formative assessment practices. An Elaastic sequence consists in asking learners to answer a choice question with a vote, a confidence degree and a rationale, then to confront their viewpoints through a peer grading activity, and finally to answer the same question a second time. Benefits of such sequences are measured through the increase of correct answers between the two votes. Our results suggest that: (A) Benefits of sequences increase when close to 50% of learners' first votes are correct; (B) Benefits of sequences increase when peers provide better grades to rationales related to correct answers than others; (C) Sequences benefits do not significantly increase when learners who provided correct answers are more confident than learners who did not; (D) Grades attributed by peers depend on such peer's confidence degree; (E) Self-grading is inaccurate in peer grading context; (F) The amount of evaluations each learner performs makes no significant difference in terms of sequences benefits. These results lead to recommendations regarding formative assessment and to a new data-informed formative assessment process which is discussed at the end of the paper.

Exploring the Agreement Between Perceived Rankings and Outcome Data Rankings for Professional Athletic Training Programs

Context The U.S. News &amp; World Report college ranking system is used to describe the best graduate programs in the country. Rankings of graduate programs are based solely on perceived ratings of quality by directors and/or deans. Athletic training is not listed by U.S. News &amp; World Report; however, the Commission on Accreditation of Athletic Training Education (CAATE) reports key metrics such as the Board of Certification pass rate and program graduation rate, which could be helpful to create rankings. Objective To evaluate and rank CAATE-accredited professional athletic training (PAT) programs using 2 models: (1) perceived rating of academic quality by program directors (PDs) and (2) CAATE outcome data. Design Cross-sectional with survey and retrospective data. Setting Web-based survey. Patients or Other Participants One hundred fifty-five PDs and 230 CAATE-accredited PAT programs. Main Outcome Measure(s) The perceived rating survey for the PDs resembled the U.S. News &amp; World Report system using a 5-point Likert scale to assess the academic quality of each program. For the CAATE outcome data, we used publicly available information for each PAT program on the CAATE website. We ranked all PAT programs using the data from each model. A Cohen κ was performed to explore the agreement between the PDs’ perceived ranking and the CAATE outcome data rankings. Results No agreement was found between the perceived peer assessment and CAATE outcome data rankings (κ = −0.003, P = .401). Conclusion Perception by PDs did not align with objective data reported by CAATE. The lack of agreement between the 2 ranking systems highlights concerns about using the U.S. News &amp; World Report system for graduate health programs. We suggest exploring a more robust and comprehensive formula including overall pass rate and graduation rate to identify top-ranked programs in athletic training.

"Show me how to use a microscope" - The development and evaluation of certification as direct assessment of practical lab skills.

Practical lab skills are rarely directly assessed. To improve constructive alignment between the described learning outcomes of practical skills and assessment, we developed and tested a certification procedure for microscopy skills. The procedure was embedded into the ordinary learning activity, so no additional time was needed. Three slightly different protocols were developed within the framework of sociocultural learning theory and built like a skill ladder, including direct peer assessment and elements of gamified learning. The protocols varied slightly in the way students were prepared for the certification, the number of steps/levels of achievement, and the consequences of failing. We tested the protocols at three different academic institutions and within 11 courses of varying sizes and academic levels in biology or geology. Feedbacks were collected through online surveys (n = 207) or orally after sessions. One protocol provided instruction videos as preparation material. Instruction videos provided increased understanding of the task, but tactile training was most important for learning. Regardless of institution, type of preparation, and level of former experience, the certification procedure made students clearly more engaged in the exercise. The majority reported that the certification procedure increased their motivation to learn, increased their perceived learning outcome, and was appropriate for assessing practical skills. Students with no or little experience in microscopy before the exercise were more positive about the certification procedure compared to skilled students, and the level of engagement and preparation was higher when there were some consequences of failing. Most students felt comfortable being certified by peers, but some students expressed concern about peers making mistakes. The presented certification procedure can easily be adapted to assess other practical skills and, with some adjustments, be an efficient method for assessment-as-learning, merging formative- and summative assessment.

Identifying Students’ Group Work Problems: Design and Field Studies of a Supportive Peer Assessment

Abstract Group projects are part of the core educational experience in higher education, but many students report bad experiences with them, characterized by negative emotions and group problems. Group problems may undermine learning and cause stress and frustration. This may be prevented by monitoring and supporting groups, but this is often not feasible for teachers, who lack time and resources. This research aims to find a method for early identification of group work problems via computer-supported assessment. First, interviews and focus groups provided insights into the most common group problems and which visual features students preferred in a peer assessment. Next, two assessment versions were created: a simple, time-efficient version, and a more engaging, interactive one. We also created an initial version of E-Mate, a virtual agent that provides feedback on the assessment. These were tested in a field study. Most students reported a positive experience with the peer assessment, regardless of the version used. Teachers were also positive about its usefulness. Based on the feedback received, new features were added to the peer assessment survey and the E-Mate was redesigned; this new version was tested in a second field study. The results are in line with the previous field study and confirm the positive reception and usefulness of the survey, supporting the use of five attributes to evaluate group collaboration and the usage of a peer assessment survey to assess group work.

The Combination of British Parliamentary Debate and Peer Assessment in Teaching Speaking

The main problems of this research were how to improve the speaking ability of EFL students converting grammar, vocabulary, pronunciation, discourse, and strategies. therefore, how to encourage them to use English. This research aimed to see whether the combination of British Parliamentary Debate and Peer Assessment may enhance the speaking ability of EFL students. A quasi-experimental method with non-equivalent group design was carried out in this research. The population was the third semester of English Department students at Universitas Muhammadiyah Parepare. Two classes from the population, representing the experimental class and the control class, were selected using the cluster random selection technique as the study's sample. This research used an interview test with a variety of themes as its instrument. Pre-test, treatment, and post-test were the three steps that made up the data collection processes. The data that obtained by the test were then submitted to a quantitative analysis. The research findings demonstrate that the experimental class students' speaking abilities were better to those of the control class. The data from the experimental class showed that the students' mean score was higher compared to the control class (59,68 &gt; 49,06). In conclusion, the combination of British Parliamentary Debate and Peer Assessment was improved the students’ speaking ability.

Co-Teaching with High School Students for Music Teaching

This article presents a didactic proposal for teaching musical notation and solfège sight-singing through co-teaching with secondary school students. The goal was to explore the advantages and limitations of using a form of peer learning where students play the role of co-teachers alongside the teacher. We wanted to find out whether this proposal would lead to significant improvements among both co-teachers and tutees, identifying the actions responsible for this progress, and paying particular attention to how students learn in the role of co-teachers. This study was based on a mixed-method explanatory sequential design, where the quantitative data were analyzed first and then the qualitative data. The quantitative results showed statistically significant improvements when comparing the pre-test and post-test results of all the students participating in the didactic proposal. These improvements can be attributed to the personalized support provided by the co-teachers, the opportunities for peer assessment that enabled progress to be monitored and provided feedback, and the possibilities of learning by teaching. In conclusion, the results of this study endorse the viability of student co-teaching in secondary school music education.

Evaluation of effectiveness and efficiency of fixed orthodontic treatment comparing standard and computer-aided design and manufacturing conventional bracket systems using indirect bonding for both: A retrospective study

To compare the effectiveness and efficiency of orthodontic treatment (OT) with standard versus computer-aided design and computer-aided manufacturing (CAD/CAM) indirect bonding of conventional brackets. This retrospective study examined two groups: standard indirect bonding group (12 males, 13 females; mean age 12.21 ± 0.52 years), and digital indirect bonding group (11 males, 14 females; mean age 12.76 ± 1.32 years), treated via a CAD/CAM indirect bonding system. Conventional brackets were used in both groups. Pretreatment and post-treatment records were acquired for all subjects. Weighted Peer Assessment Rating (W-PAR) index was used to assess the effectiveness of OT, in conjunction with five angular cephalometric measurements (ANB°, Sn-GoGn°, U1-PP°, IMPA° and FMA°) and comparatively analyzed using generalized mixed-effects models and post hoc test. Treatment efficiency was assessed in terms of the numbers of bracket repositionings, archwire bends, accidental bracket debondings, appointments and treatment months. Comparative analysis of efficiency was performed using the asymptotic Wilcoxon-Mann-Witney test. Statistical significance was set at 5%. Total W-PAR and W-PAR component scores decreased significantly during treatment for both groups and in a similar way. Cephalometric measures ANB° and IMPA° significantly decreased and increased, respectively. Significantly fewer bracket repositionings, number of appointments and treatment months were recorded in group digital indirect bonding. Although both methods investigated were effective to achieve good outcomes, CAD/CAM indirect bonding method increased the efficiency of OT, when conventional brackets are used.

Student rating consistency in online peer assessment from the perspectives of individual and class

Most of previous studies examined the consistency among two or three student raters giving scores to different peers (i.e. ratees), whereas few of them researched the consistency among more student raters giving scores to different peers (i.e. ratees). The present study aimed to see whether there is a consistency among student raters giving scores to peers in an online assessment activity. Participants of the study were 266 graduate students, with 137 males and 129 females, from 27 classes in a department at some university. Each student was required to give a presentation graded by the peers anonymously. From to the perspective of class, the result of the Kendall coefficient of concordance showed that a proportion of more than half-classes presented a significant correlation with a moderate or strong consistency among student raters giving scores to peers, and thus the number of raters or peers should be more for a higher reliability. The result of Chi-square test from the perspective of individual revealed that students held a consistent standard when giving scores to peers' presentations. Finally the implications for theories and practices are proposed.

Entrepreneurship Education Pedagogical Approaches in Higher Education

Entrepreneurship education, as an emerging scientific field, has undergone significant evolution at the conceptual and praxis levels. The concept of entrepreneurship was considered from a broad perspective as the capacity to act upon opportunities and ideas, thereby creating social, cultural, or financial value in diverse contexts. The study and cultivation of this competence are highly relevant to enhancing employability and equipping young people with the skills, knowledge, and attitudes necessary within an entrepreneurial culture for the exercise of active citizenship. This study aims to consolidate the concept and analyse the pedagogical approaches in entrepreneurship education, particularly focusing on experiential learning. It also explores the most frequently used instructional methods in higher education programmes. The research is based on a comprehensive literature review, complemented by a case study of the entrepreneurship education programme implemented at the University of Lisbon, in Portugal, enriched by data collected from one of its curricular units. Experiential learning emerges as a valid approach in entrepreneurship teaching, especially when teachers effectively combine practical experience with theory while learners assume the shared responsibility of learning from experience. As the most recommended instructional methods, one can point to collaborative pedagogical models, namely problem-based learning, project-based learning, peer assessment, design thinking, formative feedback, service learning, and active methodologies that integrate digital technologies.

The effects of using analytical rubrics in peer and self-assessment on EFL students’ writing proficiency: a Vietnamese contextual study

This research investigates the effectiveness of utilizing analytic rubrics in peer-assessment (PA) and self-assessment (SA) methodologies to enhance the proficiency of English as a Foreign Language (EFL) students’ essay writing skills in the Vietnamese context. It further contributes to the existing body of literature regarding formative assessment and its potential to improve student learning outcomes. A total of 44 university students, all English majors, were divided into two distinct groups, each consisting of 22 participants. One group applied analytic rubrics for SA, while the other used the same tool for PA. The writing performance of the two groups was assessed and compared in pre and post-tests. The findings revealed no significant difference between the SA and PA groups in the pre-test. However, in the post-test, the SA group demonstrated significantly superior performance compared to the PA group, with noticeable improvements across all evaluated criteria. Moreover, these results showed that the use of analytic rubrics in SA and PA methods positively impacted the EFL students’ writing skills, particularly in the areas of content and language use. This has practical implications for teachers, curriculum developers, and policymakers in designing and implementing formative assessment strategies for EFL learners. Further research is needed to examine the long-term effects of employing analytic rubrics, and to understand the potential influence of other contextual factors on student learning outcomes.