Peer Evaluation Process Research Articles

Utilizing peer evaluation as a collaborative learning tool: fostering autonomy satisfaction in English presentation classes

Abstract The present two-phased sequential explanatory mixed-methods research investigates first and second-year Japanese university students’ perceptions of peer evaluation through self-reflection and its relation to learner autonomy satisfaction in English presentation activities. First, the qualitative data were collected through an open-ended survey questionnaire (n = 17). The findings revealed that students held a positive view of the peer evaluation process, which was facilitated by structured evaluation criteria developed collaboratively with the students using the Leaderful Classroom Practices framework. Furthermore, students’ ownership of the evaluation process enhanced their motivation and commitment to their own presentations. To explore the relationship between the peer evaluation process and students’ autonomy satisfaction, we conducted a survey with advanced-level first-year English major students (n = 87). Despite the small sample size, our multiple regression analysis also supported the qualitative findings, indicating a significant correlation between peer evaluation and learner autonomy satisfaction in English presentation activities. While peer-based activities have been thoroughly studied in the context of academic writing courses, we recommend that the influence of peer evaluation activities on learner autonomy satisfaction be further explored in oral communication activities within the Japanese university EFL settings.

Nursing students' perspectives on peer assessment in basic nursing procedures: A qualitative study

BackgroundTraditionally, the assessment of student nursing skills relies heavily on lecturer evaluations. However, lecturers, as the primary evaluators, may be influenced by their individual subjective preferences, experience, and knowledge background. This limitation needs thorough consideration to improve students' assessment skills and ensure that assessment tasks are more objective and accurate. Peer assessment appears to take it into account. Currently, improving the quality of peer assessment in Basic Nursing skills remains a challenge for educators. AimTo assess the effectiveness of peer assessment as applied to the basic nursing skills examination process and suggestions for improvement, to understand nursing students' perceptions of it, and to explore the feasibility of peer assessment as an alternative to single faculty assessment as perceived by nursing students. DesignQualitative research. SettingsConducted in a clinical skills simulation laboratory at a university in Wuxi, Jiangsu Province, China. Participants23 Second-year nursing students applying peer assessment to Basic Nursing course exams. MethodsUsed face-to-face, semi-structured interviews, and thematic analysis to analyze data. ResultsThree major themes and nine subthemes were identified: (1) Positive sentiments towards peer evaluation (e.g., enhancing the learning capacity, increasing emotional intelligence, and improving the objectivity and fairness of nursing skills assessment); (2) Challenges encountered in the peer evaluation process (e.g., inconsistent scoring criteria, exacerbate anxiety, and triggering team conflict); (3) Suggestions for improving peer evaluation (e.g., applying the design of peer assessment exam content, incorporation of lecturer evaluation, and refinement and materialization of marking criteria). ConclusionsIn this study, while the positive impact is evident, the challenges faced by nursing students and future recommendations for peer assessment application are critical. This study provides valuable insights into the potential of peer assessment to enhance nursing education and provides recommendations for optimizing peer assessment strategies for meaningful integration into Nursing Education practice.

Peer review for handoff education in a transition to residency course: A prospective cohort study.

Association of American Medical Colleges (AAMC) and Accreditation Council for Graduate Medical Education (ACGME) mandate training in handoff delivery for students and residents. Communication errors, including errors during handoffs of patient care, account for over 2/3 of sentinel events. This study aims to assess the effectiveness of peer-assisted learning (PAL) in handoff education within a longitudinal framework. This study involved the analysis of fourth-year medical students (n = 67) enrolled in a transition to residency program designed to reinforce skills essential for success in internal medicine residencies. We modified the I-PASS handoff rubric for a single-encounter evaluation. Before attending the transitions of care workshop, students submitted one written handoff report. During high-fidelity simulation sessions, peers evaluated the written document as well as verbal handoffs, while faculty evaluated a recorded verbal version. The primary outcome measured was improvement in handoff quality and accuracy over time and secondary outcomes compared peer- and self-evaluations to faculty assessments. Overall, students demonstrated a statistically significant improvement in handoff quality and accuracy across all scoring criteria after completing the peer evaluation process. Peer evaluations did not demonstrate statistically significant differences in scores for quality or accuracy questions as compared to faculty. Peer evaluators effectively assessed handoff reports using the modified I-PASS checklist yielding outcomes similar to faculty while providing feedback. These findings provide exciting evidence that should prompt training programs to consider incorporating standardized peer review into handoff education for medical students and, potentially, residents. The detailed evaluation of individual handoff events fosters feedback skills essential for ongoing professional growth and clinical excellence.

Can Artificial Intelligence help provide more sustainable feed-back?

Peer assessment is a strategy wherein students evaluate the level, value, or quality of their peers' work within the same educational setting. Research has demonstrated that peer evaluation processes positively impact skill development and academic performance. By applying evaluation criteria to their peers' work and offering comments, corrections, and suggestions for improvement, students not only enhance their own work but also cultivate critical thinking skills. To effectively nurture students' role as evaluators, deliberate and structured opportunities for practice, along with training and guidance, are essential. Artificial Intelligence (AI) can offer a means to assess peer evaluations automatically, ensuring their quality and assisting students in executing assessments with precision. This approach allows educators to focus on evaluating student productions without necessitating specialized training in feedback evaluation. This paper presents the process developed to automate the assessment of feedback quality. Through the utilization of feedback fragments evaluated by researchers based on pre-established criteria, an Artificial Intelligence (AI) Large Language Model (LM) was trained to achieve automated evaluation. The findings show the similarity between human evaluation and automated evaluation, which allows expectations to be generated regarding the possibilities of AI for this purpose. The challenges and prospects of this process are discussed, along with recommendations for optimizing results. Artificial intelligence can offer a means to assess peer evaluations automatically, ensuring their quality and assisting students in executing assessments with precision. This approach allows educators to focus on evaluating student productions without necessitating specialized training in feedback evaluation. This paper presents the process developed to automate the assessment of feedback quality. Through the utilization of feedback fragments evaluated by researchers based on pre-established criteria, an artificial intelligence Large Language Model was trained to achieve automated evaluation. The challenges and prospects of this process are discussed, along with recommendations for optimizing results.

Rater behaviors in peer evaluation: Patterns and early detection with learner model

Peer evaluation is a common practice in team-based learning (TBL) designs, which can cover the assessment of individual or group work. However, the integrity of peer evaluation can be compromised by unserious raters—individuals who do not earnestly engage in the evaluation process. These raters may exhibit behaviors like consistently assigning the same score, rushing through evaluations, or evaluating before or long after the target presentations. This study delves into the issue of unserious peer evaluation in group presentations, with a specific focus on understanding the behavior patterns in the digital system. Utilizing evaluation behavior analysis (EBA) indicators, we identify patterns linked to unserious raters during the peer evaluation process. Meanwhile, we also connect these patterns to rating consistency and actual course performance, underscoring the significance of behavior patterns. Further, we conduct a preliminary analysis to explore the application of learner model data available before the peer evaluation starts for the early detection of unserious raters. This finding can assist teachers in providing personalized prompts and interventions before the peer evaluation stage, hence enhancing the evaluation quality through targeted interventions in a timely manner.

Design of a simple rubric to peer-evaluate the teamwork skills of engineering students

ABSTRACT Effectively working in an engineering workplace requires strong teamwork skills, yet the existing literature within various disciplines reveals discrepancies in evaluating these skills. This complicates the design of a generic teamwork peer evaluation tool for engineering students. This study aims to address this gap by introducing the DRIVE teamwork rubric, a simple, valid, and reliable tool designed to assess fundamental teamwork skills in engineering students. Informed by a comprehensive literature review and experiential learning theory, the rubric incorporates five dimensions, encapsulating crucial attributes for successful teamwork. The integration of these dimensions into the rubric aims to streamline the peer evaluation process and improves student awareness about fundamental teamwork skills. Initial implementation data analysis, including exploratory and confirmatory factor analyses, alongside McDonald's Omega, substantiate the rubric's reliability and validity. A comparison between Teaching Assistants’ (TAs’) and peer evaluations affirms students’ ability to assess peers on par with TAs, underscoring the rubric's simplicity and effectiveness.

Exploring Effective Physics Teaching Strategies in High Schools during the COVID-19 Pandemic

The need for educational approaches that comply with the restrictions arising from the COVID-19 pandemic has raised a number of critical issues for students of different age groups. The delicate transition between high school and university has become a key point to focus on, leading many institutions to replan projects dedicated to students involved in this transition. A Physics vocational training project for high school students was carried out in the school year 2020–2021, and it was replicated in the school year 2021–2022. The project included webinars, self-assembled laboratory group experiences, and peer evaluation. The starting point on which we designed our project is that learning is an experience; thus, we built the entire project by particularly focusing on two peculiarities. One peculiarity is the assessment methods: student presentations describing their own experiences were evaluated by teachers and their peers. The second peculiarity is the open approach with respect to how students handle experimental activities. We present a description of these projects along with the results of an evaluation survey filled out by the participants and a descriptive analysis of the assessment strategies. Students appreciated the design of the entire project and, better still, the peer evaluation process. Moreover, we discovered that the evaluation provided by the teachers is lower compared to the assessment reported by the students. This disparity holds potential significance from a statistical perspective and warrants further investigation.

The influence and mitigation of bad apples on group dynamics and outcomes

This research investigates the effect of exceptionally poor team members (identified as “bad apples”) on team dynamics and team performance. The negative impact of bad apples is shown to harm group dynamic factors of team effectiveness, relationships, cohesion, and increase conflict. In addition, we show that bad apples reduce team satisfaction and overall team grades. The use of a two-stage peer evaluation process is explored as a viable technique to minimize the negative impact of bad apples and appears to reduce the number of bad apples. We conclude with a discussion of applicable tools and techniques that can be utilized in a classroom to enhance the group experience and outcomes.

Evaluation of SOAP note performance in a pharmacotherapeutics course after implementation of a peer-review process

IntroductionSubjective, objective, assessment, and plan (SOAP) notes are widely utilized within pharmacy curricula. Implementation of a peer-review process for these assignments may improve student competence. Evidence is lacking regarding correlation between student-to-student peer-review and faculty grades for SOAP notes and correlation of SOAP note peer-evaluation to individual performance. MethodsThird year pharmacy students completed two SOAP notes in Pharmacotherapeutics IV. A peer-review process was implemented in spring 2020. After each SOAP note, students were assigned a peer's assignment to evaluate utilizing the same 50-point rubric as the faculty. SOAP note grades were compared between the 2019 and 2020 cohorts. ResultsAnalysis included 98 students in spring 2019 and 92 students in 2020. SOAP note faculty grades were different between 2019 and 2020 for the first SOAP note (37.6 vs. 41.1, P < .001) but not for the second (42.3 vs. 42.7, P = .49). Peer-review grades were higher for both SOAP notes compared to faculty grades. Peer-review grades did not differ between the first and second SOAP notes (45.5 vs. 45.6) while faculty grades did (41.2 vs. 42.7). The difference in scores from peer-review compared to faculty grades was −4.4 points for the first SOAP note and − 2.9 points for the second SOAP note (P = .08). ConclusionsThe peer-review process did not appear to improve SOAP note performance. Students tended to score better on the second note and appeared to gain proficiency in the peer evaluation process, suggesting a possible benefit of including additional SOAP notes.

Pupils' attitudes to formative assessment in sciences class

This paper discusses formative assessment in Sciences class, which integrates the knowledge of natural and social sciences. In formative assessment there are two different assessment processes – assessment for learning and assessment as learning. Both approaches value the idea of students’ individual progress, and they have a pedocentric orientation. The paper focuses on assessment techniques used in class, which is operationalised in the empirical section of the paper. We established a hierarchical structure of offered techniques according to the criterium of usefulness. The students expressed their views on the process of self-evaluation and peer-evaluation in four dimensions (liking, difficulty, help with learning and discomfort). The approach to research is quantitative. The results show that the techniques we can find at the bottom of the hierarchy are the techniques which require a more complex cognitive deliberation. There is a positive attitude towards self-evaluation and peer-evaluation. In establishing the difference between attitudes in the dimension of help with learning there is a lack of recognition of the potential of peer-evaluation. This paper confirms the importance of formative assessment and points to the need for further research.

Best practices for online team‐based learning: Strengthening teams through formative peer evaluation

AbstractThis chapter describes best practices for the peer evaluation process in online team‐based learning (TBL). Formative peer evaluation is an essential element of effective course delivery for online TBL classes, and when done well, shapes a group of individuals with a shared set of tasks into an interdependent team of learners. More generally, emphasizing the formative over the summative aspects of peer evaluation supports the development of the soft skills and behaviors students require to become more adaptive, resilient, reflective, and collaborative human beings.

Sistema web para la evaluación de los Programas Institucionales de Tutoría de las Instituciones de Educación Superior de la RCO de la ANUIES

This article describes the process of analysis, design and implementation of a web system for the management of the evaluation processes of the Institutional Tutoring Programs of the Higher Education Institutions that make up the Central West Region of the Asociación Nacional de Universidades e Instituciones de Educación through the collaboration of the Tutoring Network and the Tutoring Department of the Universidad Tecnológica de Jalisco. This process was developed through the agile SCRUM methodology due to the periodicity of the phases and their implementation. The objective of the system is to automate the peer evaluation process in which the members of the Network participate as a continuous improvement strategy and that directly affects the comprehensive training of students, in addition to being consistent with the Quality Management Systems of the HEIs and with the recommendations of the Accrediting Experts of the Consejo para la Acreditación de la Educación Superior and the criteria of the Federación de Instituciones Mexicanas Particulares de Educación Superior as processes that ensure quality in Educational Programs, contributing to the continuous improvement of support services for students.

Doing Our Part: The University of North Carolina at Charlotte Office of Assessment and Accreditation Supports Faculty Assessment Efforts During COVID‐19

Doing Our Part: The University of North Carolina at Charlotte Office of Assessment and Accreditation Supports Faculty Assessment Efforts During COVID‐19

Students' Perception of Peer Evaluation in Oral Presentation at the University of Indonesia

This study aims at examining students’ perception of peer evaluation in a group oral presentation by English for Academic Purposes (EAP) students at the University of Indonesia. Questionnaires were administered after the peer evaluation process to 65 students studying EAP taught at the University of Indonesia. The peer assessment was incorporated into the course to seek out whether and to what extent their speaking skills may enhance and to investigate students’ attitudes towards this form of evaluation in oral presentations. Data were obtained through a Likert scale questionnaire of peer evaluation. Data were analyzed using SPSS Software. The results specified a statistically significant result of the use of peer evaluation in oral presentation. The findings of the present study suggest that, when assessment criteria are definitely established, peer assessment empowers students to evaluate the performance of their peers in a manner comparable to those of the teachers. The responses from the questionnaires show that the evaluation and comments given by students were considered fair, useful and sufficient. Results also suggest that students strongly agree to the use of this methodology because comments and evaluations given are useful for making improvements. Nevertheless, the students preferred teachers to peers to evaluate their performance in the discussion and they wanted to get more instructive comments from the teachers. As for its limitations, students highlight the responsibility that comes with it and a certain amount of distrust in fellow students’ abilities to peer-assess. The teacher’s possible roles in peer evaluation are presented at the conclusion of this study.

Guidelines for eBook Design

Neste artigo pretendemos propor diretrizes para designers iniciantes, ou seja, sem experiência anterior com o design de e-books. O primeiro passo foi uma revisão sistemática da literatura, que resultou em 1052 diretrizes, que foram analisadas iterativamente até serem reduzidas para uma quantidade menor. Depois disso, 15 designers experientes avaliaram a utilidade dessas diretrizes e excluíram as que consideraram não úteis (para designers iniciantes). Em um esforço para incorporar conhecimento especializado, pedimos a estudantes de doutorado para proporem diretrizes, que passaram pelo mesmo processo de análise e avaliação por pares, e foram adicionadas ao conjunto anterior, o que resultou em 38 diretrizes. Finalmente, três estudantes de doutorado que também são designers com experiência em design de eBooks, comentaram cada diretriz e definiram o conjunto final. O resultado é um conjunto de 13 diretrizes.

Condoning Free Loafers: What Do Role, Care, and Justice Have to Do with it?

The purpose of this mixed-methods study was to explore students’ reactions to a case involving peer rating. We asked undergraduate and graduate students at a Midwestern liberal arts university to rate the performance of a close friend. To explore how role might impact the students’ decisions, we randomly assigned participants to one of two rating scenarios. In the first scenario, students rated a fellow student who had not completed his or her fair share of a group project. In the second scenario, students were asked to play the role of a professor who had to rate a peer. We found significant differences in the overall quantitative ratings assigned by participants in the two scenarios. Participants’ answers also involved more justice considerations in the professor scenario than in the student scenario. The results suggest that students may have difficulty in being honest during peer evaluation processes.

A revisão por pares na visão dos editores das revistas de acesso aberto coletadas pelo Portal oasisbr

A pesquisa apresenta as características da avaliação por pares realizada nas revistas coletadas pelo Portal Brasileiro de Publicações Científicas em Acesso Aberto (oasisbr), assim como, apreende a opinião dos editores dessas revistas em relação aos processos de avaliação por pares utilizados e suas alternativas. Para cumprir os objetivos propostos optou-se pelo questionário como instrumento de coleta de dados e pela abordagem quanti-qualitativa. O universo da pesquisa foi delimitado pelas revistas coletadas pelo Portal oasisbr que possuem cadastro no Diretório de Políticas Editoriais das Revistas Científicas Brasileiras (Diadorim). O questionário foi enviado para 637 revistas, destas, obtiveram-se 109 respostas, o que representa 17,1% do universo da pesquisa. Identificou-se que a relação características que correspondessentre a prática real das revistas e o que os editores consideram mais adequado é marcada tanto por disparidades quanto por compatibilidades, destaca-se a incontestável notoriedade da avaliação duplo cega que figura como a avaliação utilizada por 95,3% das revistas da amostra sendo considerada a mais eficiente por 95,1% dos editores. Já em relação a publicação do nome dos avaliadores, vê-se que apenas 31,4% das revistas publicam o nome do avaliador de alguma forma, ao mesmo tempo que 49,5% dos editores concorda que o nome do avaliador deve ser publicado. Em relação às características gerais que podem elencar um sistema de avaliação por pares percebe-se que, em geral, estas são pouco utilizadas, entretanto existe uma tendência que parece se direcionar a uma maior aceitação dessas características ou pelo menos a uma consciência da efetividade dessas características para relacionar uma avaliação por pares mais eficiente.

Dynamic online peer evaluations to improve group assignments in nursing e-learning environment

ObjectivesThe purpose of this research was to evaluate the use of online peer evaluation forms for online group activities in improving group project outcomes. DesignThe investigator developed and used a web-based Google Forms® self and peer evaluation form of 2 group assignments' rubric for junior and senior nursing students. The form covered elements of the assignments including: research activity, analysis of the literature, writing of report, participation in making of presentation, overall contribution to the project, and participation in the weekly group discussions. Items were rated from 1 (did not contribute) to 5 (outstanding contribution) in addition to NA when one activity did not apply. ProcedureThe self and peer evaluation process was conducted twice: once after group assignment 1 and once after group assignment 2. The group assignments final products were done in the form of VoiceThread online presentations that were shared with the rest of the class reflecting the groups' work on a health informatics topic of interest. Data sourcesData collected as the students completed self and peer evaluations for group assignments 1 and 2. Also, optional comments regarding member performance were collected to add contextual information in addition to ratings. Students received credit for completing the peer evaluations and the grade for the particular assignment was affected by their performance based on peer evaluations of their contributions. ResultsStudents' peer evaluations showed in a color-coded spreadsheet which enabled the course faculty to view real time results of students' ratings after each assignment. The faculty provided timely and tailored feedback to groups or individuals as needed, using positive feedback and commending high performance while urging struggling individual students and groups to improve lower ratings in specific areas. Comparing evaluations of both assignments, there were statistically significant improvements among all students. The mean scores of the entire sample were skewed toward the higher end of the scale, suggesting an overall high performance group. However, analysis of the lower performing individuals showed consistent and statistically significant improvements in all areas of the evaluation criteria. ConclusionsAnonymous peer evaluation activities and timely faculty feedback in e-Learning environment can be a useful tool to faculty to improve group performance over time by engaging the learners within their groups. Peer evaluations provided real time view of group mid-semester formative evaluations that allowed the faculty to provide timely and tailored feedback on student performance which allowed for better outcomes.

Computerized sociometric assessment for preschool children

In preschool classes, sociometric peer ratings are used to measure children’s peer relationships. The current study examined a computerized version of preschool sociometric ratings. The psychometric properties were compared of computerized sociometric ratings and traditional peer ratings for preschoolers. The distributions, inter-item correlations, and reliabilities of sociometric scores were comparable between the computerized assessment and traditional assessment. The computerized assessment provided additional data for further analysis of the peer evaluation process. Therefore, computerized peer ratings are a promising tool for sociometric measurement with preschool children.

Advances in bioinformatics and biomedical engineering--special issue of IWBBIO 2013.

In the present issue of Theoretical Biology and Medical Modelling (TBioMed), it is a pleasure to present you a selection of 8 extended versions of selected papers from the International Work-Conference on Bioinformatics and Biomedical Engineering (IWBBIO 2013) held in Granada (Spain) during March 18-20, 2013. IWBBIO 2013 seeks to provide a discussion forum for scientists, engineers, educators and students about the latest ideas and realizations in the foundations, theory, models and applications for interdisciplinary and multidisciplinary research encompassing disciplines of computer science, mathematics, statistics, biology, bioinformatics, and biomedicine. The aims of IWBBIO 2013 is to create a friendly environment that could lead to the establishment or strengthening of scientific collaborations and exchanges among attendees, and therefore, IWBBIO 2013 solicited high-quality original research papers (including significant work-in-progress) on any aspect of Bioinformatics, Biomedicine and Biomedical Engineering. New computational techniques and methods in machine learning, data mining, text analysis, pattern recognition, data integration, genomics and evolution, next generation sequencing data, protein and RNA structure, protein function and proteomics, medical informatics and translational bioinformatics, computational systems biology, modelling and simulation and their application in the life science domain, biomedicine and biomedical engineering were especially encouraged. At the end of the submission process of IWBBIO 2013, and after a careful peer review and evaluation process (each submission was reviewed by at least 2 program committee members or additional reviewer), 122 papers were accepted for oral or poster presentation, according to the recommendations of reviewers and the authors’ preferences. A number of authors were invited to submit an extended version of their conference paper to be considered for special publication in this issue of Theoretical Biology and Medical Modelling (TBioMed). These authors were selected after the recommendation of the reviewers of the conference papers, the opinion of the chairs of the different sessions and the guest editors. The extended versions were again carefully reviewed by at least two independent and anonymous experts and the accepted papers, after this new review process, are presented in this issue.