Tasks Of Peer Reviewers Research Articles

Effects of an AI-supported approach to peer feedback on university EFL students' feedback quality and writing ability

This study examined the impact of an artificial intelligence (AI)-supported approach to peer feedback provision on the feedback quality and writing ability of English as a foreign language (EFL) student reviewers. The researchers integrated an AI chatbot named Eva into an online peer review system to assist students in generating feedback. A total of 124 Chinese undergraduate students participated in nine peer review tasks over three weeks, with 64 students in the experimental group (using Eva) and 60 students in the control group (without AI support). Pre- and post-tests were conducted to assess the quality of student reviewers' peer feedback and these feedback providers' writing performance before and after the intervention. The findings revealed that the intervention significantly enhanced students' feedback quality. Additionally, the study showed that the proposed approach improved feedback providers' writing ability. This research underscores the potential of AI technology in enhancing EFL writing instruction.

Towards Understanding Teacher Mentoring, Learner WCF Beliefs, and Learner Revision Practices Through Peer Review Feedback: A Sociocultural Perspective

Background. The existing literature has focused on learner perceptions or beliefs about peer review tasks over the recent decade. However, little has been known about the relationships among learner beliefs about written corrective feedback (WCF), related teacher mentoring process, and learner revision practices.
 Purpose. We thus aimed at addressing the gap by exploring how teacher mentoring and learner WCF beliefs may inform learner revision practices in the peer-reviewed process.
 Methods. In this mixed-method study, we included four Chinese EFL students majoring in English as the participants and collected their WCF belief survey data. We also collected their actual practice data through PeerCeptiv, an online writing and rewriting platform. In addition, we traced the teacher mentoring practices and interviewed the participants about their beliefs and practices in the peer review and back-evaluation process.
 Results. Through the mixed-methods design, we reported our major findings: the student participants believed empathy and resonance was the primary advantage of peer feedback, and teacher mentoring facilitated them in understanding and performing the peer review and revision tasks; we also found the student review process consisted of evaluating, resonating, learning, and reflecting practices and the student revision process included crediting, arguing, correcting, and polishing practices.
 Implications. From a sociocultural perspective, we centered our discussion on these research findings by claiming that scaffolding in different forms work together enhance learner performance and student beliefs appear in a complex manner with student actual revision practices. We also offered insights for future studies and practical implications for language teachers.

Comparative Analysis of Model-Based and Traditional Systems Engineering Approaches for Architecting a Robotic Space System Through Automatic Information Transfer

This study seeks to compare the quantity of information that is automatically transferred through the associations generated using a model-based systems engineering (MBSE) approach versus a traditional systems engineering approach to measure the benefits of MBSE in architecting a robotic space system. Both an MBSE approach and non-MBSE approach were applied to architecting an orbiting sample Capture and Orient Module (COM) system concept for a Capture, Containment, and Return System (CCRS) payload concept for Mars Sample Return (MSR). These approaches were applied in parallel to provide a side-by-side comparison of the approaches. The approaches were analyzed using design structure matrices (DSM) and evaluated based on the amount of information transferred between process tasks manually (e.g., elements physically typed into text boxes in a presentation slide) vs automatically (e.g., elements automatically filled out within a block in a model view due to explicitly defined element associations). A total of 4,819 information element transfers were traced in DSMs and used to quantitatively compare the two approaches. The non-MBSE approach required manual transfer for all 4,819 information elements. The MBSE approach required manual transfer for 4,189 information elements and automatic transfer for 630 information elements, providing a minor increase in the automation of information transfer relative to the non-MBSE approach. By incorporating the use of additional MBSE artifacts into the trade study and peer review tasks, manual transfer could potentially be reduced to 931 information elements, and automatic transfer increased to 3,888 information elements.

Cross-cutting skills: strategies for teaching & learning

ABSTRACT This conceptual article focuses on teaching and learning strategies aimed at helping students in higher education develop cross-crossing skills such as communication, critical thinking, teamwork, problem-solving, and working with diverse others. Within the framework of high impact practices and their elements, the article shares practical ideas for developing and responding to assignments that emphasize cross-crossing skills and measuring associated learning outcomes. The article focuses on developing assignments with an emphasis on designing and structuring team ePortfolios, collaborative projects, peer review tasks, and reflection; responding to assignments with techniques such as rubrics, course communications, and self-regulation strategies; and measuring outcomes to determine the effectiveness of assigned tasks with the goal of informing curricular and pedagogical modifications.

Journal editors’ perspectives on the roles and tasks of peer reviewers in biomedical journals: a qualitative study

ObjectivePeer reviewers of biomedical journals are expected to perform a large number of roles and tasks, some of which are seemingly contradictory or demonstrate incongruities between the respective positions of...

Metacognitive Instruction for Peer Review Interaction in L2 Writing

While previous research on peer feedback in L2 writing has stressed the importance of training in giving useful comments on peer’s writing, very few studies have specifically explored metacognitive training in peer review interaction in terms of the perception and actual reviewing practices of L2 learners. This mixed-methods study employed a 12-week intervention course in L2 writing, in which eighteen Secondary One students, aged 12–13 years, received metacognitive training in peer review interaction. The training focused on metacognitive knowledge and metacognitive regulations. The results showed that, first, metacognitive training in peer review interaction helped change the perception of these young learners and increased their level of engagement and collaboration during the five peer review tasks. Second, metacognitive training appeared to encourage students to provide more content-related feedback than language-related feedback during dyadic interactions. Finally, it was found that the students were able to seriously consider their peer feedback when revising the peer-reviewed drafts, but written feedback had a much higher chance of being incorporated than did oral feedback. The pedagogical implications of these findings were discussed.

A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

BackgroundAlthough peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals.MethodsComprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes.ResultsAfter screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items).We also extracted 2026 statements related to peer reviewers’ tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items).ConclusionsPeer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.

Automatic assignment of reviewers in an online peer assessment task based on social interactions

AbstractOnline peer assessment tasks are very popular and have unique characteristics that improve learning and encourage social interactions in a distance education environment. Unfortunately, social factors have usually been ignored in the process of selecting reviewers for online peer assessment tasks. We hypothesise that this fact could have some influence on the lack of engagement and participation by some learners. For this reason, we propose an approach in which social network analysis techniques, expert criteria, and Bayesian reasoning are applied to select reviewers with the objective of increasing participation in peer review tasks. The approach is divided into two elements. On the one hand, we have developed an influence diagram template that structures a set of proposed social network analysis variables in accordance with expert criteria. This influence diagram template can be easily updated for any course simply by eliciting a minimal set of parameters. On the other hand, we have instantiated the proposed influence diagram template to produce an influence diagram network to quantify the quality of reviewer assignment for an online peer assessment task. In an online experiment, we verified that the consideration of social factors can increase participation in a peer assessment task.

Score-Group Framing Negatively Impacts Peer Evaluations

How does group membership framing affect the feedback students provide learners? This paper presents two between-subjects experiments investigating the effect of Ingroup/Outgroup membership on effort spent in peer evaluations, and whether the group membership criterion type affects quality and stringency of evaluation. Two peer-review assignments were implemented in two separate classes. In the first study, students were nominally grouped by location they sat in class and non-nominally grouped by current class score; each was asked to review an Ingroup and Outgroup peer assignment. A second study randomly assigned students to one of four group types (random, score, motivation, and location); student reviewed two Ingroup assignments. In both studies, score-grouped students graded their peers more stringently than students grouped by location. These studies illustrate for system designers the impacts of group framing - and the disclosure of that-in peer review tasks.

A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

IntroductionThe primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear...

Design and assessment of an assignment-based curriculum to teach scientific writing and scientific peer review

A writing-intensive, upper-level undergraduate course which integrates content, context, collaboration, and communication in a unique fashion, is described. The topic of the seminar is “Scientific Writing in Chemistry” and an assignment-based curriculum was developed to instruct students on best practices in all aspects of science communication and to educate students about the scientific publication process and peer review. To effectively teach students how to understand science, both the content and the process must be included. Peer review is an integral and essential part of the process of science and the peer review tasks in the course described in this paper evolve from rubric-based peer assessments to free-format peer review. The curriculum was developed for a semester-long, three-hour seminar with limited enrolment. The curriculum was taught in the Spring semesters of 2010 - 2014 and enrolment data and results of evaluations collected over four years are presented to demonstrate the success of the implementations

Evidence of Accommodation to L2 Pragmatic Norms in Peer Review Tasks of Japanese Learners of English

Evidence of Accommodation to L2 Pragmatic Norms in Peer Review Tasks of Japanese Learners of English