Peer Assessment Processes Research Articles

Implementation of educational sequences based on peer assessment for learning key concepts of statistics

This article presents the results of applying an educational sequence implemented with technological support on an LMS and focused on peer assessment that was designed specifically to address key concepts in statistics with first-year undergraduate students. Individualized information is available for a total of n=232 students to support the empirical conclusions that are drawn. Based on the comparison of the peer assessments and the academic performance obtained in the two previous academic years in which a different methodology was applied, differential effects are found in the quality of the assignments presented. This, together with the perception of the learning by the students, suggests the incorporation of peer assessment processes in future curricular design.

Peer assessment processes in a problem-solving activity with future teachers

Assessing problem-solving remains a challenge for both teachers and researchers. With the aim of contributing to the understanding of this complex process, this paper presents an exploratory study of peer assessment in mathematical problem-solving activities. The research was conducted with a group of future Secondary mathematics teachers who first were asked to individually solve an open-ended problem and then, to assess a classmate’s answer in pairs. We present a study of two cases involving two pairs of students, each of whom assessed the solution of a third classmate. The analysis was carried out in two interrelated phases: (a) individual solutions to the mathematical problem and (b) the peer assessment process. The results show that, in both cases, the assessors were strongly attached to their own solutions, which directly influenced the assessment process, focused on aspects that involve the general problem-solving process and the results. The main difference between the evaluation processes followed by the two pairs lies in the concept of assessment. While the first pair focuses on assessing the resolution process and errors, the other focuses its discussion on giving a numerical grade.

Backward evaluation in peer assessment: A scoping review

Implementing backward evaluation as part of the peer assessment process enables students to react to the feedback they receive on their work within one peer assessment activity cycle. The emergence of online peer assessment platforms has brought new opportunities to study the peer assessment process, including backward evaluation, through the digital data that the use of these systems generates. This scoping review provides an overview of peer assessment studies that use backward evaluation data in their analyses, identifies different types of backward evaluation and describes how backward evaluation data have been used to increase understanding of peer assessment processes. The review contributes to a mapping of backward evaluation terminology and shows the potential of backward evaluation data to give new insights on students’ perceptions of what is useful feedback, their reactions to the feedback received and its consequences for feedback implementation.

Peer Assessment in Writing Expository Essays: A Research on Tenth Graders of Vocational School

The present work was devoted to: (a) exploring peer assessment instruments (i.e., assessment sheets) utilized by teachers in teaching expository essay writing for tenth graders of vocational schools; (b) discussing the implementation and (c) the impact of peer assessment of the peer assessment activities. In this study, the data were from information in the peer assessment implementation. Methods of data collection involved observation, interview, and documentation. The results showed that: (a) The peer assessment instruments were in compliance with the competence standards in student assessment. (b) The implementation of expository writing teaching and learning was to find out students’ progress and outcomes; it also aimed to improve learning processes. (c) Peer assessment activities help schools enhance their learning quality and teachers in comprehending the peer assessment processes. On top of that, the assessment provides students with practical experience in assessing their peers’ works, thus boosting students’ concentration in learning. Peer assessment also measures the affective domains of students (by which it eases teachers to select appropriate teaching-learning strategies). In conclusion, the assessment on students’ social attitude fall under a good category based on the students’ average score.

Integration of different assessment approaches: application to a project-based learning engineering course

Formative assessment and self-, peer- and co-evaluation strategies for assessing students’ learning have been developed and applied, over two academic years, to 75 students on a project-based learning course in the Chemical Engineering Degree at Cantabria University (Spain). The project proposed for teamwork learning in the Process Design course was the preliminary design of a second-generation bioethanol plant. Rubrics of formative assessment and evaluation generated positive reinforcement for the student advancing through the consecutive learning stages. The negotiated co-evaluation process has proved to be an effective tool for reflecting the distinctive contributions of each working group member, which are then successfully incorporated in the final individual mark set by the teachers. Students' perceptions regarding the proposed assessment approaches have been obtained by creating and applying an anonymous and voluntary survey instrument. The active participation of the students in the design and application of rubrics using specific criteria is proposed for peer-assessment processes in learning activities with high qualitative content and more susceptible to subjective capabilities such as oral presentations. We consider that the assessment, learning methodology and the experience gained with the survey results contribute to enhanced achievement of the learning outcomes, and to enhanced quality in the teaching-learning process.

Automatic detection of inconsistencies between numerical scores and textual feedback in peer-assessment processes with machine learning

The use of peer assessment for open-ended activities has advantages for both teachers and students. Teachers might reduce the workload of the correction process and students achieve a better understanding of the subject by evaluating the activities of their peers. In order to ease the process, it is advisable to provide the students with a rubric over which performing the assessment of their peers; however, restricting themselves to provide only numerical scores is detrimental, as it prevents providing valuable feedback to others peers. Since this assessment produces two modalities of the same evaluation, namely numerical score and textual feedback, it is possible to apply automatic techniques to detect inconsistencies in the evaluation, thus minimizing the teachers' workload for supervising the whole process. This paper proposes a machine learning approach for the detection of such inconsistencies. To this end, we consider two different approaches, each of which is tested with different algorithms, in order to both evaluate the approach itself and find appropriate models to make it successful. The experiments carried out with 4 groups of students and 2 types of activities show that the proposed approach is able to yield reliable results, thus representing a valuable approach for ensuring a fair operation of the peer assessment process.

Peer evaluations within experiential pedagogy: Fairness, objectivity, retaliation safeguarding, constructive feedback, and experiential learning as part of peer assessment

This paper describes the analysis of peer assessment processes in experiential learning courses at a Midwestern public university. Factor analysis was used in the creation of constructs measuring perceptions of fairness, objectivity, retaliation safeguarding, and constructive feedback. Constructs were based on student fears that peer assessment is unfair, peers measure based on emotions (objectivity), giving a bad score will hurt me in later courses (retaliation safeguarding), and feedback is not developmental. As well, it was determined whether there was a belief, by end users, that peer assessment is a critical component of experiential learning. The authors surveyed 477 students within semester courses practicing experiential learning pedagogies and developed constructs for fairness (α = 0.785), objectivity (α = 0.774), retaliation safeguarding (α = 0.862), constructive feedback (α = 0.692) and experiential learning (α = 0.743). No within-factor multicollinearity was found in any construct.

On business processes of computer-supported collaborative learning: A case of peer assessment system development

Andrei I. Kolomiets - Student, Faculty of Computer Science, National Research University Higher School of EconomicsAddress: 20, Myasnitskaya Street, Moscow, 101000, Russian FederationE-mail: aikolomiets@edu.hse.ruOlga V. Maksimenkova - Senior Lecturer, School of Software Engineering, Junior Researcher, International Laboratory for Intelligent Systems and Structural Analysis, National Research University Higher School of EconomicsAddress: 20, Myasnitskaya Street, Moscow, 101000, Russian FederationE-mail: omaksimenkova@hse.ruAlexey A. Neznanov - Associate Professor, School of Data Analysis and Artificial Intelligence, Senior Researcher, International Laboratory for Intelligent Systems and Structural Analysis, National Research University Higher School of EconomicsAddress: 20, Myasnitskaya Street, Moscow, 101000, Russian FederationE-mail: aneznanov@hse.ru Nowadays peer assessment is recognized as a crucial part of a wide range of active learning routines. Nevertheless, practitioners and educators speak of the complexity and high resource consumption for the implementation of this type of assessment. Undoubtedly, convenient software that supports peer assessment processes may substantially raise productivity of its participants. A review of educational literature and free software shows there are several bottlenecks in the business processes of peer assessment and key user roles. First, most of the programs examined are web-based and expand a set of tools for teachers and learners by extra interfaces. Moreover, this logically creates a new branch in the learning business process. Second, there is probably no peer assessment system which allows users to attach something other than the text to be reviewed. There is a gap in the market of free peer assessment software. This paper offers a peer assessment system specification that attempts to eliminate these disadvantages in order to improve user experience and thus increase the use of information technologies in peer assessment. The specification is based on a thorough description of the peer assessment process involving complex artifacts and double-blinded peer review. Software called PASCA (peer assessment system for complex artifacts) is introduced to illustrate the specification achieved. PASCA uses habitual e-mail services and does not affect other business processes. It supports standard features like blinding and randomization, and it provides a set of original features. They contain evaluation of arbitrary artifacts, creation of complex peer review forms with their validation and scoring, and easy analysis of data from peer assessment sessions. [1] The article was prepared within the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) and supported within the framework of a subsidy by the Russian Academic Excellence Project “5-100”.

Latest Trends in Problem Solving Assessment

Problem solving is the skill that coordinates all the cognitive, metacognitive and behavioral processes taking place when individuals encounter a previously unprecedented situation or difficulty. Metacognitive processes seem to play the most important role for resolving a problematic situation as individuals reflect on their acquired knowledge, skills and experiences, thus become aware of their capabilities and how to regulate them. Therefore, metacognitive awareness is the competence that mostly assists individuals in their attempt to construct new knowledge and reach their goals. Furthermore, individuals’ self-assessment and peer-assessment processes could reveal their level of metacognitive awareness and therefore, by far, their problem solving competency. Consequently, ICTs could capture individuals’ problem solving skills through tracking down and analyzing the latters’ cognitive and metacognitive processes as well as their behavioral patterns. The aforementioned computer-based assessment could consist of a fuzzy expert system with domain knowledge from an automated task-based test with particular solution strategies in combination with log data for identifying and classifying one’s level of problem solving ability according to specific criteria.

The PASCA: A Mail Based Randomized Blinded Peer Assessment System for Complex Artifacts

Active learning and formative assessment techniques are the cutting edge of the modern education. This paper considers peer assessment automation and touches the topic of high actuality in advancing ICT for active learning. The roots of the study are obtained difficulties in adoption of the currently available peer assessment systems to engineering education introductory courses. The main goal of the paper is to collect software requirements and to build a peer assessment system, which may be easily agreed with standard educational routines (e.g. e-mailing) and which is be able to support complex artifacts interchange during a peer assessment session. The software requirements specification (SRS) for such a system has been created as a result of reviewing educational studies, technical reports and academic publications on common peer assessment processes and existing peer assessment systems. We also introduce the open source Mail-based Randomized Double-Blinded Peer-assessment System for Complex Artifacts called PASCA, which is developed according to this SRS and the results of PASCA adoption to an introductory programming course.

Creativity in and between Collaborative Peer Assessment Processes in Higher Distance Education

The study investigates in what ways the combination of self-assessment and collaborative peer assessment can support students’ creative-and critical-abilities, as well as providing opportunities for meta-cognitive learning. The study is informed by sociocultural theories research traditions and computer supported collaborative learning, CSCL. Data were collected from 22 student teachers peer assessment processes, including peer feedback and self-assessment during two consecutive 15 credit web-based courses. The analytical framework was based on Toulmin’s argument model (1958) and Hattie and Timperley’s (2007) feedback model. The results provide a broader perspective on collaborative peer assessment processes by distinguishing, identifying and describing the meaning content in the students’ peer feedback and self-assessment, and the relationships between these. Quality of content and creativity in formulating the responses can be linked to creativity as “higher order thinking skills”. Peer assessment processes can thus function as creative exercises or as a tool to support such skills.

Peer and Self-assessment of Professionalism in Undergraduate Medical Students at the University of Calgary

Background: Peer and self assessment processes are integral to the development of professional behaviours. The purpose of this study was to assess the Rochester Peer Assessment Tool (RPAT) among a group of volunteer first year students.Methods: We assessed feasibility through participation rates. The evidence for the validity of instrument scores was ascertained through an exploratory factor analysis, MANOVA to determine age and gender differences, and a discrepancy analysis between the self and peer data. Reliability analyses included the Cronbach's alpha analysis and G- and D-studies. Students completed a feedback questionnaire to provide data about acceptability.Results: Self and peer data were collected for 46 and 44 students, respectively. Each student had a mean of 7.2 peer assessments (out of a possible 8). The factor analysis identified two factors, interpersonal skills and work study habits. The discrepancy analysis showed students in the lowest/highest quartiles, as assessed by peers, had higher/lower self means than peer means. The G-coefficient was Ep2 = 0.77. Student feedback was positive.Conclusions: RPAT was feasible in our setting, was acceptable to the students, and has been adopted as a mandatory part of our program for first and second year students. The study added to the evidence base for the reliability and validity of the RPAT instrument scores as a method of assessing professional behaviours.

Give and take: A re‐analysis of assessor and assessee's roles in technology‐facilitated peer assessment

AbstractThe study examined how playing two roles as assessors and assessees in technology‐assisted peer assessment contributes to students' performance. Data from a previous study was recoded and analysed to understand peer assessment processes from a different angle. Findings of our previous study supported the importance of the assessor's role, but not the assessee's role. In the present study, the assessee's role was re‐examined based on the assessee's ability to critically judge and act upon peer feedback, instead of quality of peer feedback that they received. Regression analysis was conducted, and results suggested that how students responded to peer feedback, as indicated by the number of good versus misleading suggestions incorporated, significantly predicted their final project scores. The findings support the importance of both assessor and assessee's roles in peer assessment and provide valuable implications for effective implementation of peer assessment.

Improving self‐ and peer assessment processes with technology

PurposeAs a way of focusing curriculum development and learning outcomes universities have introduced graduate attributes, which their students should develop during their degree course. Some of these attributes are discipline‐specific, others are generic to all professions. The development of these attributes can be promoted by the careful use of self‐ and peer assessment. The authors have previously reported using the self‐ and peer assessment software tool SPARK in various contexts to facilitate opportunities to practise, develop, assess and provide feedback on these attributes. This research and that of the other developers identified the need to extend the features of SPARK, to increase its flexibility and capacity to provide feedback. This paper seeks to report the results of the initial trials to investigate the potential of these new features to improve learning outcomes.Design/methodology/approachThe paper reviews some of the key literature with regard to self‐ and peer assessment, discusses the main aspects of the original online self‐ and peer assessment tool SPARK and the new version SPARKPLUS, reports and analyses the results of a series of student surveys to investigate whether the new features and applications of the tool have improved the learning outcomes in a large multi‐disciplinary Engineering Design subject.FindingsIt was found that using self‐ and peer assessment in conjunction with collaborative peer learning activities increased the benefits to students and improved engagement. Furthermore it was found that the new features available in SPARKPLUS facilitated efficient implementation of additional self‐ and peer assessment processes (assessment of individual work and benchmarking exercises) and improved learning outcomes. The trials demonstrated that the tool assisted in improving students' engagement with and learning from peer learning exercises, the collection and distribution of feedback and helping them to identify their individual strengths and weaknesses.Practical implicationsSPARKPLUS facilitates the efficient management of self‐ and peer assessment processes even in large classes, allowing assessments to be run multiple times a semester without an excessive burden for the coordinating academic. While SPARKPLUS has enormous potential to provide significant benefits to both students and academics, it is necessary to caution that, although a powerful tool, its successful use requires thoughtful and reflective application combined with good assessment design.Originality/valueIt was found that the new features available in SPARKPLUS efficiently facilitated the development of new self‐ and peer assessment processes (assessment of individual work and benchmarking exercises) and improved learning outcomes.

Effective peer assessment processes: Research findings and future directions

Despite the popularity of peer assessment (PA), gaps in the literature make it difficult to describe exactly what constitutes effective PA. In a literature review, we divided PA into variables and then investigated their interrelatedness. We found that (a) PA's psychometric qualities are improved by the training and experience of peer assessors; (b) the development of domain-specific skills benefits from PA-based revision; (c) the development of PA skills benefits from training and is related to students' thinking style and academic achievement, and (d) student attitudes towards PA are positively influenced by training and experience. We conclude with recommendations for future research.

Assessment of Family Physicians' Performance Using Patient Charts

Peer-assessment processes with chart review have been used for many years to assess the clinical performance of physicians. The Quebec medical licensing authority has been required by provincial law to assess the practicing Quebec physicians on a nonvoluntary basis. During the period from January 2001 to November 2004, 25 family physicians in active practice were randomly selected from a pool of about 300. For each physician, 25 to 40 patients' medical charts were randomly selected to evaluate the interrater reliability of peer-review assessment of medical charts and to compare ratings based on chart review with a chart-stimulated recall interview to those based on chart review alone. The concordance between chart review alone and that of chart review with chart-stimulated recall interview was 75% for chart keeping, 69% for clinical investigation, 81% for diagnostic accuracy, and 74% for treatment plan. Ratings based on chart review alone achieve moderate levels of reliability (Kappa = 0.44 to 0.56). It appears that some important information about quality of care is missed when only chart review is used.

Students’ experience of peer assessment in a professional course

This paper reports a case study that investigated the practice of peer assessment in a professional course, with particular emphasis on students’ experience. It was found that peer assessment processes were beneficial to students’ learning and development as professionals. The case study pointed to several conditions for effective implementation of peer assessment, which also relate to assessment in general. These conditions include: (a) adequate and appropriate preparation for the use of peer assessment; (b) alignment of assessment, learning objectives and the broader purpose of the course (e.g. preparation as professionals); (c) the availability of assistance from a teacher throughout the peer assessment process; and (d) constructive discussions following peer assessment, sensitively handled by a teacher.

Explicitness of criteria in peer assessment processes for first-year engineering students

This article will outline the motives for the implementation of peer assessment as put into action at the first year of the Civil Engineering course of the University of Minho. The implementation of new assessment methods was a consequence of the successful implementation of peer assessment at other engineering courses. During the semester, three assessment moments took place, in which students assessed the work of their peers. Assessment criteria were defined in a negotiation process between students and teacher. The students subsequently graded the work of their colleagues and had to provide a clear justification for each given grade. The aim of this method is to involve students in the subject, enhance their motivation and deepen their learning. Advantages and disadvantages of peer assessment are discussed. The results that have been obtained so far draw attention to the nature of assessment criteria and the corresponding explicitness.

Peer feedback: the learning element of peer assessment

This paper focuses on peer feedback in relation to assessment processes. It examines the rationale for peer feedback, emphasizing its potential for enhanced student learning. We draw on relevant literature to argue that the dominance of peer assessment processes using grades can undermine the potential of peer feedback for improving student learning. The paper throws further light on the issue by drawing on a large-scale questionnaire survey of tertiary students (1740) and academics (460) in Hong Kong, supplemented by interview data. The findings indicate that a significant number of academics and students resist peer assessment using grades and that the majority report that students never or rarely grade each other in assessment activities. This paper explores why there is resistance, in particular, by academics to peer assessment and argues the case for a peer feedback process as an end in itself or as a precursor to peer assessment involving the awarding of marks. It also recommends some strategies for promoting peer feedback, through engaging students with criteria and for embedding peer involvement within normal course processes.

Self-, peer- and teacher-assessment of student essays

This study focuses on comparing the results of self-, peerand teacher-assessment of student essays, as well as on exploring students' experiences of the self- and peer-assessment processes. Participants were 15 law students. The scoring matrix used in the study made assessment easy, according to both teachers and students alike. Self-assessment was sometimes considered difficult, because the students felt it impossible to be objective when considering their own work. In peer-assessment, the students found it difficult to be critical when assessing the essay of a peer. The students found it easier to assess technical aspects of the essays when compared to aspects related to content.