Online Peer Assessment Research Articles

Students' interpersonal perspectives on, conceptions of and approaches to learning in online peer assessment

<span>The purpose of this study is to qualitatively explore students' interpersonal perspectives (i.e.,</span><em>psychological safety, value diversity, trust</em><span> and </span><em>social interdependence</em><span>) on, conceptions of and approaches to learning in an online peer assessment activity required for creating digital artistic works. Twenty-three college students in Taiwan volunteered for interview after the assessment activity. This study found that, the more positive interpersonal perspectives (i.e., high psychological safety, low value diversity for goals, trust in self as assessors and positive social interdependence) the students perceived, the higher level of conceptions and approaches (i.e., cohesive conceptions and deep approaches) they held. However, for certain students, high value diversity for criteria and trust in peers as assessors might play a role in their use of deep approaches, but not in their level of conceptions. Accordingly, a model of the associations between students' interpersonal perspectives, conceptions and approaches is proposed.</span>

Understanding the Effectiveness of Online Peer Assessment: A Path Model

Peer assessment has been implemented in schools as both a learning tool and an assessment tool. Earlier studies have explored the effectiveness of peer assessment from different perspectives, such as domain knowledge and skills, peer assessment skills, and attitude changes. However, there is no holistic model describing the effects of cognitive and affective feedback, grading, and prior knowledge, and earlier studies often discuss learning outcomes without ascribing them to particular causes. Moreover, few studies have differentiated between the effects of peer assessment on both assessors and assessees. This study used a path model to investigate how two online peer-assessment activities—rubric-based assessment and peer feedback—affected the learning performance of assessors and assessees. One hundred and eighty-one high school students engaged in peer assessment via an online system—iLAP. Several path models were tested and we found that the original model did not fit when the variable of cognitive feedback from peers was included. The best fit model was the one in which direct paths from cognitive feedback from peers and student exam scores in a prior Humanities course were removed.

Online peer assessment: helping to facilitate learning through participation

The focus of this paper is on the combination of enquiry-based learning, information literacy and e-learning and how they are embedded in an online peer assessment exercise. What it shall present is a structure and strategy that aids student learning in the short and long-term. Ninety eight students completed a questionnaire before and after a three-week online peer assessment exercise during a first year undergraduate research and study skills module. The results demonstrate that a significant number of students valued the design of the exercise and the benefits it can have on their future learning and development. The paper concludes by suggesting that new and innovative ways of assessment are needed to keep engaging students and develop their learning in different ways.

Enhancing the quality of peer review by reducing student “free riding”: Peer assessment with positive interdependence

AbstractCollaborative learning tasks have been hampered by problems of unequal participation such as the presence of a free rider or dominance by a member of a group. Researchers in the collaborative learning domain respond to these problems with approaches designed to establish personal accountability for each student and relations of positive interdependence between students. These two notions have been borrowed and integrated with the present approach, which is designed for use in both traditional and online peer assessment. The key assumption is that introducing a greater degree of positive interdependence and personal accountability into the assessment process may reduce or eliminate free riding, while with students performing their ratings individually, dominance is unlikely to be an issue.Practitioner NotesWhat is already known about this topic A number of studies have found generally positive results for peer assessment. When implementing peer assessment, teachers are faced with a number of threats, including favoritism, unfairness, distortion of marking, low‐quality comments and a lack of engagement. Numerous peer assessment tools have been developed to make assessment more fair for students. What this paper adds The focus of this paper is on countering the threats of carelessness and favoritism. The paper describes an innovative approach called “Peer Assessment with Positive Interdependence” (PAPI), which was designed to counter these threats and enhance the overall quality of peer review. The PAPI approach encourages students to take a broader and more comprehensive perspective during peer assessment, enhancing their ability as “critical assessors.” Implications for practice and/or policy PAPI enables students to benefit from the known advantages of peer assessment while minimizing threats to its effectiveness. PAPI makes it possible to target particular instructional goals, through the use of different weightings at different stages of the learning process. In particular, if teaching a class in which most students are “quality achievers” with only a few “critical assessors” teachers might adjust the weightings to promote the development of critical thinking skills. Conversely, if the class contains a high number of “critical assessors” but only a small number of “quality achievers,” the weightings could be adjusted to focus more on the individual assignments.

Exploring students’ emotional responses and participation in an online peer assessment activity: a case study

In the social interactions among individuals of learning communities, including those individuals engaged in peer assessment activities, emotion may be a key factor in learning. However, research regarding the emotional response of learners in online peer assessment activities is relatively scarce. Detecting learners’ emotion when they make comments and responses to peers may be helpful to better understand their learning and participation behaviours. Therefore, the aim of this study is to explore the emotional responses and participation of university students in online peer assessment. The self-assessment manikin (SAM) (Lang, P.J. (1985). The cognitive psychophysiology of emotion: Fear and anxiety, In A.H. Tuma & J.D. Maser (Eds.), Anxiety and the anxiety disorder (pp. 131–170). Hillsdale, NJ: Lawrence Erlbaum.) measurement was integrated into the online peer assessment system to measure learners’ emotional responses, and a cluster analysis was implemented to identify various features of the different groups. The results show that response behaviours may reflect a learner’s participation and status of emotional response during an online peer assessment process. Using cluster analysis, the groups were divided into three clusters; each cluster's representation of emotional response patterns was then analysed and discussed. Finally, this study proposed technical mechanisms in online peer assessment systems (i.e. data mining or RSS feeds) for improving learners’ participation and supporting teachers or researchers to intervene to provide relevant assistance and guidance when necessary.

Online peer assessment: effects of cognitive and affective feedback

This study reports the effects of online peer assessment, in the form of peer grading and peer feedback, on students’ learning. One hundred and eighty one high school students engaged in peer assessment via an online system—iLap. The number of grade-giving and grade-receiving experiences was examined and the peer feedback was coded according to different cognitive and affective dimensions. The effects, on both assessors and assessees, were analyzed using multiple regression. The results indicate that the provision by student assessors of feedback that identified problems and gave suggestions was a significant predictor of the performance of the assessors themselves, and that positive affective feedback was related to the performance of assessees. However, peer grading behaviors were not a significant predictor of project performance. This study explains the benefits of online peer assessment in general and highlights the importance of specific types of feedback. Moreover, it expands our understanding of how peer assessment affects the different parties involved.

Different identity revelation modes in an online peer-assessment learning environment: Effects on perceptions toward assessors, classroom climate and learning activities

The effects of four different identity revelation modes (three fixed modes: real-name, anonymity, nickname and one dynamic user self-choice mode) on participants' perceptions toward their assessors, classroom climate, and past experience with the learning activity in which they were engaged were examined. A pretest-posttest quasi-experimental research design was adopted. Eight fifth-grade classes (age 10-11, N = 243) were randomly assigned to four different identity revelation modes in order for them to participate in the study. An online learning system that allows students to contribute to and benefit from the process of question-generation and peer-assessment was adopted. Data analysis confirmed that different identity modes lead participants to view their assessors differently. Specifically, participants assigned to the self-choice and real-name identity revelation modes tended to view their assessors more favorably than those in the anonymity and nickname groups. The empirical significance of the study as well as suggestions for learning system development, instructional implementation and future study are provided.

Project-Based Learning for First-Year Engineering Students: Design of CO2 Sequestration

This paper evaluates the teaching and learning outcomes from a first-year project-based engineering course at the University of New South Wales. The students worked in teams to design and build a device to sequester “CO2” into a reservoir. We present our observation of the peer assessments, performance prediction and diversity of the students. We found that some teams collaborated more effectively than others. We quantified this effectiveness using an online peer assessment that enables students to rate the performance of their teammates. These ratings were incorporated into the assignment of individual marks for group assignments. The analysis of the marks students achieved for various assessments and components of the course is presented. We observed that female students tended to out-perform their peers in both individual and team assessments, and students who did well in individual components of the course also did well in the team components. The use of early assessments to predict performance in later assessments seems to be difficult. However, there is a strong connection between students’ results for their design proposal and their results for the final design work. We are able to show statistically a significant improvement for marks for presentations but not for report writing. Anonymous surveys showed that about 90% of students were satisfied with their experience of the course. Students found this course to be challenging and enjoyable, giving them a good start in developing their skills as engineers.

Anonymous Peer Assessment of Medication Management Reviews

To investigate whether pharmacy students' anonymous peer assessment of a medication management review (MMR) was constructive, consistent with the feedback provided by an expert tutor, and enhanced the students' learning experience. Fourth-year undergraduate pharmacy students were randomly and anonymously assigned to a partner and participated in an online peer assessment of their partner's MMR. An independent expert graded a randomly selected sample of the MMR's using a schedule developed for the study. A second expert evaluated the quality of the peer and expert feedback. Students also completed a questionnaire and participated in a focus group interview. Student peers gave significantly higher marks than an expert for the same MMR; however, no significant difference between the quality of written feedback between the students and expert was detected. The majority of students agreed that this activity was a useful learning experience. Anonymous peer assessment is an effective means of providing additional constructive feedback on student performance on the medication review process. Exposure to other students' work and the giving and receiving of peer feedback were perceived as valuable by students.

Learning through science writing via online peer assessment in a college biology course

This study used an online peer assessment activity to help 47 college students to learn biology through writing. Each student submitted a biology writing report to an online system and then experienced three rounds of peer assessment. During the online peer assessment process, self, peer and expert evaluation scores for the writing were gathered across three rounds. It was found that self-assessment scores were not quite consistent with the expert's scores, but the peer assessment scores demonstrated adequate validity with the expert's evaluation. In particular, when the students had more rounds of peer assessment for reviewing the writing, the validity of the peer scores was enhanced. An examination of the students' writing scores, allocated by peers and expert, indicated that the students significantly improved the writing as the peer assessment activity proceeded. Content analyses of the students' writing also revealed that their writing gradually developed with significantly better coverage, richness and organization resulting from the online peer assessment activity.

Students’ perceptions about peer assessment for writing: their origin and impact on revision work

We investigate students’ negative perceptions about an online peer assessment system for undergraduate writing across the disciplines. Specifically, we consider the nature of students’ resistance to peer assessment; what factors influence that resistance; and how students’ perceptions impact their revision work. We do this work by first examining findings from an end-of-course survey administered to 250 students in ten courses across six universities using an online peer assessment system called SWoRD for their writing assignments. Those findings indicate that students have the most positive perceptions of SWoRD in those courses where an instructor graded their work in addition to peers (as opposed to peer-only grading). We then move to an in-depth examination of perceptions and revision work among 84 students using SWoRD and no instructor grading for assessment of writing in one university class. Findings from that study indicate that students sometimes regard peer assessment as unfair and often believe that peers are unqualified to review and assess students’ work. Furthermore, students’ perceptions about the fairness of peer assessment drop significantly following students’ experience in doing peer assessment. Students’ fairness perceptions—and drops in those perceptions—are most significantly associated with their perceptions about the extent to which peers’ feedback is useful and positive. However, students’ perceptions appear to be unrelated to the extent of their revision work. This research fills a considerable gap in the literature regarding the origin of students’ negative perceptions about peer assessment, as well as how perceptions influence performance.

Taiwan college students' self-efficacy and motivation of learning in online peer assessment environments

Online peer assessment is an innovative evaluation method that has caught both educators' and practitioners' attention in recent years. The purpose of this study was to develop relevant questionnaires for teachers to understand student self-efficacy and motivation in online peer assessment learning environments. A total of 205 college students with experience in online peer assessment participated in this study. Two questionnaires measuring students' online peer assessment self-efficacy (OPASS) and their motivations in online peer assessment learning environments (MOPAS) were developed. The former included three self-efficacy scales: evaluating, receiving and reacting. The latter included two scales: intrinsic motivation and extrinsic motivation. Through factor analysis, both revealed highly satisfactory validity and reliability in assessing students' self-efficacy and motivation in online peer assessment learning environments. Moreover, the students' responses also showed that they were highly confident and strongly intrinsically motivated when participating in an online peer assessment learning environment. Finally, the interplay between the scales of OPASS and those of MOPAS was explored and the reciprocal relationship between students' self-efficacy and motivation in an online peer assessment learning environment was also highlighted.

Conceptions of and approaches to learning through online peer assessment

The present study investigated junior college students' conceptions of and approaches to learning via online peer assessment (PA) using a phenomenographic approach. Participants were 163 college students. Students were asked to accomplish a given learning task via an online PA system. Of the participants, 62 were interviewed after the activity. The interviews revealed hierarchically related and qualitatively different categories of conceptions and approaches to learning via online PA. The main and achieved levels of conceptions of and approaches to learning were determined. The results showed that, within each level, conceptions emphasizing a fragmented and cohesive learning tended to be associated with approaches focusing on surface and deep learning, respectively. In addition, students with cohesive learning conceptions and deep learning approaches were likely to make greater progress in the early stages of online PA activity. The present study finally found that approaches to learning via online PA were less related to the learning outcomes than conceptions of learning.

An educational research course facilitated by online peer assessment

This study implemented an online peer assessment activity to help 52 Taiwanese in‐service science and mathematics teachers to develop research proposals in an educational research course. The participants were divided into 16 groups, and each group was required to submit a proposal via three rounds of online peer‐assessment activity. This study found that, through the online peer‐assessment activity, the in‐service teachers could enhance their proposals at the initial stage of the activity, but there was almost no substantial improvement at the later stage. In addition, the amount of feedback they obtained was positively related to the performance from the first round to the second round; however, few significant relationships were found from the second to the third round. This study also showed that the process of peer assessment could be used to clarify assessment criteria and that sufficient support from the instructor can maintain participants' attitudes towards this activity, which were relatively optimistic throughout.

Redefining the Student-Teacher Relationship Through Online Sharing Activities

The student-teacher relationship should be a critical factor for successful teaching and learning in design education. In tradition, the relationship is defined as a master-apprentice, so design teachers’ visual assessment capability and technical standards significantly affect students’ quality of learning and achievements. However, there are some negative aspects of the master-apprentice relationship in design education that it may restrict student experiences to cultural diversity and interdisciplinary learning through various interactions with other students. A visual design subject was designed to adapt a new learning method that is to share students’ work and assessment through an asynchronous communication tool. This method was expected to reduce the negative aspects of the master-apprentice relationship and enhance peer-to-peer interactions and individualistic collaboration. A survey with two types of student groups in terms of their levels of participation was conducted to evaluate student experiences to this method. The outcomes implicate that online peer assessment is helpful to reduce the negative aspects of master-apprentice relation and can be useful for achieving the ultimate purpose of design education.

Peering into large lectures: examining peer and expert mark agreement using peerScholar, an online peer assessment tool

AbstractAs class sizes increase, methods of assessments shift from costly traditional approaches (e.g. expert‐graded writing assignments) to more economic and logistically feasible methods (e.g. multiple‐choice testing, computer‐automated scoring, or peer assessment). While each method of assessment has its merits, it is peer assessment in particular, especially when made available online through a Web‐based interface (e.g. our peerScholar system), that has the potential to allow a reintegration of open‐ended writing assignments in any size class – and in a manner that is pedagogically superior to traditional approaches. Many benefits are associated with peer assessment, but it was the concerns that prompted two experimental studies (n = 120 in each) using peerScholar to examine mark agreement between and within groups of expert (graduate teaching assistants) and peer (undergraduate students) markers. Overall, using peerScholar accomplished the goal of returning writing into a large class, while producing grades similar in level and rank order as those provided by expert graders, especially when a grade accountability feature was used.

Online peer assessment in an inservice science and mathematics teacher education course

Online or web-based peer assessment is a valuable and effective way to help the learner to examine his or her learning progress, and teachers need to be familiar with the practice before they use it in their classrooms. Therefore, the purpose of our study was to design an online peer assessment activity for 37 inservice science and mathematics teachers. The teachers were divided into 12 groups, and each group was required to present a master's thesis proposal at the end of a research methods course. Each participant gave feedback and ratings to several group projects during a three-round online peer assessment activity. Two instructors’ scores and peers’ scores were in low to medium correlation. The quality of group projects and participants’ feedback improved, but a decrease in attitudes toward peer assessment was found. The results of the study are discussed and implications are provided.

The development of science activities via on-line peer assessment: the role of scientific epistemological views

This study implemented an online peer assessment learning module to help 36 college students with the major of pre-school education to develop science activities for future instruction. Each student was asked to submit a science activity project for pre-school children, and then experienced three rounds of peer assessment. The effects of the online peer assessment module on student learning were examined, and the role of Scientific Epistemological Views (SEVs) in the learning process was carefully investigated. This study found that student peers displayed valid scoring that was consistent with an expert’s marks. Through the online peer assessment, the students could enhance the design of science activities for future instruction; for instance, the science activities became more creative, science-embedded, feasible and more suitable for the developmental stage of pre-school children. More importantly, students with more sophisticated (constructivist-oriented) SEVs tended to progress significantly more for designing science activities with more fun, higher creativity and greater relevancy to scientific knowledge, implying that learners with constructivist-oriented SEVs might benefit more from the online peer assessment learning process. These students also tended to offer more feedback to their peers, and much of the peer feedback provided by these students was categorized as guiding or helping peers to carefully appraise and plan their science activity projects. This study finally suggested that an appropriate understanding regarding the constructivist epistemology may be a prerequisite for utilizing peer assessment learning activities in science education.

On-line peer assessment and the role of the peer feedback: A study of high school computer course

The purposes of this study were to explore the effects and the validity of on-line peer assessment in high schools and to analyze effects of various types of peer feedback on students. The participating students, a total of 184 10th graders, developed their individual course projects by involving the on-line peer assessment learning activities in a computer course. The peer assessment activities consisted of three rounds, and each of the students acted as an author and a reviewer. Research data as evaluated from peers and experts indicated that students significantly improved their projects as involving the peer assessment activities. The scores determined by the learning peers were highly correlated with those marked by the experts, indicating that peer assessment in high school could be perceived as a valid assessment method. Moreover, this study also examined the relationships between the types of peer feedbacks in which students obtained from peer assessment and the subsequent performance of their projects. We categorized peer feedbacks into four types: Reinforcing, Didactic, Corrective and Suggestive. It was found that Reinforcing peer feedback was useful in helping students’ development of better projects; however, Didactic feedback and perhaps Corrective feedback provided by peers might play an unfavorable role for subsequent improvement of students’ projects. The Suggestive feedback may be helpful in the beginning of peer assessment activities; however, in the later parts of peer assessment, the effect of this type of feedback on learning might not be significant.

Attitudes towards peer assessment: a comparison of the perspectives of pre‐service and in‐service teachers

In recent years, peer assessment has been increasingly used as an alternative method of assessment in classrooms. The study described in this paper employed a 13‐item Likert‐scale instrument to evaluate participants’ attitudes towards both general (seven items) and online peer assessment (six items). A sample of 280 pre‐service teachers and 108 in‐service teachers from northern Taiwan participated in the study. A difference in attitude was found between pre‐service and in‐service teachers, because the latter viewed peer assessment as a learning aid. Our results also showed that male pre‐service teachers had more positive attitudes towards peer assessment in general; male in‐service teachers also liked the online approach more than did their female counterparts. Item‐by‐item analyses have been conducted in order to explore both the differences in attitude between pre‐service and in‐service teachers and to identify any gender effects.