Development and Validation of a Peer Evaluation Tool for Team‐Based Learning

Abstract

Background and Aims Team-based learning (TBL) is an increasingly common pedagogical approach in health science education to promote active, small group learning. Peer evaluation is an essential component of classic TBL that has been used to assess interpersonal skills, evaluate personal effort, and promote professional behavior. While a variety of approaches exist for TBL peer evaluation, no standardized instrument exists and there is variability in the use of peer evaluation for summative and formative assessments. Further, some studies have questioned the reliability and validity of current instruments and health science educators have encountered difficulty using peer evaluation for summative assessment purposes. Therefore, the purpose of this research was to develop and validate a peer evaluation tool for formative assessment in the context of a graduate-level TBL course in histology. Methods A formative peer evaluation tool (PET) was developed based on recommendations for TBL peer evaluations in the literature. The PET contained 10 items that relate to preparedness, interpersonal skills, and team engagement rated on a four-point scale from never to always. Students enrolled in a graduate histology course between 2018-2020 at Indiana University completed the PET at the middle and end of the semester, PET1 and PET2, respectively. Students completed a PET for each member of their TBL team (4-6 members) and points were awarded for completion of the PET but not for received scores. An exploratory factor analysis (EFA) with a promax rotation and Cronbach's alpha were used to assess the validity and reliability of the PET, respectively. A confirmatory factor analysis (CFA) tested the dimensionality of the tool. Pearson's correlations were used to evaluate the relationships between scores on PET1 and PET2 and average scores on TBL quizzes (i.e., iRAT [individual readiness assurance test]) at the middle and end of the semester. Results The EFA of peer evaluation scores for PET1 (n=467) revealed a three-dimensional model: (1) Accountability and preparedness for TBL activity (3 items), (2) Providing constructive feedback (3 items), and (3) Receptiveness to feedback and teamwork (4 items). Internal consistency reliability was acceptable for individual factors and overall (Factor 1 α = 0.76, Factor 2 α = 0.786, Factor 3 α = 0.624, Overall α = 0.813). A CFA performed on scores on PET2 scores (n=451) demonstrated an acceptable model fit for the three-factor model (c2 = 0.000, RMSEA = 0.08). Pearson's correlations (n=90) indicated low to moderate positive correlations between PET1 and mean iRAT scores at mid-semester (r=0.266, p=0.011) and PET2 and iRAT mean scores at the end of the semester (r=.379, p<0.001). Conclusion These results indicate that the PET is a reliable and valid tool for peer evaluation within a graduate TBL course. Further, this research demonstrates that student peers are able to provide reliable and valid feedback about students’ preparedness, engagement, and teamwork skills. Future research will focus on determining the optimum number of evaluations needed to improve the reliability of the PET using generalizability analysis.