Validation of a Rubric Used for Skills-Based Assessment of Veterinary Students Performing Simulated Ovariohysterectomy on a Model.

Abstract

Surgical skills are an important competency for new graduates. Simulators offer a means to train and assess veterinary students prior to their first surgical performance. A simulated ovariohysterectomy (OVH) rubric's validity was evaluated using a framework of content evidence, internal structure evidence, and evidence of relationship with other variables, specifically subsequent live surgical performance. Clinically experienced veterinarians (n = 13) evaluated the utility of each rubric item to collect evidence; each item's content validity index was calculated to determine its inclusion in the final rubric. After skills training, veterinary students (n = 57) were assessed using the OVH model rubric in March and August. Internal structure evidence was collected by video-recording 14 students' mock surgeries, each assessed by all five raters to calculate inter-rater reliability. Relationship with other variables evidence was collected by assessing 22 students performing their first live canine OVH in November. Experienced veterinarians included 22 items in the final rubric. The rubric generated scores with good to excellent internal consistency; inter-rater reliability was fair. Students' performance on the March model assessment was moderately correlated with their live surgical performance (ρ = 0.43) and moderately negatively correlated with their live surgical time (ρ = -0.42). Students' performance on the August model assessment, after a summer without surgical skills practice, was weakly correlated with their live surgical performance (ρ = 0.17). These data support validation of the simulated OVH rubric. The continued development of validated assessment instruments is critical as veterinary medicine seeks to become competency based.