Development of a bench station objective structured assessment of technical skills

Abstract

OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach’s ∝ were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.