SS10. Advanced Vascular Surgery Surgical Skills and Simulated Assessment Program for Senior Vascular Surgery Trainees

Abstract

Cognitive competence in vascular surgery is determined by oral and written examinations administered by the Vascular Surgery Board. Technical competence of vascular surgery trainees is determined solely by subjective assessment from faculty members of the trainee's institution. This program represents the first attempt to objectively evaluate the advanced vascular surgical skills of senior vascular trainees using high-fidelity simulation models by an independent faculty proctor. Nineteen senior vascular trainees were evaluated over a 3-day period. Twenty-five board-certified vascular surgeons served as assessment proctors. Assessment stations included thoracic and abdominal aortic aneurysm endograft repair, open abdominal aortic aneurysm repair, carotid endarterectomy, femoral-popliteal bypass, arteriovenous fistula creation, mismatched end-to-side anastomosis at depth, carotid, renal, iliac, and superficial femoral artery angioplasty and stenting, and use of a percutaneous closure device. Trainees were given 1 hour to complete each station. Technical performances were evaluated by a single faculty member using global rating scales, critical task and time assessment metrics, and a global rating summary. Competency was defined as achieving >80% global rating scale score, 100% achievement of all critical task metrics, or attainment of a level 4 (proficient) or level 5 (advanced) global rating summary score. Immediate feedback was provided after each performance. Six simulation stations were videotaped for future assessment metrics validation. A total of 202 skill and simulation assessments were performed. Vascular trainees were graded as being proficient for 83% (global rating scale), 75% (global rating summary), and 49% (critical task metrics) of all surgical skills and simulated performances. Similar patterns of trainee technical and procedural errors were noted to occur for each simulated procedure. The performance results identified a range of technical skills among the trainees. Faculty assessment was inconsistent with reporting errors or omissions occurring in 18%. Endovascular simulator malfunction occurred in just 4% of procedures. No malfunctions occurred in the open vascular simulators. These preliminary results indicate that it is feasible to perform objective assessment of senior vascular trainees by board-certified vascular surgery faculty using multiple objective assessment metrics. Simulation models overall exhibited good durability and consistency, and therefore appear to be useful in the assessment of advanced vascular surgical skills. Data from this initial feasibility study can be used to determine reproducible objective assessment metrics. These metrics, once validated, can than serve as a foundation for a technical competence examination in vascular surgery.