Expert consensus on the conceptual alignment of Accreditation Council for Graduate Medical Education competencies with patient outcomes after common vascular surgical procedures.

Abstract

The quality and effectiveness of vascular surgery education should be evaluated based on patient care outcomes. To investigate predictive associations between trainee performance and subsequent patient outcomes, a critical first step is to determine the conceptual alignment of educational competencies with clinical outcomes in practice. We sought to generate expert consensus on the conceptual alignment of the Accreditation Council for Graduate Medical Education (ACGME) Vascular Surgery subcompetencies with patient care outcomes across different Vascular Quality Initiative (VQI) registries. A national panel of vascular surgeons with expertise in both clinical care and education were recruited to participate in a modified Delphi expert consensus building process to map ACGME Vascular Surgery subcompetencies (educational markers of resident performance) to VQI clinical modules (patient outcomes). A master list of items for rating was created, including the 31 ACGME Vascular Surgery subcompetencies and 8 VQI clinical registries (endovascular abdominal aortic aneurysm repair, open abdominal aortic aneurysm, thoracic endovascular aortic repair, carotid endarterectomy, carotid artery stent, infrainguinal, suprainguinal, and peripheral vascular intervention). These items were entered into an iterative Delphi process. Positive consensus was reached when 75% or more of the participants ranked an item as mandatory. Intraclass correlations (ICCs) were used to evaluate consistency between experts for each Delphi round. A total of 13 experts who contributed to the development of the Vascular Surgery Milestones participated; 12 experts (92%) participated in both rounds of the Delphi process. Two rounds of Delphi were conducted, as suggested by excellent expert agreement (round 1, ICC= 0.79 [95% confidence interval, 0.74-0.84]; round 2, ICC= 0.97 [95% confidence interval, 0.960-.98]). Using the predetermined consensus cutoff threshold, the Delphi process reduced the number of subcompetencies mapped to patient care outcomes from 31 to a range of 9 to 15 across the 8 VQI clinical registries. Practice-based learning and improvement, and professionalism subcompetencies were identified as less relevant to patient outcome variables captured by the VQI registries after the final round, and the only the systems-based practice subcompetency that was identified as relevant was radiation safety in two of the endovascular registries. A national panel of vascular surgeon experts reported a high degree of agreement on the relevance of ACGME subcompetencies to patient care outcomes as captured in the VQI clinical registry. Systems-based practice, practice-based learning and improvement, and professionalism competencies were identified as less relevant to patient outcomes after specific surgical procedures.