ObjectiveThe study aimed to assess a novel web-based surgical training approach among young vascular surgeons and validate web-based evaluation methods for assessing anastomosis skills through the online Distal Bypass Competition (DBC). MethodsThe OSATS was used to assess anastomosis proficiency using a simulator among 42 participants (17 junior and 25 senior surgeons) in the DBC 2021. An initial evaluation of basic anastomosis skills was conducted via recorded videos, comparing OSATS scores with instructor-derived impression-based scores (Comprehensive Evaluation Score [CES]) during the nomination round. Real-time evaluations conducted over Zoom focused on assessing technical and composition scores of the anastomoses to determine finalists. The ice-skating method addresses potential variability among judges by excluding the highest and lowest scores for each procedural element from the final evaluation. This method, demonstrated in real time, provides trainees with valuable feedback on which movements are highly rated, enhancing the educational impact. Evaluation criteria included precision in needlework, external appearance, handling of the intima, and internal morphology of the anastomosis. Furthermore, voluntary survey questionnaires were administered to gather participant feedback on their experience with DBC 2021. ResultsThe OSATS was associated with a lower coefficient of variation (CV) than was the CES (0.19 ± 0.07 vs 0.24 ± 0.09) (P = .01) with regard to the evaluation of basic anastomotic skills in the nomination round. Interestingly, there was no significant difference in CV between the Procedural Element Score based on OSATS and CES in the senior class (0.16 ± 0.06 vs 0.22 ± 0.09) (P = .13), but a significant difference was particularly evident in the junior class (0.12 ± 0.07 vs 0.27 ± 09) (P = .03). Lower scores were associated with higher CVs in both classes, but the OSATS improved the CV among participants who had inadequate anastomosis skills. In real-time evaluation, composition scores exhibited higher CVs compared with technical scores, even among high-scoring participants. Notably, the internal morphology of the anastomosis had the largest CV in the composition scores. The ice-skating method decreased the CV as compared with the original data evaluated using the OSATS (0.15 ± 0.04 vs 0.09 ± 0.02 for the technical score [P < .01] and .18 ± 1.00 vs 0.07 ± 0.05 for the composition score [P = .03]); accordingly, the ice-skating method was more conducive to objective, real-time evaluation. The results of the post-DBC survey indicated that 89% of participants reported increased training time, and one-half of these participants indicated that their confidence in their clinical skills related to this procedure increased owing to the process of training for the DBC. ConclusionsThe OSATS yielded more detailed evaluations of anastomosis skill with a lower variability compared with the instructor's impression-based evaluation in this web-based training. Real-time evaluation methods, such as the ice-skating method, have proven to be valuable tools for the standardized assessment of web-based surgical education.
Read full abstract