Benchmarks for 3-D Systems (Simbionix) Bladder Flap Module for the Xi Robot: Differentiating Novice from Experienced and Expert Surgeons

Abstract

Study Objective To develop competency benchmarks for the 3-D Systems (Simbionix) Bladder Flap Module for the Xi Robotic System. Design Prospective cohort. Setting Multi-center, academic medical institutions. Patients or Participants Residents, fellows, and faculty in OBGYN were invited to participate at 3 institutions. Participants were categorized by hysterectomy experience level: <10 (novice), 10-49 (experienced), and 50+ (expert). Ten novice and 22 experienced/expert surgeons were included. Interventions The 3-D Systems Hysterectomy Modules for the DaVinci Xi, released in 2017, include ureter dissection, bladder flap development, colpotomy, and full hysterectomy. In a prior study, recordings of participant simulator performance were graded in duplicate using the Modified Global Evaluative Assessment of Robotic Skills (GEARS) tool. The bladder flap module demonstrated correlation with experience. Competency benchmarks were developed for this module using contrasting group methods. Results were analyzed by comparing the novice participants to a combined experienced/expert group. Distributions of scores from both groups were plotted for domains of assessment where correlation had previously been demonstrated, and a cutoff score and its associated false positive (FP) and false negative (FN) rates were calculated. Measurements and Main Results Cutoff scores were developed for Depth Perception (cutoff=3.3), Bimanual Dexterity (cutoff 2.8), Efficiency (cutoff=3.25), Force Sensitivity (cutoff=3.15), and Overall Score (cutoff=1.9). Overall Score is scored from 1-3, while the other domains are on a 5-point Likert scale. Using the established cutoffs, the observed FP and FN rates for each domain were as follows: Depth Perception 0% (0/10) and 22.7% (5/22), Bimanual Dexterity 20% (2/10) and 27.3% (6/22), Efficiency 20% (2/10) and 40.9% (9/22), and Overall Score 30% (3/10) and 27.3% (6/22). Conclusion The development of target competency benchmarks is helpful to guide trainees and educators in the application of simulation tools. Using the robotic 3-D systems (Simbionix) simulator module with the strongest evidence of validity, we established benchmarks to aid in interpreting and anchoring resident simulator performance to experienced surgeons. To develop competency benchmarks for the 3-D Systems (Simbionix) Bladder Flap Module for the Xi Robotic System. Prospective cohort. Multi-center, academic medical institutions. Residents, fellows, and faculty in OBGYN were invited to participate at 3 institutions. Participants were categorized by hysterectomy experience level: <10 (novice), 10-49 (experienced), and 50+ (expert). Ten novice and 22 experienced/expert surgeons were included. The 3-D Systems Hysterectomy Modules for the DaVinci Xi, released in 2017, include ureter dissection, bladder flap development, colpotomy, and full hysterectomy. In a prior study, recordings of participant simulator performance were graded in duplicate using the Modified Global Evaluative Assessment of Robotic Skills (GEARS) tool. The bladder flap module demonstrated correlation with experience. Competency benchmarks were developed for this module using contrasting group methods. Results were analyzed by comparing the novice participants to a combined experienced/expert group. Distributions of scores from both groups were plotted for domains of assessment where correlation had previously been demonstrated, and a cutoff score and its associated false positive (FP) and false negative (FN) rates were calculated. Cutoff scores were developed for Depth Perception (cutoff=3.3), Bimanual Dexterity (cutoff 2.8), Efficiency (cutoff=3.25), Force Sensitivity (cutoff=3.15), and Overall Score (cutoff=1.9). Overall Score is scored from 1-3, while the other domains are on a 5-point Likert scale. Using the established cutoffs, the observed FP and FN rates for each domain were as follows: Depth Perception 0% (0/10) and 22.7% (5/22), Bimanual Dexterity 20% (2/10) and 27.3% (6/22), Efficiency 20% (2/10) and 40.9% (9/22), and Overall Score 30% (3/10) and 27.3% (6/22). The development of target competency benchmarks is helpful to guide trainees and educators in the application of simulation tools. Using the robotic 3-D systems (Simbionix) simulator module with the strongest evidence of validity, we established benchmarks to aid in interpreting and anchoring resident simulator performance to experienced surgeons.