Proficiency-based Progression Research Articles

Surgical skill simulation training to proficiency reduces procedural errors among novice cardiac device implanters: a randomized study.

In cardiac device implantation, having both surgical skills and ability to manipulate catheter/lead/wire is crucial. Few cardiologists, however, receive formal surgical training prior to implanting. Skills are mostly acquired directly on-the-job and surgical technique varies across institutions; suboptimal approaches may increase complications. We investigated how novel proficiency-based progression (PBP) simulation training impacts the surgical quality of implantations, compared to traditional simulation (SIM) training. In this international prospective study, novice implanters were randomized (blinded) 1:1 to participate in a simulation-based procedure training curriculum, with proficiency demonstration requirements for advancing (PBP approach) or without (SIM). Ultimately, trainees performed the surgical tasks of an implant on a porcine tissue that was video-recorded and then scored by two independent assessors (blinded to group), using previously validated performance metrics. Primary outcomes were the number of procedural Steps Completed, Critical Errors, Errors (non-critical), and All Errors Combined. Thirty novice implanters from 10 countries participated. Baseline experiences were similar between groups. Compared to SIM-trained, the PBP-trained group completed on average 11% more procedural Steps (P < 0.001) and made 61.2% fewer Critical Errors (P < 0.001), 57.1% fewer Errors (P = 0.140), and 60.7% fewer All Errors Combined (P = 0.001); 11/15 (73%) PBP trainees demonstrated the predefined target performance level vs. 3/15 SIM trainees (20%) in the video-recorded performance. Proficiency-based progression training produces superior objectively assessed novice operators' surgical performance in device implantation compared with traditional (simulation) training. Systematic PBP incorporation into formal academic surgical skills training is recommended before in vivo device practice. Future studies will quantify PBP training's effect on surgery-related device complications.

Development and evaluation of a societal core robotic surgery accreditation curriculum for the UK

Standardised proficiency-based progression is the cornerstone of safe robotic skills acquisition, however, is currently lacking within surgical training curricula. Expert consensuses have defined a modular pathway to accredit surgeons. This study aimed to address the lack of a formal, pre-clinical core robotic skills, proficiency-based accreditation curriculum in the UK. Novice robotic participants underwent a four-day pre-clinical core robotic skills curriculum incorporating multimodal assessment. Modifiable-Global Evaluative Assessment of Robotic Skills (M-GEARS), VR-automated performance metrics (APMs) and Objective Clinical Human Reliability Analysis (OCHRA) error methodology assessed performance at the beginning and end of training. Messick’s validity concept and a curriculum evaluation model were utilised. Feedback was collated. Proficiency-based progression, benchmarking, tool validity and reliability was assessed through comparative and correlational statistical methods. Forty-seven participants were recruited. Objective assessment of VR and dry models across M-GEARS, APMs and OCHRA demonstrated significant improvements in technical skill (p < 0.001). Concurrent validity between assessment tools demonstrated strong correlation in dry and VR tasks (r = 0.64–0.92, p < 0.001). OCHRA Inter-rater reliability was excellent (r = 0.93, p < 0.001 and 81% matched error events). A benchmark was established with M-GEARS and for the curriculum at 80%. Thirty (63.82%) participants passed. Feedback was 5/5 stars on average, with 100% recommendation. Curriculum evaluation fulfilled all five domains of Messick’s validity. Core robotic surgical skills training can be objectively evaluated and benchmarked to provide accreditation in basic robotic skills. A strategy is necessary to enrol standardised curricula into national surgical training at an early stage to ensure patient safety.

Proficiency-based progression training in robot-assisted laparoscopy for endometrial cancer: peri-operative and survival outcomes from an observational cohort study.

Over the last decade there has been a transition from traditional laparoscopy to robotic surgery for the treatment of endometrial cancer. A number of gynecological oncology surgical fellowship programmes have adopted robot-assisted laparoscopy, but the effect of training on complications and survival has not been evaluated. Our aim was to assess the impact of a proficiency-based progression training curriculum in robot-assisted laparoscopy on peri-operative and survival outcomes for endometrial cancer. This is an observational cohort study performed in a tertiary referral and subspecialty training center. Women with primary endometrial cancer treated with robot-assisted laparoscopic surgery between 2015 and 2022 were included. Surgery would normally include a hysterectomy and salpingo-oophorectomy with some form of pelvic lymph node dissection (sentinel lymph nodes or lymphadenectomy). Training was provided according to a training curriculum which involves step-wise progression of the trainee based on proficiency to perform a certain surgical technique. Training cases were identified pre-operatively by consultant surgeons based on clinical factors. Case complexity matched the experience of the trainee. Main outcome measures were intra- and post-operative complications, blood transfusions, readmissions < 30 days, return to theater rates and 5-year disease-free and disease-specific survival for training versus non-training cases. Mann-Witney U, Pearson's chi-squared, multivariable regression, Kaplan-Meier and Cox proportional hazard analyses were performed to assess the effect of proficiency-based progression training on peri-operative and survival outcomes. Training cases had a lower BMI than non-training cases (30 versus 32 kg/m2, p = 0.013), but were comparable in age, performance status and comorbidities. Training had no influence on intra- and post-operative complications, blood transfusions, readmissions < 30 days, return to theater rates and median 5-year disease-free and disease-specific survival. Operating time was longer in training cases (161 versus 137 min, p = < 0.001). The range of estimated blood loss was smaller in training cases. Conversion rates, critical care unit-admissions and lymphoedema rates were comparable. Proficiency-based progression training can be used safely to teach robot-assisted laparoscopic surgery for women with endometrial cancer. Prospective trails are needed to further investigate the influence of distinct parts of robot-assisted laparoscopic surgery performed by a trainee on endometrial cancer outcomes.

International consensus panel for transurethral resection of bladder tumours metrics: assessment of face and content validity.

To develop performance metrics that objectively define a reference approach to a transurethral resection of bladder tumours (TURBT) procedure, seek consensus on the performance metrics from a group of international experts. The characterisation of a reference approach to a TURBT procedure was performed by identifying phases and explicitly defined procedure events (i.e., steps, errors, and critical errors). An international panel of experienced urologists (i.e., Delphi panel) was then assembled to scrutinise the metrics using a modified Delphi process. Based on the panel's feedback, the proposed metrics could be edited, supplemented, or deleted. A voting process was conducted to establish the consensus level on the metrics. Consensus was defined as the panel majority (i.e., >80%) agreeing that the metric definitions were accurate and acceptable. The number of metric units before and after the Delphi meeting were presented. A core metrics group (i.e., characterisation group) deconstructed the TURBT procedure. The reference case was identified as an elective TURBT on a male patient, diagnosed after full diagnostic evaluation with three or fewer bladder tumours of ≤3 cm. The characterisation group identified six procedure phases, 60 procedure steps, 43 errors, and 40 critical errors. The metrics were presented to the Delphi panel which included 15 experts from six countries. After the Delphi, six procedure phases, 63 procedure steps, 47 errors, and 41 critical errors were identified. The Delphi panel achieved a 100% consensus. Performance metrics to characterise a reference approach to TURBT were developed and an international panel of experts reached 100% consensus on them. This consensus supports their face and content validity. The metrics can now be used for a proficiency-based progression training curriculum for TURBT.

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review.

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity. A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis. A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes. Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

Development and validation of metrics for a new RAPN training model.

Robot-assisted partial nephrectomy (RAPN) is a complex and index procedure that urologists need to learn how to perform safely. No validated performance metrics specifically developed for a RAPN training model (TM) exist. A Core Metrics Group specifically adapted human RAPN metrics to be used in a newly developed RAPN TM, explicitly defining phases, steps, errors, and critical errors. A modified Delphi meeting concurred on the face and content validation of the new metrics. One hundred percent consensus was achieved by the Delphi panel on 8 Phases, 32 Steps, 136 Errors and 64 Critical Errors. Two trained assessors evaluated recorded video performances of novice and expert RAPN surgeons executing an emulated RAPN in the newly developed TM. There were no differences in procedure Steps completed by the two groups. Experienced RAPN surgeons made 34% fewer Total Errors than the Novice group. Performance score for both groups was divided at the median score using Total Error scores, into HiError and LoError subgroups. The LowErrs Expert RAPN surgeons group made 118% fewer Total Errors than the Novice HiErrs group. Furthermore, the LowErrs Expert RAPN surgeons made 77% fewer Total Errors than the HiErrs Expert RAPN surgeons. These results established construct and discriminative validity of the metrics. The authors described a novel RAPN TM and its associated performance metrics with evidence supporting their face, content, construct, and discriminative validation. This report and evidence support the implementation of a simulation-based proficiency-based progression (PBP) training program for RAPN.

Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial.

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments. Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

PE047 - Single trainee experience of the ERUS proficiency-based progression training curriculum in an accredited ERUS Host centre

PE047 - Single trainee experience of the ERUS proficiency-based progression training curriculum in an accredited ERUS Host centre

Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters

In cardiac device implant training, there is no common system to objectively assess trainees' ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators' early learning curve. To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT). In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022. Training with PBP vs SBT. The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined. Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees. In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications. ClinicalTrials.gov Identifier: NCT05952908.

THTP3.4 Performances measures in Robotic-Assisted Surgery- A Systematic Review

Abstract Background Robotic-Assisted Surgery (RAS) has seen a global rise in the past 30 years. Despite the increased adoption, there is no current standardised training curricula or measure of performance. The aim of this systematic review was to define the RAS metrics used to assess technical performance across the surgical specialities. Methods Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published 31st January 2020 – 31st January 2022. All randomised controlled trials (RCTs) and observational studies were considered. Review articles and systematic reviews were excluded. The papers’ quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. Results The initial search yielded 1189 papers of which 74 were eligible. The majority were published in urology (45% n=33). 27 unique metrics were identified and categorised as “task based”, “procedural based”, “cognitive assessments” and “global assessments”. Global assessments were the most common category of assessment (n=12); the most frequently used was GEARs (Global Evaluative Assessment of Robotic Skills). Only 4 metrics (automated, Proficiency Based Progression (PBP), pupillary measures and time to completion) used quantitative measures to assess performance whilst the remaining relied upon Likert scales thereby creating variability amongst users. Conclusion There is wide variation in tools used to assess performance in RAS. The majority of tools are subjective which increases the risk of bias amongst users and therefore accuracy in the measure of performance. A validated, objective global assessment tool that utilises quantitative measures is required.

Analysis of training pathway to reach expert performance levels based on proficiency-based progression in robotic-assisted minimally invasive esophagectomy (RAMIE).

Robotic-assisted minimally invasive esophagectomy (RAMIE) was first introduced in 2003 and has since then shown to significantly improve the postoperative course. Previous studies have shown that a structured training pathway based on proficiency-based progression using individual skill levels as measures of reach of competence can enhance surgical performance. The aim of this study was to evaluate and help understand our pathway to reach surgical expert levels using a proficiency-based approach introducing RAMIE at our German high-volume center. All patients undergoing RAMIE performed by two experienced surgeons for esophageal cancer since the introduction of the robotic technique in 2017 was included in this analysis. Intraoperative outcomes and postoperative outcomes were included in the analysis. The cumulative sum method was used to analyze how many cases are needed to reach expert levels for different performance characteristics and skill sets during robotic-assisted minimally invasive esophagectomy. From 06/2017 to 03/2022, a total of 154 patients underwent RAMIE at our facility and were included in the analysis. An advancement in performance level was observed for total operating time after 70 cases and for thoracic operative time after 79 cases. Lymph node yield showed an increase up until case 60 in the CUSUM analysis. Length of hospital stay stabilized after case 55. The CCI score inflection point was at case 55 in both CUSUM and regression analyses. Anastomotic leak rate stabilized at case 38 and showed another inflection point after case 83. Our data and analysis showed the progression from proficient to expert performance levels during the implementation of RAMIE at a European high-volume center. Further analysis of surgeons, especially with a different training status has yet to reveal if the caseloads found in this study are universally applicable. However, skill acquisition and respective measures of such are diverse and as a great range of number of cases was observed, we believe that the learning curve and ascent in performance levels cannot be defined by one parameter alone.

Effectiveness of a proficiency-based progression e-learning approach to training in communication in the context of clinically deteriorating patients: a multi-arm randomised controlled trial

ObjectiveTo determine the effectiveness of proficiency-based progression (PBP) e-learning in training in communication concerning clinically deteriorating patients.DesignSingle-centre multi-arm randomised double-blind controlled trial with three parallel arms.Randomisation, setting and participantsA computer-generated...

MP25-15 INTERNATIONAL EXPERTS CONSENSUS ON PERFORMANCE METRICS FOR A TRANSURETHRAL RESECTION OF BLADDER

MP25-15 INTERNATIONAL EXPERTS CONSENSUS ON PERFORMANCE METRICS FOR A TRANSURETHRAL RESECTION OF BLADDER

Virtual classroom proficiency-based progression for robotic surgery training (VROBOT): a randomised, prospective, cross-over, effectiveness study

Robotic surgery training has lacked evidence-based standardisation. We aimed to determine the effectiveness of adjunctive interactive virtual classroom training (VCT) in concordance with the self-directed Fundamentals of Robotic Surgery (FRS) curriculum. The virtual classroom is comprised of a studio with multiple audio–visual inputs to which participants can connect remotely via the BARCO weConnect platform. Eleven novice surgical trainees were randomly allocated to two training groups (A and B). In week 1, both groups completed a robotic skills induction. In week 2, Group A received training with the FRS curriculum and adjunctive VCT; Group B only received access to the FRS curriculum. In week 3, the groups received the alternate intervention. The primary outcome was measured using the validated robotic-objective structured assessment of technical skills (R-OSAT) at the end of week 2 (time-point 1) and 3 (time-point 2). All participants completed the training curriculum and were included in the final analyses. At time-point 1, Group A achieved a statistically significant greater mean proficiency score compared to Group B (44.80 vs 35.33 points, p = 0.006). At time-point 2, there was no significant difference in mean proficiency score in Group A from time-point 1. In contrast, Group B, who received further adjunctive VCT showed significant improvement in mean proficiency by 9.67 points from time-point 1 (95% CI 5.18–14.15, p = 0.003). VCT is an effective, accessible training adjunct to self-directed robotic skills training. With the steep learning curve in robotic surgery training, VCT offers interactive, expert-led learning and can increase training effectiveness and accessibility.

Impact of simulation-based training in endourology: A systematic review of the literature

The impact of simulation on the acquisition of surgical skills as well as their transfer to the operating room is still debated. The objective was to assess these two specific points, focusing on the field of endourology. A systematic review of the literature, following the PRIMA statement, was performed using Medline database through September 2021 without time limit. Studies focusing on the impact of simulators on the acquisition of surgical technical skills as well as their transfer to the operating room in the field of endourology were included. The endourological procedures were: cystoscopy, ureteroscopy, percutaneous nephrolithotomy, endoscopic treatment of benign prostatic hyperplasia, endoscopic bladder resection. Among the 11,442 publications identified, fifty-two studies were included in the analysis. The majority reported an improvement in procedure time of the requested tasks and dexterity of participants, regardless of the type of simulator and procedure. The level of evidence of included studies was often low. Few studies evaluated the transfer of acquired skills from the simulator to the operating room. This review showed the positive impact of simulation on the acquisition of technical skills in endourology. However, in order to include proficiency-based progression in the curriculum of trainees, some parameters such as the choice of reference simulators, choice of tasks, and method of validation of acquired skills must be validated in a consensual manner to offer a quality training.

291 Virtual Classroom Proficiency-Based Progression for Robotic Surgery Training (VROBOT): A Prospective, Cross-Over, Effectiveness Study

Abstract Aim Robotic surgical training has historically lacked evidence-based standardisation. The Fundamentals of Robotic Surgery (FRS) curriculum addresses this by incorporating proficiency-based modules which certify trainees’ skills. However, the curriculum is self-directed and non-interactive. We aim to determine the effectiveness of interactive virtual classroom training (VCT) as an adjunct to the FRS curriculum for robotic skills training. Method 11 novice surgical trainees were randomly allocated to two groups. Both groups completed a one-week robotic skills induction. In week two, Group A received training with the FRS curriculum and adjunctive VCT; Group B only received access to the FRS curriculum. In week three, the groups received the alternate intervention. The primary outcome was measured using the validated robotic-objective structured assessment of technical skills (R-OSAT). Results 11 participants completed the training curriculum; all were included in analyses. Both groups demonstrated significantly improved proficiency upon completion of the training. At time point 1 (end of week two) Group A achieved a statistically significant greater mean proficiency score compared to Group B (44.80 vs 35.33 points, p=0.006). At time point 2 (end of week three), there was no significant difference in Group A's mean score from time point 1. In contrast, Group B showed significant improvement in mean proficiency by 9.67 points from time point 1 (95% CI 5.18–14.15, p=0.003). Conclusions VCT is an effective training adjunct to the FRS curriculum for the learning of basic robotic skills. With the steep learning curve in robotic surgery training, VCT offers interactive learning and can increase training effectiveness.

SP7.1 Virtual classroom proficiency-based progression for robotic surgery training (VROBOT): a prospective, cross-over, effectiveness study

Abstract Aims Robotic surgery is an evolving surgical technique, requiring specialist training. The standardised Fundamentals of Robotic Surgery (FRS) curriculum incorporates proficiency-based modules to certify the skills of novice trainees. However, the curriculum is self-directed and non-interactive. We aim to determine the effectiveness of interactive virtual classroom training (VCT) in concordance with the FRS for robotic skills training. Methods 11 novice surgical trainees were randomly allocated to two training groups. Both groups completed a one-week robotic skills induction. In week two, Group A received training with the FRS curriculum and adjunctive VCT; Group B only received access to the FRS curriculum. In week three, the groups received the alternate intervention. The primary outcome was measured using the validated R-OSAT score. This was collected post-intervention at time point 1(end of week two) and time point 2(end of week three). Results Participants attained higher mean proficiency scores with both the FRS curriculum and VCT, compared to the FRS curriculum alone. At timepoint 1, Group A achieved a greater mean proficiency score compared to Group B (44.80 vs 35.33, p=0.006). At timepoint 2, there was no significant difference in mean proficiency score in Group A from timepoint 1. In contrast, Group B, who received adjunctive VCT showed significant improvement in mean proficiency by 9.67points from timepoint 1 (95% CI 5.18–14.15, p=0.003). Conclusion VCT is an effective training adjunct to the FRS curriculum for learning basic robotic skills. With the steep learning curve in robotic surgery training, VCT offers interactive learning and can increase training effectiveness and accessibility.

Intraoperative robotic-assisted low anterior rectal resection performance assessment using procedure-specific binary metrics and a global rating scale.

BackgroundThis study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure.MethodA prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated.ResultsTwenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 – Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019–0.008; GEARS scores, P = 0.029–0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15–41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58–64 per cent).ConclusionsPBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.

MP41-06 A PROSPECTIVE, RANDOMIZED, MULTI-CENTRE TRIAL ON THE EFFICIENCY AND EFFECTIVENESS OF PROFICIENCY BASED PROGRESSION ROBOTIC SURGICAL SKILLS TRAINING

You have accessJournal of UrologyCME1 May 2022MP41-06 A PROSPECTIVE, RANDOMIZED, MULTI-CENTRE TRIAL ON THE EFFICIENCY AND EFFECTIVENESS OF PROFICIENCY BASED PROGRESSION ROBOTIC SURGICAL SKILLS TRAINING Stefano Puliatti, Marco Amato, Rui Farinha, Artur Paludo, Giuseppe Rosiello, Ruben De Groote, Laura Langhendries, Maxime Lassel, Andrea Mari, Lorenzo Bianchi, Pietro Piazza, Ben Van Cleynenbreugel, Elio Mazzone, Kobe Van Hove, Simon Meganck, Léonore Maertens, Louis-Philippe Boret, Filippo Migliorini, Saverio Forte, Bernardo Rocco, Salvatore Micali, Alexandre Mottrie, and Anthony Gallagher Stefano PuliattiStefano Puliatti More articles by this author , Marco AmatoMarco Amato More articles by this author , Rui FarinhaRui Farinha More articles by this author , Artur PaludoArtur Paludo More articles by this author , Giuseppe RosielloGiuseppe Rosiello More articles by this author , Ruben De GrooteRuben De Groote More articles by this author , Laura LanghendriesLaura Langhendries More articles by this author , Maxime LasselMaxime Lassel More articles by this author , Andrea MariAndrea Mari More articles by this author , Lorenzo BianchiLorenzo Bianchi More articles by this author , Pietro PiazzaPietro Piazza More articles by this author , Ben Van CleynenbreugelBen Van Cleynenbreugel More articles by this author , Elio MazzoneElio Mazzone More articles by this author , Kobe Van HoveKobe Van Hove More articles by this author , Simon MeganckSimon Meganck More articles by this author , Léonore MaertensLéonore Maertens More articles by this author , Louis-Philippe BoretLouis-Philippe Boret More articles by this author , Filippo MiglioriniFilippo Migliorini More articles by this author , Saverio ForteSaverio Forte More articles by this author , Bernardo RoccoBernardo Rocco More articles by this author , Salvatore MicaliSalvatore Micali More articles by this author , Alexandre MottrieAlexandre Mottrie More articles by this author , and Anthony GallagherAnthony Gallagher More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002607.06AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: We report the use of full Proficiency Based Progression (PBP) methodology to train robotic surgical skills in comparison to other conventional training approaches. We evaluated: 1) if all trainees reached the proficiency benchmark, 2) the number of training trials and time required. METHODS: In this study 47 participants were randomized to 4 different groups. i) Full PBP group that received eLearning on the ORSI chicken anastomosis task with the requirement to reach the proficiency benchmark before starting practical training. ii) eLearning group, received the exact same information as Full PBP group, but they were not required to reach the eLearning benchmark. iii) Traditional group received the same content in face-to-face lectures, before the practical module. iv) The Apprenticeship group received a conventional preparation (Figure 1). The task evaluated was the robotic suturing and knot tying anastomosis in chicken model. RESULTS: All of the participants, except five in group 4, demonstrated the proficiency benchmark (Figures 2A-C). Group 1 took ∼6 trials or ∼3 hours; Group 2 required 14% longer, Group 3, 103% and Group 4, 162% longer to reach proficiency (Figures 3A-C). CONCLUSIONS: The efficiency and effectiveness observed with a full application of a PBP approach to skills training could have profound implications for how training is conducted in the future. Source of Funding: No © 2022 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 207Issue Supplement 5May 2022Page: e720 Advertisement Copyright & Permissions© 2022 by American Urological Association Education and Research, Inc.MetricsAuthor Information Stefano Puliatti More articles by this author Marco Amato More articles by this author Rui Farinha More articles by this author Artur Paludo More articles by this author Giuseppe Rosiello More articles by this author Ruben De Groote More articles by this author Laura Langhendries More articles by this author Maxime Lassel More articles by this author Andrea Mari More articles by this author Lorenzo Bianchi More articles by this author Pietro Piazza More articles by this author Ben Van Cleynenbreugel More articles by this author Elio Mazzone More articles by this author Kobe Van Hove More articles by this author Simon Meganck More articles by this author Léonore Maertens More articles by this author Louis-Philippe Boret More articles by this author Filippo Migliorini More articles by this author Saverio Forte More articles by this author Bernardo Rocco More articles by this author Salvatore Micali More articles by this author Alexandre Mottrie More articles by this author Anthony Gallagher More articles by this author Expand All Advertisement PDF DownloadLoading ...

MP41-13 DOES QUALITY ASSURED ELEARNING PROVIDE ADEQUATE PREPARATION FOR ROBOTIC SURGICAL SKILLS; A PROSPECTIVE, RANDOMIZED AND MULTI-CENTER STUDY

You have accessJournal of UrologyCME1 May 2022MP41-13 DOES QUALITY ASSURED ELEARNING PROVIDE ADEQUATE PREPARATION FOR ROBOTIC SURGICAL SKILLS; A PROSPECTIVE, RANDOMIZED AND MULTI-CENTER STUDY Stefano Puliatti, Marco Amato, Rui Farinha, Artur Paludo, Giuseppe Rosiello, Ruben De Groote, Andrea Mari, Lorenzo Bianchi, Pietro Piazza, Ben Van Cleynenbreugel, Elio Mazzone, Simon Meganck, Léonore Maertens, Louise Callens, Filippo Migliorini, Saverio Forte, Bernardo Rocco, Salvatore Micali, Alexandre Mottrie, and Anthony Gallagher Stefano PuliattiStefano Puliatti More articles by this author , Marco AmatoMarco Amato More articles by this author , Rui FarinhaRui Farinha More articles by this author , Artur PaludoArtur Paludo More articles by this author , Giuseppe RosielloGiuseppe Rosiello More articles by this author , Ruben De GrooteRuben De Groote More articles by this author , Andrea MariAndrea Mari More articles by this author , Lorenzo BianchiLorenzo Bianchi More articles by this author , Pietro PiazzaPietro Piazza More articles by this author , Ben Van CleynenbreugelBen Van Cleynenbreugel More articles by this author , Elio MazzoneElio Mazzone More articles by this author , Simon MeganckSimon Meganck More articles by this author , Léonore MaertensLéonore Maertens More articles by this author , Louise CallensLouise Callens More articles by this author , Filippo MiglioriniFilippo Migliorini More articles by this author , Saverio ForteSaverio Forte More articles by this author , Bernardo RoccoBernardo Rocco More articles by this author , Salvatore MicaliSalvatore Micali More articles by this author , Alexandre MottrieAlexandre Mottrie More articles by this author , and Anthony GallagherAnthony Gallagher More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002607.13AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: In particular after the onset of the COVID-19 pandemic, there was a precipitous rush to implement virtual and online learning strategies in surgery and medicine. It is essential to understand whether this approach is sufficient and adequate to allow the development of robotic basic surgical skills. The main aim of the authors was to verify if the quality assured eLearning is sufficient to prepare individuals to perform a basic surgical robotic task. METHODS: A prospective, randomized and multi-center study conducted in September 2020 in the ORSI Academy, International surgical robotic training center. 47 participants with no experience but a special interest in robotic surgery, were matched and randomized into 4 groups who underwent a didactic preparation with different formats before carrying out a robotic suturing and anastomosis task. Didactic preparation methods, ranged from a complete eLearning path to peer-reviewed published manuscripts describing the suturing, knot tying and task assessment metrics. RESULTS: The primary outcome was the percentage of trainees who demonstrated the quantitatively defined proficiency benchmark after learning to complete an assisted but unaided robotic vesico-urethral anastomosis task. The quantitatively defined benchmark was based on the objectively assessed performance (i.e., procedure steps completed, errors and critical errors) of experienced robotic surgeons for a proficiency based progression (PBP) training course. None of the trainees in this study demonstrated the proficiency benchmarks in completing the robotic surgery task (Figure 1a-c). CONCLUSIONS: Quality assured online learning is insufficient preparation for robotic suturing and knot tying anastomosis skills. Source of Funding: No © 2022 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 207Issue Supplement 5May 2022Page: e724 Advertisement Copyright & Permissions© 2022 by American Urological Association Education and Research, Inc.MetricsAuthor Information Stefano Puliatti More articles by this author Marco Amato More articles by this author Rui Farinha More articles by this author Artur Paludo More articles by this author Giuseppe Rosiello More articles by this author Ruben De Groote More articles by this author Andrea Mari More articles by this author Lorenzo Bianchi More articles by this author Pietro Piazza More articles by this author Ben Van Cleynenbreugel More articles by this author Elio Mazzone More articles by this author Simon Meganck More articles by this author Léonore Maertens More articles by this author Louise Callens More articles by this author Filippo Migliorini More articles by this author Saverio Forte More articles by this author Bernardo Rocco More articles by this author Salvatore Micali More articles by this author Alexandre Mottrie More articles by this author Anthony Gallagher More articles by this author Expand All Advertisement PDF DownloadLoading ...