Global Evaluative Assessment Of Robotic Skills Research Articles

Validation of a Simulation Model for Robotic Myomectomy

Study ObjectiveSeveral simulation models have been evaluated for gynecologic procedures such as hysterectomy, but there are limited published data for myomectomy. This study aimed to assess the validity of a low-cost robotic myomectomy model for surgical simulation training. DesignProspective cohort simulation study. SettingSurgical simulation lab. ParticipantsTwelve obstetrics and gynecology residents and four fellowship-trained minimally invasive gynecologic surgeons were recruited for a 3:1 novice-to-expert ratio. InterventionsA robotic myomectomy simulation model was constructed using <$5 worth of materials: a foam cylinder, felt, a stress ball, bandage wrap, and multipurpose sealing wrap. Participants performed a simulation task involving two steps: fibroid enucleation and hysterotomy repair. Video recorded performances were timed and scored by two blinded reviewers using the validated Global Evaluative Assessment of Robotic Skills (GEARS) scale (5-25 points) and a modified GEARS scale (5-40 points), which adds three novel domains specific to robotic myomectomy. Performance was also scored using predefined task errors. Participants completed a post-task questionnaire assessing the model's realism and utility. Measurements and Main ResultsMedian task completion time was shorter for experts compared to novices (9.7 vs. 24.6 minutes, P=.001). Experts scored higher than novices on both the GEARS scale (median 23 vs. 12, P=.004) and modified GEARS scale (36 vs. 20, P=.004). Experts made fewer task errors than novices (median 15.5 vs. 37.5, P=.034). For interrater reliability of scoring, the intraclass correlation coefficient was calculated to be 0.91 for the GEARS assessment, 0.93 for the modified GEARS assessment, and 0.60 for task errors. Using the contrasting groups method, the passing mark for the simulation task was set to a minimum modified GEARS score of 28 and a maximum of 28 errors. Most participants agreed that the model was realistic (62.5%) and useful for training (93.8%). ConclusionWe have demonstrated evidence supporting the validity of a low-cost robotic myomectomy model. This simulation model and the performance assessments developed in this study provide further educational tools for robotic myomectomy training.

IDEAL-D Phase 0 Evaluation of the Avatera System in Robot-Assisted Prostate, Bladder and Renal Surgery.

Purpose: To evaluate the utilization of novel Avatera system in urological operations according to the IDEAL-D framework recommendations for high-risk invasive surgical devices. Materials and Methods: Three surgeons attempted to perform 23 upper and lower urinary tract operations on human cadavers and in live porcine models using the Avatera system. Total operative time and the duration of the substeps were evaluated. Surgical performance was assessed with the Global Evaluative Assessment of Robotic Skills (GEARS) score. Suturing was rated using the technical checklist for the assessment of suturing in robotic surgery. Attending surgeons rated their satisfaction with the Avatera system on a scale of 1-5. Results and Limitation: Seventeen out of 18 operations performed on cadavers were completed, while one pyeloplasty was discontinued. All five operations performed in porcine models were completed. Although 1 pig was euthanized on the fifth postoperative day, its symptoms were unrelated to surgery. Mean GEARS and Suturing scores in the upper urinary tract were 29 ± 0.7 and 29.5 ± 0.95, respectively, and in the lower urinary 28.5 ± 1.2 and 29.5 ± 0.5, respectively. Surgeons' satisfaction was high or very high for all procedures. Conclusions: The Avatera system was associated with good surgical performance and high surgeons' satisfaction rates. All urological procedures performed were shown to be feasible, with comparable risks to other robot-assisted surgery systems.

The application of objective clinical human reliability analysis (OCHRA) in the assessment of basic robotic surgical skills.

Using a validated, objective, and standardised assessment tool to assess progression and competency is essential for basic robotic surgical training programmes. Objective clinical human reliability analysis (OCHRA) is an error-based assessment tool that provides in-depth analysis of individual technical errors. We conducted a feasibility study to assess the concurrent validity and reliability of OCHRA when applied to basic, generic robotic technical skills assessment. Selected basic robotic surgical skill tasks, in virtual reality (VR) and dry lab equivalent, were performed by novice robotic surgeons during an intensive 5-day robotic surgical skills course on da Vinci® X and Xi surgical systems. For each task, we described a hierarchical task analysis. Our developed robotic surgical-specific OCHRA methodology was applied to error events in recorded videos with a standardised definition. Statistical analysis to assess concurrent validity with existing tools and inter-rater reliability were performed. OCHRA methodology was applied to 272 basic robotic surgical skills tasks performed by 20 novice robotic surgeons. Performance scores improved from the start of the course to the end using all three assessment tools; Global Evaluative Assessment of Robotic Skills (GEARS) [VR: t(19) = - 9.33, p < 0.001] [dry lab: t(19) = - 10.17, p < 0.001], OCHRA [VR: t(19) = 6.33, p < 0.001] [dry lab: t(19) = 10.69, p < 0.001] and automated VR [VR: t(19) = - 8.26, p < 0.001]. Correlation analysis, for OCHRA compared to GEARS and automated VR scores, shows a significant and strong inverse correlation in every VR and dry lab task; OCHRA vs GEARS [VR: mean r = - 0.78, p < 0.001] [dry lab: mean r = - 0.82, p < 0.001] and OCHRA vs automated VR [VR: mean r = - 0.77, p < 0.001]. There is very strong and significant inter-rater reliability between two independent reviewers (r = 0.926, p < 0.001). OCHRA methodology provides a detailed error analysis tool in basic robotic surgical skills with high reliability and concurrent validity with existing tools. OCHRA requires further evaluation in more advanced robotic surgical procedures.

Surgical skill level classification model development using EEG and eye-gaze data and machine learning algorithms

The aim of this study was to develop machine learning classification models using electroencephalogram (EEG) and eye-gaze features to predict the level of surgical expertise in robot-assisted surgery (RAS). EEG and eye-gaze data were recorded from 11 participants who performed cystectomy, hysterectomy, and nephrectomy using the da Vinci robot. Skill level was evaluated by an expert RAS surgeon using the modified Global Evaluative Assessment of Robotic Skills (GEARS) tool, and data from three subtasks were extracted to classify skill levels using three classification models—multinomial logistic regression (MLR), random forest (RF), and gradient boosting (GB). The GB algorithm was used with a combination of EEG and eye-gaze data to classify skill levels, and differences between the models were tested using two-sample t tests. The GB model using EEG features showed the best performance for blunt dissection (83% accuracy), retraction (85% accuracy), and burn dissection (81% accuracy). The combination of EEG and eye-gaze features using the GB algorithm improved the accuracy of skill level classification to 88% for blunt dissection, 93% for retraction, and 86% for burn dissection. The implementation of objective skill classification models in clinical settings may enhance the RAS surgical training process by providing objective feedback about performance to surgeons and their teachers.

Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial.

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments. Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

Validation of a multidisciplinary virtual reality (VR) robotic surgical curriculum.

The objective is to identify whether trainees demonstrate improvement in a standardized knot-tying task as assessed by Global Evaluative Assessment of Robotic Skills (GEARS) score after completion of a virtual reality (VR) robotic curriculum. An IRB-exempt prospective study was conducted with surgical trainees from August 2021 to February 2023. Participants initially performed a baseline robotic suturing task in which they were instructed to tie interrupted square knots in 10min. Participants then completed a virtual reality simulation curriculum involving 23 exercises until they achieved 90% proficiency on all tasks. Participants then repeated the suturing task. Pre- and post-curriculum suturing tasks were recorded, de-identified, and scored by expert graders using a GEARS score. Trainees from three academic centers were invited to participate. Medical students (MS1-MS3) and surgical residents from gynecology, urology, and general surgery were invited to participate. Twenty-five trainees completed the pre-curriculum suturing task, the VR curriculum, and the post-curriculum suturing task. Trainees demonstrated significant improvement in their post-test GEARS score by 2.43 points (p < 0.05) and were able to tie three additional knots within 10min after completion of the curriculum (p < 0.05). Trainees also demonstrated a faster time to complete first knot (114s improvement, p < 0.05) after completion of the curriculum. All participants agreed or strongly agreed that completion of the robotic curriculum helped them feel more comfortable using the robotic console, and improved their robotic surgical skills. Surgical trainees and medical students with limited prior robotic surgical experience demonstrated objective improvement after completion of a standardized VR curriculum.

Developing surgical skill level classification model using visual metrics and a gradient boosting algorithm.

Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient boosting classification model (GBM) to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted surgery (RAS) using visual metrics. Eye gaze data were recorded from 11 participants performing four subtasks; blunt dissection, retraction, cold dissection, and hot dissection using live pigs and the da Vinci robot. Eye gaze data were used to extract the visual metrics. One expert RAS surgeon evaluated each participant's performance and expertise level using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The extracted visual metrics were used to classify surgical skill levels and to evaluate individual GEARS metrics. Analysis of Variance (ANOVA) was used to test the differences for each feature across skill levels. Classification accuracies for blunt dissection, retraction, cold dissection, and burn dissection were 95%, 96%, 96%, and 96%, respectively. The time to complete only the retraction was significantly different among the 3 skill levels (p-value = 0.04). Performance was significantly different for 3 categories of surgical skill level for all subtasks (p-values<0.01). The extracted visual metrics were strongly associated with GEARS metrics (R2>0.7 for GEARS metrics evaluation models). Machine learning (ML) algorithms trained by visual metrics of RAS surgeons can classify surgical skill levels and evaluate GEARS measures. The time to complete a surgical subtask may not be considered a stand-alone factor for skill level assessment.

Intentional enterotomies: validation of a novel robotic surgery training exercise.

While laparoscopic simulation-based training is a well-established component of general surgery training, no such requirement or standardized curriculum exists for robotic surgery. Furthermore, there is a lack of high-fidelity electrocautery simulation training exercises in the literature. Using Messick's validity framework, we sought to determine the content, response process, internal content and construct validity of a novel inanimate tissue model that utilizes electrocautery for potential incorporation in such curricula. A multi-institutional, prospective study involving medical students (MS) and general surgery residents (PGY1-3) was conducted. Participants performed an exercise using a biotissue bowel model on the da Vinci Xi robotic console during which they created an enterotomy using electrocautery, followed by approximation with interrupted sutures. Participant performance was recorded and then scored by crowd-sourced assessors of technical skill, along with three of the authors. Construct validity was determined via difference in Global Evaluative Assessment of Robotic Skills (GEARS) score, time to completion, and total number of errors between the two cohorts. Upon completion of the exercise, participants were surveyed on their perception of the exercise and its impact on their robotic training to determine content validity. 31 participants were enrolled and separated into two cohorts: MS + PGY1 vs. PGY2-3. Time spent on the robotic trainer (0.8 vs. 8.13h, p = 0.002), number of bedside robotic assists (5.7 vs. 14.8, p < 0.001), and number of robotic cases as primary surgeon (0.3 vs. 13.1, p < 0.001) were statistically significant between the two groups. Differences in GEARS scores (18.5 vs. 19.9, p = 0.001), time to completion (26.1 vs. 14.4min, p < 0.001), and total errors (21.5 vs. 11.9, p = 0.018) between the groups were statistically significant as well. Of the 23 participants that completed the post-exercise survey, 87% and 91.3% reported improvement in robotic surgical ability and confidence, respectively. On a 10-point Likert scale, respondents rated the realism of the exercise 7.5, educational benefit 9.1, and effectiveness in teaching robotic skills 8.7. Controlling for the upfront investment of certain training materials, each exercise iteration cost ~ $30. This study confirmed the content, response process, internal structure and construct validity of a novel, high-fidelity and cost-effective inanimate tissue exercise which successfully incorporates electrocautery. Consideration should be given to its addition to robotic surgery training programs.

MP26-09 EYE MOVEMENT BEHAVIOR ASSOCIATES WITH EXPERTISE LEVEL IN ROBOT-ASSISTED SURGERY

You have accessJournal of UrologyCME1 Apr 2023MP26-09 EYE MOVEMENT BEHAVIOR ASSOCIATES WITH EXPERTISE LEVEL IN ROBOT-ASSISTED SURGERY Mehdi Seilanian Toussi, Saeed Shadpour, Kristopher Attwood, Qian Liu, Camille Gutierrez, james L. Mohler, and Somayeh B. Shafiei Mehdi Seilanian ToussiMehdi Seilanian Toussi More articles by this author , Saeed ShadpourSaeed Shadpour More articles by this author , Kristopher AttwoodKristopher Attwood More articles by this author , Qian LiuQian Liu More articles by this author , Camille GutierrezCamille Gutierrez More articles by this author , james L. Mohlerjames L. Mohler More articles by this author , and Somayeh B. ShafieiSomayeh B. Shafiei More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003254.09AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: An objective method for assessing robot-assisted surgery (RAS) skills that can be used in clinical settings has not been established. This study aimed to compare visual metrics at three different skill levels—inexperienced, competent, or experienced— to determine whether eye gaze data differed among RAS surgeons. METHODS: Eleven right-handed participants who performed coagulation subtasks (monopolar or bipolar cautery) during hysterectomy, cystectomy, or nephrectomy on live pigs with the Da Vinci robot had their eye gaze data captured using TobiiPro2 eyeglasses. Visual metrics were extracted using eye gaze data (Table 1). An expert RAS surgeon evaluated each participant's performance and level of expertise after watching operation videos using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. Each metric was compared across the three skill levels—novice, competent, or experienced—using ANOVA. RESULTS: When participants performed coagulation using the dominant hand, the entropy of the right eye’s pupil diameter differed among skill levels (p-value = 0.01). When this subtask was completed with the non-dominant hand, the rates of change of the gaze directions of the left eye in the horizontal and vertical dimensions and the right eye in the vertical dimension differed (p-values = 2x10-6, 6x10-6, and 6x10-4, respectively) across skill levels. Table 2 shows the significance level for the difference in visual metrics between skill level pairs. CONCLUSIONS: Visual metrics differed significantly across RAS skill levels. The ease and efficacy of recording and evaluating visual data without interfering with the surgeon's work suggest a new objective surgical skill evaluation method that could be applied to clinical settings. Source of Funding: National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award number R01EB029398, the Alliance Foundation of Roswell Park Comprehensive Cancer Center, and the National Cancer Institute (NCI) grant P30CA016056 involving the use of Roswell Park Comprehensive Cancer Center’s Laboratory Animal Shared Resource and Applied Technology Laboratory for Advanced Surgery © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e355 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Mehdi Seilanian Toussi More articles by this author Saeed Shadpour More articles by this author Kristopher Attwood More articles by this author Qian Liu More articles by this author Camille Gutierrez More articles by this author james L. Mohler More articles by this author Somayeh B. Shafiei More articles by this author Expand All Advertisement PDF downloadLoading ...

The Surgical Learning Curve: Does Robotic Technical Skill Explain Differences in Operative Performance?

Background: Prior studies on technical skills use small collections of videos for assessment. However, there is likely heterogeneity of performance among surgeons and likely improvement after training. If technical skill explains these differences, then it should vary among practicing surgeons and improve over time. Materials and Methods: Sleeve gastrectomy cases (n = 162) between July 2018 and January 2021 at one health system were included. Global evaluative assessment of robotic skills (GEARS) scores were assigned by crowdsourced evaluators. Videos were manually annotated. Analysis of variance was used to compare continuous variables between surgeons. Tamhane's post hoc test was used to define differences between surgeons with the eta-squared value for effect size. Linear regression was used for temporal changes. A P value <.05 was considered significant. Results: Variations in operative time discriminated between individuals (e.g., between 2 surgeons, means were 91 and 112 minutes, Tamhane's = 0.001). Overall, GEARS scores did not vary significantly (e.g., between those 2 surgeons, means were 20.32 and 20.6, Tamhane's = 0.151). Operative time and total GEARS score did not change over time (R2 = 0.0001-0.096). Subcomponent scores showed idiosyncratic temporal changes, although force sensitivity increased among all (R2 = 0.172-0.243). For a novice surgeon, phase-adjusted operative time (R2 = 0.24), but not overall GEARS scores (R2 = 0.04), improved over time. Conclusions: GEARS scores showed less variability and did not improve with time for a novice surgeon. Improved technical skill does not explain the learning curve of a novice surgeon or variation among surgeons. More work could define valid surrogate metrics for performance analysis.

Value of the Crowd-Sourced Assessment of Technical Skills (C-SATS) Platform in Surgical Procedures: A Systematic Review of Evidence

Background: Crowd-Sourced Assessment of Technical Skills (C-SATS) is a surgical data management and learning platform that leverages the knowledge of large expert surgeon and lay groups to assess the technique and technical skills of surgeons in a highly efficient manner. The aim of this systematic review was to summarize published literature on the performance of C-SATS as compared to expert evaluations and assess its use as a training and validation tool in minimally invasive surgery (MIS). Methods: A comprehensive literature search was performed per PRISMA guidelines using the Medline, Embase, and Google Scholar databases on published studies that evaluated the use of C-SATS following MIS, such as laparoscopic or robotic-assisted surgery. Results: A total of 21 reports were included in the review. Twelve studies comparing crowd-sourcing evaluations against expert opinion indicated overall excellent or good correlation with Global Operative Assessment of Laparoscopic Skills (GOALS), Global Evaluative Assessment of Robotic Skills (GEARS), and Robotic Objective Structured Assessment of Technical Skills (R-OSATS) scores, with correlation coefficients (Pearson or Spearman) ranging from 0.69 to 0.95 and reliability index (Cronbach’s alpha) from 0.63 to 0.93 across different specialties and surgical approaches. When using C-SATS to assess performance and validation, assessments positively correlated with traditional methods of time and error-based scoring and global rating scale. Conclusions: Based on the current published literature, the C-SATS platform has been shown to efficiently provide crowd-sourced evaluations that correlate favorably with expert evaluation across a range of surgical specialties and approaches. Use of crowdsourcing has uniformly yielded accurate evaluations of surgeons’ technical skills in a markedly shorter time than expert reviews. C-SATS may be a cost-effective complement or alternative to traditional models of evaluating surgical proficiency. Future studies are needed to determine whether the use of C-SATS will lead to improved surgical performance and patient outcomes.

Parallel, component training in robotic total mesorectal excision.

There has been widespread adoption of robotic total mesorectal excision (TME) for rectal cancer in recent years. There is now increasing interest in training robotic novice surgeons in robotic TME surgery using the principles of component-based learning. The aims of our study were to assess the feasibility of delivering a structured, parallel, component-based, training curriculum to surgical trainees and fellows. A prospective pilot study was undertaken between January 2021 and May 2021. A dedicated robotic training pathway was designed with two trainees trained in parallel per each robotic case based on prior experience, training grade and skill set. Component parts of each operation were allocated by the robotic trainer prior to the start of each case. Robotic proficiency was assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) and the EARCS Global Assessment Score (GAS). Three trainees participated in this pilot study, performing a combined number of 52 TME resections. Key components of all 52 TME operations were performed by the trainees. GEARS scores improved throughout the study, with a mean overall baseline score of 17.3 (95% CI 15.1-1.4) compared to an overall final assessment mean score of 23.8 (95% CI 21.6-25.9), p = 0.003. The GAS component improved incrementally for all trainees at each candidate assessment (p < 0.001). Employing a parallel, component-based approach to training in robotic TME surgery is safe and feasible and can be used to train multiple trainees of differing grades simultaneously, whilst maintaining high-quality clinical outcomes.

Does Surgeon Experience Correlate with Crowd-Sourced Skill Assessment in Robotic Bariatric Surgery?

The Global Evaluative Assessment of Robotic Skills (GEARS) rubric provides a measure of skill in robotic surgery. We hypothesize surgery performed by more experienced operators will be associated with higher GEARS scores. Patients undergoing sleeve gastrectomy from 2016 to 2020 were analyzed. Three groups were defined by time in practice: less than 5, between 5 and 15, and more than 15years. Continuous variables were compared with ANOVA and multivariable regression was performed. Fourteen operators performing 154 cases were included. More experienced surgeons had higher GEARS scores and shorter operative times. On multivariable regression, operative time (P = 0.027), efficiency (P = .022), depth perception (P = 0.033), and bimanual dexterity (P = 0.047) were associated with experience. In our video-based assessment (VBA) model, operative time and several GEARS subcomponent scores were associated with surgical experience. Further studies should determine the association between these metrics and surgical outcomes.

Remote Surgical Training (ReST): The Road to a Fully-fledged Remote Robotic Surgery

Introduction: Remote control of surgical robots represents a valuable opportunity for computer-assisted surgery. Since signal latency is still not yet compatible with safe clinical practice, we explored a field of application that is free from patient-safety issues, namely training. Our research aims to demonstrate the feasibility and effectiveness of Remote Surgical Training (ReST) in the cadaver lab or on alternative models. Methods: ReST tests were carried out over a short distance, using the Xi da Vinci dual console and a cable connection. A trainee performed surgical tasks assisted by a trainer, and a tutor was dislocated remotely in an adjacent OR. ReST tests were compared with control events with tutors and trainees in the same robotic OR. Trainee’s performance was evaluated by supervisors using the Global Evaluative Assessment of Robotic Skills (GEARS) score. At the end of each event, all subjects filled in an online Likert scale-based questionnaire to evaluate the educational process. Results: Fifteen ReST tests were successfully completed and compared with fifteen control events. Statistical analysis of questionnaire data showed no significant differences in the educational process between the ReST group and the control group. At GEARS assessment, trainees achieved similar mean performance scores in both groups. Conclusion: Remote surgical training appears to be feasible and effective. Our results support further investigation on more advanced forms of remote training, including remotely displacing the trainee and testing by cable or wireless network connection over a long distance. This research is the first step towards remote robotic surgery.

The transferability of laparoscopic and open surgical skills to robotic surgery

BackgroundWithin the last decades, robotic surgery has gained popularity. Most robotic surgeons have changed their main surgical activity from open or laparoscopic without prior formal robotic training. With the current practice, it is of great interest to know whether there is a transfer of surgical skills. In visualization, motion scaling, and freedom of motion, robotic surgery resembles open surgery far more than laparoscopic surgery. Therefore, our hypothesis is that open-trained surgeons have more transfer of surgical skills to robotic surgery, compared to surgeons trained in laparoscopy.MethodsThirty-six surgically inexperienced medical students were randomized into three groups for intensive simulation training in an assigned modality: open surgery, laparoscopy, or robot-assisted laparoscopy. The training period was, for all study subjects, followed by performing a robot-assisted bowel anastomosis in a pig model. As surrogate markers of surgical quality, the anastomoses were tested for resistance to pressure, and video recordings of the procedure were evaluated by two blinded expert robotic surgeons, using a global rating scale of robotic operative performance (Global Evaluative Assessment of Robotic Skills (GEARS)).ResultsThe mean leak pressure of bowel anastomosis was 36.25 (7.62–64.89) mmHg in the laparoscopic training group and 69.01 (28.02–109.99) mmHg in the open surgery group, and the mean leak pressure for the robotic training group was 108.45 (74.96–141.94) mmHg. The same pattern was found with GEARS as surrogate markers of surgical quality. GEARS score was 15.71 (12.37–19.04) in the laparoscopic training group, 18.14 (14.70–21.58) in the open surgery group, and 22.04 (19.29–24.79) in the robotic training group. In comparison with the laparoscopic training group, the robotic training group had a statistically higher leak pressure (p = 0.0015) and GEARS score (p = 0.0023). No significant difference, for neither leak pressure nor GEARS, between the open and the robotic training group.ConclusionIn our study, training in open surgery was superior to training in laparoscopy when transitioning to robotic surgery in a simulation setting performed by surgically naive study subjects.

Tele-assessment of bandwidth limitation for remote robotics surgery

PurposeWe investigated the communication bandwidth (CB) limitation for remote robotics surgery (RRS) using hinotori™ (Medicaroid, Kobe, Japan).MethodsThe operating rooms of the Hokkaido University Hospital and Kyushu University Hospital were connected using the Science Information NETwork (SINET). The minimum required CB for the RRS was verified by decreasing the CB from 500 to 100 Mbps. Ten surgeons were tested on a task (intracorporeal suturing) at different levels of video compression (VC) (VC1: 120 Mbps, VC2: 40 Mbps, VC3: 20 Mbps) with the minimum required CB, and assessed based on the task completion time, Global Evaluative Assessment of Robotic Skills (GEARS), and System and Piper Fatigue Scale-12 (PFS-12).ResultsPacket loss was observed at 3–7% and image degradation was observed at 145 Mbps CB. The task performance with VC1 was significantly worse than that with VC2 and VC3 according to the task completion time (VC1 vs VC2, P = 0.032; VC1 vs. VC3, P = 0.032), GEARS (VC1 vs VC2; P = 0.029, VC1 vs VC3; P = 0.031), and PFS-12 (VC1 vs. VC2; P = 0.032, VC1 vs. VC3; P = 0.032) with 145 Mbps.ConclusionOur findings provide evidence that RRS using hinotori™ requires a CB ≥ 150 Mbps. We also found that when there is insufficient CB, RRS can be continued by compressing the image.

Intraoperative robotic-assisted low anterior rectal resection performance assessment using procedure-specific binary metrics and a global rating scale.

BackgroundThis study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure.MethodA prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated.ResultsTwenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 – Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019–0.008; GEARS scores, P = 0.029–0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15–41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58–64 per cent).ConclusionsPBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.

MP10-07 THE RELATIONSHIP OF TECHNICAL SKILLS AND COGNITIVE WORKLOAD TO ERRORS DURING ROBOTIC SURGICAL EXERCISES

You have accessJournal of UrologyCME1 May 2022MP10-07 THE RELATIONSHIP OF TECHNICAL SKILLS AND COGNITIVE WORKLOAD TO ERRORS DURING ROBOTIC SURGICAL EXERCISES Sidney I. Roberts, Steven Cen, Jessica H. Nguyen, Laura C. Perez, Luis G. Medina, Runzhuo Ma, Sandra P. Marshall, Rafal Kocielnik, Anima Anandkumar, and Andrew J. Hung Sidney I. RobertsSidney I. Roberts More articles by this author , Steven CenSteven Cen More articles by this author , Jessica H. NguyenJessica H. Nguyen More articles by this author , Laura C. PerezLaura C. Perez More articles by this author , Luis G. MedinaLuis G. Medina More articles by this author , Runzhuo MaRunzhuo Ma More articles by this author , Sandra P. MarshallSandra P. Marshall More articles by this author , Rafal KocielnikRafal Kocielnik More articles by this author , Anima AnandkumarAnima Anandkumar More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002532.07AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Surgical performance and skill in robotic surgery has been shown to impact patient outcomes and complication rates. Cognitive workload, or mental strain in the working memory, has been shown to differ between surgeon experience levels. Herein, we attempt to understand the relationship between surgeon technical skills, cognitive workload, and discrete errors committed during a simulated robotic dissection task. METHODS: Participant surgeons performed a robotic surgery dissection exercise (peeling a clementine, removing a single wedge). Participants were grouped based on surgical experience: novice (no prior surgical experience), intermediate (<100 robotic cases), and expert (≥100 cases). Technical skills were evaluated utilizing the validated Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The task was also evaluated for errors during active dissection or passive retraction. Cognitive workload was quantified as an Index of Cognitive Activity (ICA), derived from Task-Evoked-Pupillary-Response metrics; ICA ranged 0-1, with 1 representing maximum ICA. Generalized Estimating Equation (GEE) was used for all modelings to establish relationships between surgeon technical skills, cognitive workload and errors. RESULTS: Overall there were 22 patients: 7 novices, 9 intermediates, and 6 experts. We found a strong association between technical skills, as measured by multiple GEARS domains (depth perception, force sensitivity and robotic control), and passive errors - with higher GEARS scores associated with a lower relative risk of errors (all p <0.01). For novice surgeons, as average GEARS scores increased, the average estimated ICA decreased. In contrast, as average GEARS increased for expert surgeons, the average estimated ICA increased. When exhibiting optimal technical skill (maximal GEARS scores) novices and experts had a similar range of ICA scores (ICA 0.47 and 0.42, respectively) (Figure 1). CONCLUSIONS: This study found that there is an optimal cognitive workload level for surgeons of all experience levels during our robotic surgical exercise. Select technical skills were strong predictors of errors. Future research will explore whether an ideal cognitive workload range truly optimizes surgical performance and reduce surgical errors. Source of Funding: This study was supported in part by the National Institute Of Biomedical Imaging And Bioengineering of the National Institutes of Health under Award Number K23EB026493 © 2022 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 207Issue Supplement 5May 2022Page: e148 Advertisement Copyright & Permissions© 2022 by American Urological Association Education and Research, Inc.MetricsAuthor Information Sidney I. Roberts More articles by this author Steven Cen More articles by this author Jessica H. Nguyen More articles by this author Laura C. Perez More articles by this author Luis G. Medina More articles by this author Runzhuo Ma More articles by this author Sandra P. Marshall More articles by this author Rafal Kocielnik More articles by this author Anima Anandkumar More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement PDF DownloadLoading ...

Surgical Science–Simbionix Robotic Hysterectomy Simulator: Validating a New Tool

Study ObjectiveTo gather validity evidence for and determine acceptability of Surgical Science–Simbionix Hysterectomy Modules for the DaVinci Xi console simulation system (software; 3D Systems by Simbionix [now Surgical Science–Simbionix], Littleton, CO, and hardware; Intuitive Surgical, Inc., Sunnyvale, CA) and evaluate performance benchmarks between novice and experienced or expert surgeons. DesignProspective education study (Messick validity framework). SettingMulticenter, academic medical institutions. ParticipantsResidents, fellows, and faculty in obstetrics and gynecology were invited to participate at 3 institutions. Participants were categorized by experience level: fewer than 10 hysterectomies (novice), 10 to 50 hysterectomies (experienced), and more than 50 hysterectomies (expert). A total of 10 novice, 10 experienced, and 14 expert surgeons were included. InterventionsParticipants completed 4 simulator modules (ureter identification, bladder flap development, colpotomy, complete hysterectomy) and a qualitative survey. Simulator recordings were reviewed in duplicate by educators in minimally invasive gynecologic surgery using the Modified Global Evaluative Assessment of Robotic Skills (GEARS) rating scale. Measurements and Main ResultsMost participants felt that the simulator realistically simulated robotic hysterectomy (64.7%) and that feedback provided by the simulator was as or more helpful than feedback from previous simulators (88.2%) but less helpful than feedback provided in the operating room (73.5%). Participants felt that this simulator would be helpful for teaching junior residents. Simulator-generated metrics correlated with GEARS performance for the bladder flap and ureter identification modules in multiple domains including total movements and total time for completion. GEARS performance for the bladder flap module correlated with experience level (novice vs experienced/expert) in the domains of interest and total score but did not consistently correlate for the other procedural modules. Performance benchmarks were evaluated for the bladder flap module for each GEARS domain and total score. ConclusionThe modules were well received by participants of all experience levels. Individual simulation modules appear to better discriminate between novice and experienced/expert users than overall simulator performance. Based on these data and participant feedback, the use of individual modules in early residency education may be helpful for providing feedback and may ultimately serve as 1 component of determining readiness to perform robotic hysterectomy.

The Relationship Between Technical Skills, Cognitive Workload, and Errors During Robotic Surgical Exercises.

Purpose: We attempt to understand the relationship between surgeon technical skills, cognitive workload, and errors during a simulated robotic dissection task. Materials and Methods: Participant surgeons performed a robotic surgery dissection exercise. Participants were grouped based on surgical experience. Technical skills were evaluated utilizing the validated Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The dissection task was evaluated for errors during active dissection or passive retraction maneuvers. We quantified cognitive workload of surgeon participants as an index of cognitive activity (ICA), derived from task-evoked pupillary response metrics; ICA ranged 0 to 1, with 1 representing maximum ICA. Generalized estimating equation (GEE) was used for all modelings to establish relationships between surgeon technical skills, cognitive workload, and errors. Results: We found a strong association between technical skills as measured by multiple GEARS domains (depth perception, force sensitivity, and robotic control) and passive errors, with higher GEARS scores associated with a lower relative risk of errors (all p < 0.01). For novice surgeons, as average GEARS scores increased, the average estimated ICA decreased. In contrast, as average GEARS increased for expert surgeons, the average estimated ICA increased. When exhibiting optimal technical skill (maximal GEARS scores), novices and experts reached a similar range of ICA scores (ICA: 0.47 and 0.42, respectively). Conclusions: This study found that there is an optimal cognitive workload level for surgeons of all experience levels during our robotic surgical exercise. Select technical skill domains were strong predictors of errors. Future research will explore whether an ideal cognitive workload range truly optimizes surgical training and reduces surgical errors.