Robotic Suturing Skills Research Articles

Editorial Comment

No AccessJournal of UrologyNew Technology and Techniques1 Aug 2022Editorial CommentThis article comments on the following:Tailored Feedback Based on Clinically Relevant Performance Metrics Expedites the Acquisition of Robotic Suturing Skills—An Unblinded Pilot Randomized Controlled Trialis a letter which has replyReply by Authors Alexandra L. Tabakin, and Sammy E. Elsamra Alexandra L. TabakinAlexandra L. Tabakin Division of Urology, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey More articles by this author , and Sammy E. ElsamraSammy E. Elsamra Division of Urology, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002691.02AboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail "Editorial Comment." The Journal of Urology, 208(2), p. 423

Tailored Feedback Based on Clinically Relevant Performance Metrics Expedites the Acquisition of Robotic Suturing Skills—An Unblinded Pilot Randomized Controlled Trial

Previously, we identified 8 objective suturing performance metrics highly predictive of urinary continence recovery after robotic-assisted radical prostatectomy. Here, we aimed to test the feasibility of providing tailored feedback based upon these clinically relevant metrics and explore the impact on the acquisition of robotic suturing skills. Training surgeons were recruited and randomized to a feedback group or a control group. Both groups completed a baseline, midterm and final dry laboratory vesicourethral anastomosis (VUA) and underwent 4 intervening training sessions each, consisting of 3 suturing exercises. Eight performance metrics were recorded during each exercise: 4 automated performance metrics (derived from kinematic and system events data of the da Vinci® Robotic System) representing efficiency and console manipulation competency, and 4 suturing technical skill scores. The feedback group received tailored feedback (a visual diagram+verbal instructions+video examples) based on these metrics after each session. Generalized linear mixed model was used to compare metric improvement (Δ) from baseline to the midterm and final VUA. Twenty-three participants were randomized to the feedback group (11) or the control group (12). Demographic data and baseline VUA metrics were comparable between groups. The feedback group showed greater improvement than the control group in aggregate suturing scores at midterm (mean Δ feedback group 4.5 vs Δ control group 1.1) and final VUA (Δ feedback group 5.3 vs Δ control group 4.9). The feedback group also showed greater improvement in the majority of the included metrics at midterm and final VUA. Tailored feedback based on specific, clinically relevant performance metrics is feasible and may expedite the acquisition of robotic suturing skills.

Road to automating robotic suturing skills assessment: Battling mislabeling of the ground truth

ObjectiveTo automate surgeon skills evaluation using robotic instrument kinematic data. Additionally, to implement an unsupervised mislabeling detection algorithm to identify potentially mislabeled samples that can be removed to improve model performance. MethodsVideo recordings and instrument kinematic data were derived from suturing exercises completed on the Mimic FlexVR robotic simulator. A structured human consensus-building process was developed to determine Robotic Anastomosis Competency Evaluation technical scores across 3 human graders. A 2-layer long short-term memory–based classification model used instrument kinematic data to automate suturing skills assessment. An unsupervised label analyzer (NoiseRank) was used to identify potential mislabeling of skills data. Performance of the long short-term memory model’s technical skill score prediction was measured by best area under the curve over the training runs. NoiseRank outputted a ranked list of rated skills assessments based on likelihood of mislabeling. Results22 surgeons performed 226 suturing attempts, which were broken down into 1,404 individual skill assessment points. Automation of needle entry angle, needle driving, and needle withdrawal technical skill scores performed better (area under the curve 0.698–0.705) than needle positioning (0.532) at baseline using all available data. Potential mislabels were subsequently identified by NoiseRank and removed, improving model performance across all domains (area under the curve 0.551–0.766). ConclusionUsing ground truth labels from human graders and robotic instrument kinematic data, machine learning models have automated assessment of detailed suturing technical skills with good performance. Further, an unsupervised mislabeling detection algorithm projected mislabeled data, allowing for their removal and subsequent improvement of model performance.

Gender differences in the acquisition of suturing skills with the da Vinci surgical system

The percentage of female medical students has been significant elevating worldwide. The demographic shift is expected to influence the proportion of male versus female surgeons soon. The objective of this study was to evaluate the gender differences in the acquisition of robotic suturing skills. We compared the robotic suturing performance between 39 male and 19 female medical students. We separated the training into two parts: phase I, involving virtual reality (VR) robotic simulation, and phase II, involving robotic dry-laboratory simulation training. Participants first conducted step-by-step exercises on the VR robotic simulator and then the robotic skin-suturing pad using the da Vinci robot. The metric analysis of the VR task "suture sponge" showed that female students required less time (difference: -170.7 seconds, 95% CI: -247.4 to -94.0) and had fewer errors (error difference: -50, 95% CI: -74.2 to -25.8) to complete the suture sponge exercise compared to male students. Moreover, female students completed more stitches than male students (differences in mean stitch achieved: .35; 95% CI: .06 to .65). However, there was no difference in the quality scores of stitches by gender (p=0.85). Female medical students performed better in the VR task of suture spongy and achieved more stitches than male students with the da Vinci system despite no difference in robotic suture quality by gender. Because this is the first study comparing gender performance on a robotic platform, further studies are required to investigate if different training approaches will affect the performance by gender.

An Efficient Single-session Spatial Skill Trainer for Robot-assisted Surgery: A Randomized Trial

To introduce and examine a single session of spatial skill training as an efficient means of improving surgical suturing performance in robot-assisted surgery. A randomized, controlled trial. A tertiary university medical center in Israel. A purposive sample composed of 41 residents with no robotic suturing skills. A computer-based simulator training of spatial skills. Participants were randomly assigned to training (n = 21: mean age of 34 years [standard deviation (SD) = 1.92]) and control (n = 20: mean age of 32 years [SD = 3.17]) conditions. The training group underwent a session of spatial skills training, whereas the control group engaged in a neutral activity. After 1 participant was lost to the follow-up of the posttraining performance test, data of 40 participants were analyzed. Robotic suturing task performance with the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA) was evaluated using the da Vinci Skills Simulator built-in measure of "excess tissue piercing" and an expert rating of "tissue tearing." The mean number of excess tissue piercing after training (but not after the neutral activity) was significantly lower than before training (3.25 [SD = 1.996] vs 6.75 [SD = 3.68], respectively; p <.001), reflecting an improvement of 52% (decreasing the mean number of excess tissue piercing in a single suture by 3.5 excess piercing trials). After the interventions, the extent of tissue tearing was rated lower in the training group (p = .01), and there was no change in the control group (p = .14). We showed the efficiency of a training approach that focuses on spatial skills critical in robot-assisted surgery. We showed that surgeons who received a 1 session spatial skill training with a cognitive spatial skill trainer immediately improved the performance of a robotic suturing task compared with surgeons who did not receive such training.

Objective assessment of robotic suturing skills with a new computerized system: A step forward in the training of robotic surgeons.

The purpose of this study was to assess robot-assisted suturing skills on a laparoscopic intestinal anastomosis model by using a novel computerized objective assessment system. This study compared the suturing skills of 13 surgically naïve participants on an artificial intestinal anastomosis model that mimics real tissue. Each examinee sutured using da Vinci robot assistance under 2-D and 3-D visualization and with conventional laparoscopy (CL). Pressure-measuring and image-processing devices were employed to quantitatively evaluate suturing skills. Five unique criteria were used to evaluate the skills of participants. Suturing under 3-D visualization (P < 0.01) and with CL (P < 0.05) were significantly faster than under 2-D visualization. Sutures placed under 3-D (P < 0.05) and 2-D (P < 0.01) visualization had significantly better suture tension than those placed with CL, which did not meet acceptable values for suture tension. Sutures placed with CL had significantly better air pressure leakage than those placed under 2-D visualization (P < 0.05), which did not meet acceptable values for air pressure leakage and wound opening area. One participants failed to achieve full-thickness sutures with 2-D, two participants with CL, and one participant with 3-D. Using 3-D vision is necessary for complex maneuvering during robot-assisted minimally invasive surgery. Our quantitative assessment system is useful for evaluating the skill acquisition of surgeon-trainees undergoing robotic surgery training.

The Effect of Visual-Spatial Ability on the Learning of Robot-Assisted Surgical Skills

The aim of this study was to determine the correlation of visual-spatial ability with progression along the learning curve for robotic surgical skills training. A total of 21 novice participants were recruited. All participants completed a training program consisting of 5 training sessions of 30 minutes of virtual reality (VR) simulation and 30 minutes of dry laboratory training. The VR simulation part was the subject of the present study. During VR simulation training, participants performed the basic skill exercises of Camera Targeting 1, Pick and Place, and Peg Board 1 followed by advanced skill exercises of Suture Sponge 1 and Thread the Rings. The visual-spatial ability was assessed using a mental rotation test (MRT). Pearson correlation coefficients were used to assess the relationship between the MRT score and simulator score for the aforementioned 5 tasks. Student t test was used to compare the simulator score between high- and low-MRT score groups. A median MRT score of 26/40 (range: 13-38) was observed. Approximately 19 participants completed the full curriculum but 2 did not complete "Thread the Rings" during the study period. A significant correlation was observed between the MRT score and simulator score only in "Suture Sponge 1" over the first 3 attempts (first: r = 0.584, p = 0.0054; second: r = 0.443, p = 0.0443; third: r = 0.4458, p = 0.0428). After the third attempt, this significant correlation was lost. Comparison of the score for "Suture Sponge 1" between the high-MRT and low-MRT scoring participants divided by a median MRT score of 26 also showed a significant difference in the score until the third trial. Our observations suggest that the spatial cognitive ability influences the initial learning of robotic suturing skills. Further studies are necessary to verify the usefulness of an individual's spatial ability to tailor the surgical training program.

Transferability of Virtual Reality, Simulation-Based, Robotic Suturing Skills to a Live Porcine Model in Novice Surgeons: A Single-Blind Randomized Controlled Trial

Study ObjectiveTo assess whether a robotic simulation curriculum for novice surgeons can improve performance of a suturing task in a live porcine model. DesignRandomized controlled trial (Canadian Task Force classification I). SettingAcademic medical center. PatientsThirty-five medical students without robotic surgical experience. InterventionsParticipants were enrolled in an online session of training modules followed by an in-person orientation. Baseline performance testing on the Mimic Technologies da Vinci Surgical Simulator (dVSS) was also performed. Participants were then randomly assigned to the completion of 4 dVSS training tasks (camera clutching 1, suture sponge 1 and 2, and tubes) versus no further training. The intervention group performed each dVSS task until proficiency or up to 10 times. A final suturing task was performed on a live porcine model, which was video recorded and blindly assessed by experienced surgeons. The primary outcomes were Global Evaluative Assessment of Robotic Skills (GEARS) scores and task time. The study had 90% power to detect a mean difference of 3 points on the GEARS scale, assuming a standard deviation (SD) of 2.65, and 80% power to detect a mean difference of 3 minutes, assuming an SD of 3 minutes. Measurements and Main ResultsThere were no differences in demographics and baseline skills between the 2 groups. No significant differences in task time in minutes or GEARS scores were seen for the final suturing task between the intervention and control groups, respectively (9.2 [2.65] vs 9.9 [2.07] minutes, p = .406; and 15.37 [2.51] vs 15.25 [3.38], p = .603). The 95% confidence interval for the difference in mean task times was –2.36 to .96 minutes and for mean GEARS scores −1.91 to 2.15 points. ConclusionsLive suturing task performance was not improved with a proficiency-based virtual reality simulation suturing curriculum compared with standard orientation to the da Vinci robotic console in a group of novice surgeons.

Virtual reality robotic surgery simulation curriculum to teach robotic suturing: a randomized controlled trial.

The objective of this randomized, controlled trial was to assess whether voluntary participation in a proctored, proficiency-based, virtual reality robotic suturing curriculum using the da Vinci(®) Skills Simulator™ improves robotic suturing performance. Residents and attending surgeons were randomized to participation or non-participation during a 5week training curriculum. Robotic suturing skills were evaluated before and after training using an inanimate vaginal cuff model, which participants sutured for 10min using the da Vinci(®) Surgical System. Performances were videotaped, anonymized, and subsequently graded independently by three robotic surgeons. 27 participants were randomized. 23 of the 27 completed both the pre- and post-test, 13 in the training group and 10 in the control group. Mean training time in the intervention group was 238±136min (SD) over the 5weeks. The primary outcome (improvement in GOALS+ score) and the secondary outcomes (improvement in GEARS, total knots, satisfactory knots, and the virtual reality suture sponge 1 task) were significantly greater in the training group than the control group in unadjusted analysis. After adjusting for lower baseline scores in the training group, improvement in the suture sponge 1 task remained significantly greater in the training group and a trend was demonstrated to greater improvement in the training group for the GOALS+ score, GEARS score, total knots, and satisfactory knots.

Effect of Residents’ Previous Laparoscopic Surgery Experience on Initial Robotic Suturing Experience

Objective. To assess the impact of gynecology residents' previous laparoscopic experience on the learning curve of robotic suturing techniques and the value of initial structured teaching in dry lab prior to surgery. Methods. Thirteen gynecology residents with no previous robotic surgery experience were divided into Group 1, consisting of residents with 2 or fewer laparoscopic experiences, and Group 2, consisting of residents with 3 or more laparoscopic experiences. Group 1 had a dry-laboratory training in suturing prior to their initial experience in the operating room. Results. For all residents, it took on average 382 ± 159 seconds for laparoscopic suturing and 326 ± 196 seconds for robotic suturing (P = 0.12). Residents in Group 1 had a lower mean suture time than residents in Group 2 for laparoscopic suturing (P = 0.009). The residents in Group 2, however, had a lower mean suture time on the robot compared to Group 1 (P = 0.5). Conclusion. Residents with previous laparoscopic suturing experience may gain more from a robotic surgery experience than those with limited laparoscopic surgery experience. In addition, dry lab training is more efficient than hands-on training in the initial phase of teaching for both laparoscopic and robotic suturing skills.

Learning curves in expert and non-expert laparoscopic surgeons for robotic suturing with the da Vinci(®) Surgical System.

We investigated learning curves for robotic suturing of expert and non-expert laparoscopic surgeons to explore the length of time required to reach an acceptable plateau of technical skills. Laparoscopic suturing skills were evaluated in a training box with conventional laparoscopic instrumentation in phase 1. In phase 2, robotic suturing skills were evaluated during a training program on non-surviving animals by analyzing time required for five intracorporal stitches on the small bowel. Learning curves were plotted. A significant difference in technical skills between the expert and non-expert surgeons was demonstrated in phase 1 and at the beginning of phase 2. Both surgeons reached a learning-curve plateau exhibiting similar robotic suturing skills at the end of 90min of training. Skills were subsequently retained equally by both surgeons. Short duration of training was sufficient for the non-expert laparoscopic surgeon to match the robotic suturing performance of the expert laparoscopic surgeon.

Global Evaluative Assessment of Robotic Skills: Validation of a Clinical Assessment Tool to Measure Robotic Surgical Skills

We developed and validated a standardized assessment tool for robotic surgical skills, and report its usefulness, reliability and construct validity in a clinical setting. The Global Evaluative Assessment of Robotic Skills is a tool developed by deconstructing the fundamental elements of robotic surgical procedures in consultation with expert robotic surgeons. Surgical performance was assessed during robot-assisted laparoscopic prostatectomy on a 5-point anchored Likert scale across 6 domains. An overall performance score was derived by summing the ratings in each domain. Expert surgeons and postgraduate year 4 to 6 urology residents were evaluated to determine construct validity. Assessments were completed by the attending surgeon, a trained observer and the operator. A total of 29 evaluations of 25 trainees and 4 attending surgeons were completed. Experts scored significantly higher on the assessment than novice operators (p = 0.004). Postgraduate year 4 and 5 residents scored significantly lower than the expert group (p <0.05) while no difference was observed between mean performance scores of postgraduate year 6 trainees and attending surgeons (p >0.05). The internal consistency of the assessment tool was excellent (Cronbach's α = 0.90 to 0.93). The overall assessment score ICC among raters was 0.80 (95% CI 0.65-0.90). The Global Evaluative Assessment of Robotic Skills is simple to administer and able to differentiate levels of robotic surgical expertise. This standardized assessment tool shows excellent consistency, reliability and validity. Further study is warranted to evaluate its usefulness for surgical education and the establishment of competency in robotic surgery.

Ahead of the curve: factors affecting the acquisition of robotic suturing skills

Ahead of the curve: factors affecting the acquisition of robotic suturing skills