Global Operative Assessment Of Laparoscopic Skills Research Articles

Structured feedback and operative video debriefing with critical view of safety annotation in training of laparoscopic cholecystectomy: a randomized controlled study

BackgroundThe learning curve in minimally invasive surgery (MIS) is lengthened compared to open surgery. It has been reported that structured feedback and training in teams of two trainees improves MIS training and MIS performance. Annotation of surgical images and videos may prove beneficial for surgical training. This study investigated whether structured feedback and video debriefing, including annotation of critical view of safety (CVS), have beneficial learning effects in a predefined, multi-modal MIS training curriculum in teams of two trainees.MethodsThis randomized-controlled single-center study included medical students without MIS experience (n = 80). The participants first completed a standardized and structured multi-modal MIS training curriculum. They were then randomly divided into two groups (n = 40 each), and four laparoscopic cholecystectomies (LCs) were performed on ex-vivo porcine livers each. Students in the intervention group received structured feedback after each LC, consisting of LC performance evaluations through tutor-trainee joint video debriefing and CVS video annotation. Performance was evaluated using global and LC-specific Objective Structured Assessments of Technical Skills (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS) scores.ResultsThe participants in the intervention group had higher global and LC-specific OSATS as well as global and LC-specific GOALS scores than the participants in the control group (25.5 ± 7.3 vs. 23.4 ± 5.1, p = 0.003; 47.6 ± 12.9 vs. 36 ± 12.8, p < 0.001; 17.5 ± 4.4 vs. 16 ± 3.8, p < 0.001; 6.6 ± 2.3 vs. 5.9 ± 2.1, p = 0.005). The intervention group achieved CVS more often than the control group (1. LC: 20 vs. 10 participants, p = 0.037, 2. LC: 24 vs. 8, p = 0.001, 3. LC: 31 vs. 8, p < 0.001, 4. LC: 31 vs. 10, p < 0.001).ConclusionsStructured feedback and video debriefing with CVS annotation improves CVS achievement and ex-vivo porcine LC training performance based on OSATS and GOALS scores.

Keep it simple: a crosshair on the screen during proctored laparoscopic surgeries.

Inside the operating room, experts use verbal instructions to guide surgical novices through laparoscopic procedures. In this study, we evaluated the use of a crosshair attached to the video monitor, as a hands-free pointing tool to simplify instructions during operation. Ten surgical novices performed two elective laparoscopic cholecystectomies within a week of each other, randomized such that one was performed with and the other without using the crosshair. Directly after operation, questionnaires were completed by the novices and the consultant surgeons. Measures including the comprehensibility of instructions, subjective feeling of safety during preparation, time delays due to different instruction options, and disruptive influence while instructors used the crosshair. Differences in operative performance were evaluated based on the global operative assessment of laparoscopic skills (GOALS) scores. When the crosshair was used, surgical novices had a better understanding of which anatomical structure should be shown (p = 0.028). Operating time (p = 0.222) and feeling of confidence during preparation did not differ with versus without crosshair use (p = 0.081). All participants stated that the crosshair did not negatively affect the field of vision. In terms of the median GOALS score, the operative performance was improved when the crosshair was used compared with verbal instructions only (median 15, IQR (11; 21) vs. median 12, IQR (5; 19), p < 0.001). The crosshair is a simple, inexpensive, and widely available method to improve communication between instructors and novices in everyday training.

Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery.

Surgical assessment instruments are used for formative and summative trainee evaluations. To characterize the features of existing instruments and a novel 12-item objective, procedure-specific assessment tool for Roux-en-Y Gastric Bypass (RYGB-OPSA), we evaluated the progress of a single surgical fellow over 17 consecutive surgeries. Seventeen consecutive RYGB videos completed between 8/2021 and 1/2022 by an academic hospital surgical fellow were de-identified and assessed by four board-certified bariatric surgeons using Global Operative Assessment of Laparoscopic Skills (GOALS), General Assessment of Surgical Skill (GASS), and RYGB-OPSA which includes the reflection of transverse colon, identification of ligament of Treitz, biliopancreatic and Roux limbs orientation, jejunal division point selection, stapler use, mesentery division, bleeding control, jejunojejunostomy (JJ) anastomotic site selection, apposition of JJ anastomotic site, JJ creation, common enterotomy closure of JJ, and integrity of anastomosis. The GASS measured economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, and achievement of hemostasis. RYGB-OPSA and GASS items were scored "poor-unsafe," "acceptable-safe," or "good-safe." Change in performance was measured by linear trendline slope. Over the course of 17 procedures, significant improvement was demonstrated by three GOALS items, GOALS overall score, GASS bimanual dexterity, and three RYGB-OPSA tasks: JJ creation, jejunal division point selection, and stapler use. Achievement of hemostasis declined but never rated "poor-unsafe." Overall RYGB-OPSA and GOALS trendlines documented significant increase across the 17 procedures. This examination of a bariatric surgery fellow's operative training experience as measured by three surgical assessment instruments demonstrated anticipated improvements in general skills and safe completion of procedure-specific tasks. Effective surgical assessment instruments have enough sensitivity to show improvement to enable meaningful trainee feedback (low-stakes assessments) as well as the ability to determine safe surgical practice to enable promotion to greater autonomous practice.

Crowd-sourced and expert video assessment in minimally invasive esophagectomy.

Video-based assessment by experts may structurally measure surgical performance using procedure-specific competency assessment tools (CATs). A CAT for minimally invasive esophagectomy (MIE-CAT) was developed and validated previously. However, surgeon's time is scarce and video assessment is time-consuming and labor intensive. This study investigated non-procedure-specific assessment of MIE video clips by MIE experts and crowdsourcing, collective surgical performance evaluation by anonymous and untrained laypeople, to assist procedure-specific expert review. Two surgical performance scoring frameworks were used to assess eight MIE videos. First, global performance was assessed with the non-procedure-specific Global Operative Assessment of Laparoscopic Skills (GOALS) of 64 procedural phase-based video clips < 10min. Each clip was assessed by two MIE experts and > 30 crowd workers. Second, the same experts assessed procedure-specific performance with the MIE-CAT of the corresponding full-length video. Reliability and convergent validity of GOALS for MIE were investigated using hypothesis testing with correlations (experience, blood loss, operative time, and MIE-CAT). Less than 75% of hypothesized correlations between GOALS scores and experience of the surgical team (r < 0.3), blood loss (r = -0.82 to 0.02), operative time (r = -0.42 to 0.07), and the MIE-CAT scores (r = -0.04 to 0.76) were met for both crowd workers and experts. Interestingly, experts' GOALS and MIE-CAT scores correlated strongly (r = 0.40 to 0.79), while crowd workers' GOALS and experts' MIE-CAT scores correlations were weak (r = -0.04 to 0.49). Expert and crowd worker GOALS scores correlated poorly (ICC ≤ 0.42). GOALS assessments by crowd workers lacked convergent validity and showedpoor reliability. It is likely that MIE is technically too difficult to assess for laypeople. Convergent validity of GOALS assessments by experts could also not be established. GOALS might not be comprehensive enough to assess detailed MIE performance. However, expert's GOALS and MIE-CAT scores strongly correlated indicating video clip (instead of full-length video) assessments could be useful to shorten assessment time.

Global assessment of surgical skills (GASS): validation of a new instrument to measure global technical safety in surgical procedures.

Broad implementation of the American Board of Surgery's entrustable professional activities initiative will require assessment instruments that are reliable and easy to use. Existing assessment instruments of general laparoscopic surgical skills have limited reliability, efficiency, and validity across the spectrum of formative (low-stakes) and summative (high-stakes) assessments. A novel six-item global assessment of surgical skills (GASS) instrument was developed and evaluated with a focus upon safe versus unsafe surgical practice scoring rubric. The GASS was developed by iterative engagement with expert laparoscopic surgeons and includes six items (economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, achievement of hemostasis, overall performance) with a uniform three-point scoring rubric ("poor-unsafe", "adequate-safe", "good-safe"). To test inter-rater reliability, a cross-sectional study of four bariatric surgeons with experience ranging from 4 to 28years applied the GASS and the global operative assessment of laparoscopic skills (GOALS) to 30 consecutive Roux-en-Y gastric bypass procedure operative videos. Inter-rater reliability was assessed for a simplified dichotomous "safe" versus "unsafe" scoring rubric using Gwet's AC2. The GASS inter-rater reliability was very high across all six domains (0.88-1.00). The GASS performed comparably to the GOALS inter-rater reliability scores (0.96-1.00). The economy of motion and bimanual dexterity items had the highest percentage of unsafe ratings (9.2% and 5.8%, respectively). The GASS, a novel six-item instrument of general laparoscopic surgical skills, was designed with a simple scoring rubric (poor-safe, adequate-safe, good-safe) to minimize rater burden and focus feedback to trainees and promotion evaluations on safe surgical performance. Initial evaluation of the GASS is promising, demonstrating high inter-rater reliability. Future research will seek to assess the GASS against a broader spectrum of laparoscopic procedures.

Laparoscopic training on virtual-reality simulators or live pigs-a randomized controlled trial.

Thirty-six novice surgical residents without independent laparoscopic experience were randomly paired with a peer and randomized into three groups: VR simulator group (dyad training on LapSim VR simulators), pig surgery group (training on live, anesthetized pigs) and control group (training by a lecture on laparoscopic surgery, surgical videos and textbooks). After 6h of training, all participants performed a simulated cholecystectomy procedure using a pig liver with adherent gallbladder working in pairs. All procedures were video-recorded and the recordings were saved on USB-sticks in a blinded fashion identifiable only by the unique participant number. All video-recordings were scored blindly and independently by two expert raters using the Global Operative Assessment of Laparoscopic Skills (GOALS) assessment instrument. The performances in the three groups were significantly different, P less than 0.001. Both the VR simulation training group and the live pigs training group performed significantly better than the control group, both P values less than 0.001. However, there was no significant difference in the performance of the two simulation-based training groups, P=0.66. Novice surgical trainees can benefit from both VR simulator training and pig surgery simulation compared with traditional studying and there was no significant difference between the two modalities. The authors recommend that VR simulators should be used for basic training of laparoscopic skills and surgery on live animals should be reserved for higher-level surgical training.

Standardizing Resident Laparoscopic Hysterectomy Performance Evaluations Using Global Operative Assessment of Laparoscopic Skills

Objective: Surgical feedback is critical for resident education, but lacks standardization. Global Operative Assessment of Laparoscopic Skills (GOALS) is used to evaluate resident training in laparoscopic cholecystectomy and appendectomy. This study examined the effectiveness of adapting GOALS for laparoscopic hysterectomy in a single residency-training program. Methods: A single-page evaluation form was used for senior resident–attending dyads who were performing laparoscopic hysterectomies together. Senior residents completed self-evaluations and attendings completed trainee evaluations after each case. A 10-item checklist detailed key steps in the operation. Evaluators used the checklist to indicate if trainees completed each step successfully. Five laparoscopic domains (GOALS) and assessment of case difficulty were scored on a 5-point scale. Overall satisfaction with resident performance was rated on a 10-point scale. Results: Over 21 months, 10 residents and 3 attendings completed 97 evaluations. The mean step completion resident score was 62% (attending score: 63%; p = 0.10), indicating agreement between operators and evaluators in case assessments. Compared to attending ratings of resident performances, residents rated themselves lower overall in the 5 laparoscopic domains (3.26 versus 3.53; p < 0.05) despite similar case difficulty estimates (2.66 versus 2.76; p = 0.43). Positive correlations between attending and resident scores indicated consistency between users. Conclusions: GOALS can standardize resident performance evaluation during laparoscopic hysterectomy. There was interevaluator agreement; trainees were more self-critical. An electronic format could make the form easier to use. (J GYNECOL SURG 20XX:000)

Value of the Crowd-Sourced Assessment of Technical Skills (C-SATS) Platform in Surgical Procedures: A Systematic Review of Evidence

Background: Crowd-Sourced Assessment of Technical Skills (C-SATS) is a surgical data management and learning platform that leverages the knowledge of large expert surgeon and lay groups to assess the technique and technical skills of surgeons in a highly efficient manner. The aim of this systematic review was to summarize published literature on the performance of C-SATS as compared to expert evaluations and assess its use as a training and validation tool in minimally invasive surgery (MIS). Methods: A comprehensive literature search was performed per PRISMA guidelines using the Medline, Embase, and Google Scholar databases on published studies that evaluated the use of C-SATS following MIS, such as laparoscopic or robotic-assisted surgery. Results: A total of 21 reports were included in the review. Twelve studies comparing crowd-sourcing evaluations against expert opinion indicated overall excellent or good correlation with Global Operative Assessment of Laparoscopic Skills (GOALS), Global Evaluative Assessment of Robotic Skills (GEARS), and Robotic Objective Structured Assessment of Technical Skills (R-OSATS) scores, with correlation coefficients (Pearson or Spearman) ranging from 0.69 to 0.95 and reliability index (Cronbach’s alpha) from 0.63 to 0.93 across different specialties and surgical approaches. When using C-SATS to assess performance and validation, assessments positively correlated with traditional methods of time and error-based scoring and global rating scale. Conclusions: Based on the current published literature, the C-SATS platform has been shown to efficiently provide crowd-sourced evaluations that correlate favorably with expert evaluation across a range of surgical specialties and approaches. Use of crowdsourcing has uniformly yielded accurate evaluations of surgeons’ technical skills in a markedly shorter time than expert reviews. C-SATS may be a cost-effective complement or alternative to traditional models of evaluating surgical proficiency. Future studies are needed to determine whether the use of C-SATS will lead to improved surgical performance and patient outcomes.

Video Playback as an Educational Tool for Improving Resident Laparoscopic Performance: A Randomized Controlled Trial

Study ObjectiveIn this single-masked randomized controlled study, we evaluate whether watching video recordings of oneself performing Fundamentals of Laparoscopic Skills (FLS) exercises results in an improvement on the 30-point Global Operative Assessment of Laparoscopic Skills (GOALS) assessment among Obstetrics and Gynecology (Ob/Gyn) residents. DesignTwenty-three Ob/Gyn residents in the 2020–2021 academic year completed the FLS exercises while being timed, video recorded, and receiving real-time feedback from an Ob/Gyn faculty member. Baseline GOALS assessment was completed by participants and faculty. After the intervention, all participants then repeated the FLS exercises while being timed and were again scored using the GOALS assessment. Each participant completed the study in a single session. In addition, all participants completed a pre- and post-test survey. SettingUniversity of Louisville Laparoscopic Skills Labortaory. ParticipantsUniversity of Louisville Ob/Gyn residents in the 2020–2021 academic year. InterventionTwelve participants were randomized to the intervention and were allowed to watch their video recording in addition to receiving verbal feedback whereas the remaining 11 received verbal feedback only. Measurements and Main ResultsThere were significant improvements in faculty (p <.01) and self-reported GOALS scores (p <.01) when comparing both the intervention and control group with baseline scores. The intervention group improved by 3.2 points more than that of the control group based on masked faculty evaluation (95% confidence interval, 1.4–5.0 points; p <.01). The difference was not significant in resident self-scores. Overall time improved for all participants (15:54 ± 0.21 minutes before and 13:13 ± 0.14 minutes after), but this difference was not significant between the 2 groups. Higher postgraduate year (PGY) residents reported significantly more comfort performing laparoscopic tasks, earned higher GOALS score (faculty and self-scores), and completed the first set of exercises in less time. A significant interaction between PGY and intervention was detected with improvement in GOALS score most strongly associated with PGY4 residents (p <.01). ConclusionAlthough all learners objectively benefit from watching recordings of themselves performing surgical tasks, advanced learners may benefit the most when video recordings are used as an educational tool.

282. VALIDITY AND RELIABILITY OF CROWD-SOURCED AND EXPERT VIDEO ASSESSMENT IN MINIMALLY INVASIVE ESOPHAGECTOMY USING GLOBAL RATING SCALES

Abstract Minimally Invasive Esophagectomy (MIE) is a complex and technically demanding procedure and implementation is associated with a substantial learning curve. Suboptimal surgical performance in complex minimally invasive procedures has convincingly been related to less favorable patient outcomes. Structured video-based assessment by other surgeons can objectively evaluate surgical performance, but requires a surgeon’s scarce time. This study investigated whether crowdsourcing, collective surgical performance evaluation by anonymous and untrained laypeople, could be a valid alternative for MIE. Two video assessments types were conducted via the C-SATS platform with eight MIE operative videos. First, videos were cut into eight procedural phase-based video clips of maximum ten minutes. These 64 video clips were assessed with the validated Global Operative Assessment of Laparoscopic Skills (GOALS)‘s four domains (depth perception, bimanual dexterity, efficiency and tissue handling) using a 1-5 Likert scale. Each video clip was assessed by a pair of two MIE experts and &amp;gt; 30 crowd workers, recruited via Amazon Mechanical Turk. Second, the same expert pairs assessed the corresponding complete video with the procedure-specific MIE competency assessment tool (MIE-CAT). Experts assessed 128 clips and sixteen complete videos in 3.5 months. Crowd workers assessed 1984 clips in &amp;lt; two days. Expert scores showed a wider range (expert GOALS score range 7.50-19.50 versus 12.84-15.63 for crowds). When comparing experts and crowd workers scores, a poor correlation was found on all domains and total GOALS scores (ICC ≤ 0.42). When correlating GOALS scores with MIE-CAT scores a strong, positive correlation between experts’ scores was found (r¬s = 0.76, p = 0.03) while between crowd’s GOALS and expert’s MIE-CAT scores a small correlation was found (r¬s = 0.23, p = 0.58) (Figure 1). Despite rapid results from crowd workers, a poor agreement between crowd and experts’ GOALS scores was observed. MIE might be technically too difficult to assess for laypeople. Furthermore, GOALS might not be extensive enough to assess MIE performance in detail, while the procedure-specific MIE-CAT might be more comprehensive. However, the correlation between the experts’ MIE-CAT and GOALS scores indicates video clip assessments (instead of full length video assessments) could be helpful to shorten assessment time.

Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy.

IntroductionSelf-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we aimed to contribute evidence regarding the validity of intraoperative performance assessment tools for video-based self-assessment by general surgery trainees when performing laparoscopic cholecystectomies.Methods and proceduresUsing a web-based platform, general surgery trainees in a university-based residency program submitted recorded laparoscopic cholecystectomy procedures where they acted as the supervised primary surgeon. Attending surgeons measured operative performance at the time of surgery using general and procedure-specific assessment tools [Global Operative Assessment of Laparoscopic Skills (GOALS) and Operative Performance Rating System (OPRS), respectively] and entrustability level (O-SCORE). Trainees self-evaluated their performance from video-review using the same instruments. The validity of GOALS and OPRS for trainee self-assessment was investigated by testing the hypotheses that self-assessment scores correlate with (H1) expert assessment scores, (H2) O-SCORE, and (H3) procedure time and that (H4) self-assessment based on these instruments differentiates junior [postgraduate year (PGY) 1–3] and senior trainees (PGY 4–5), as well as (H5)simple [Visual Analogue Scale (VAS) ≤ 4] versus complex cases (VAS > 4). All hypotheses were based on previous literature, defined a priori, and were tested according to the COSMIN consensus on measurement properties.ResultsA total of 35 videos were submitted (45% female and 45% senior trainees) and self-assessed. Our data supported 2 out of 5 hypotheses (H1 and H4) for GOALS and 3 out of 5 hypotheses (H1, H4 and H5) for OPRS, for trainee self-assessment.ConclusionsOPRS, a procedure-specific assessment tool, was better able to differentiate between groups expected to have different levels of intraoperative performance, compared to GOALS, a general assessment tool. Given the interest in video-based learning, there is a need to further develop valid procedure-specific tools to support video-based self-assessment by trainees in a range of procedures.Supplementary InformationThe online version contains supplementary material available at 10.1007/s00464-022-09466-6.

Assessment of laparoscopic skills: comparing the reliability of global rating and entrustability tools

Competence by design (CBD) residency programs increasingly depend on tools that provide reliable assessments, require minimal rater training, and measure progression through the CBD milestones. To assess intraoperative skills, global rating scales and entrustability ratings are commonly used but may require extensive training. The Competency Continuum (CC) is a CBD framework that may be used as an assessment tool to assess laparoscopic skills. The study aimed to compare the CC to two other assessment tools: the Global Operative Assessment of Laparoscopic Skills (GOALS) and the Zwisch scale. Four expert surgeons rated thirty laparoscopic cholecystectomy videos. Two raters used the GOALS scale while the remaining two raters used both the Zwisch scale and CC. Each rater received scale-specific training. Descriptive statistics, inter-rater reliabilities (IRR), and Pearson's correlations were calculated for each scale. Significant positive correlations between GOALS and Zwisch (r = 0.75, p < 0.001), CC and GOALS (r = 0.79, p < 0.001), and CC and Zwisch (r = 0.90, p < 0.001) were found. The CC had an inter-rater reliability of 0.74 whereas the GOALS and Zwisch scales had inter-rater reliabilities of 0.44 and 0.43, respectively. Compared to GOALS and Zwisch scales, the CC had the highest inter-rater reliability and required minimal rater training to achieve reliable scores. The CC may be a reliable tool to assess intraoperative laparoscopic skills and provide trainees with formative feedback relevant to the CBD milestones. Further research should collect further validity evidence for the use of the CC as an independent assessment tool.

The impact of virtual reality simulation training on operative performance in laparoscopic cholecystectomy: meta-analysis of randomized clinical trials.

BackgroundSimulation training can improve the learning curve of surgical trainees. This research aimed to systematically review randomized clinical trials (RCT) evaluating the performance of junior surgical trainees following virtual reality training (VRT) and other training methods in laparoscopic cholecystectomy.MethodsMEDLINE (PubMed), Embase (Ovid SP), Web of Science, Scopus and LILACS were searched for trials randomizing participants to VRT or no additional training (NAT) or simulation training (ST). Outcomes of interest were the reported performance using global rating scores (GRS), the Objective Structured Assessment of Technical Skill (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS), error counts and time to completion of task during laparoscopic cholecystectomy on either porcine models or humans. Study quality was assessed using the Cochrane Risk of Bias Tool. PROSPERO ID: CRD42020208499.ResultsA total of 351 titles/abstracts were screened and 96 full texts were reviewed. Eighteen RCT were included and 15 manuscripts had data available for meta-analysis. Thirteen studies compared VRT and NAT, and 4 studies compared VRT and ST. One study compared VRT with NAT and ST and reported GRS only. Meta-analysis showed OSATS score (mean difference (MD) 6.22, 95%CI 3.81 to 8.36, P < 0.001) and time to completion of task (MD −8.35 min, 95%CI 13.10 to 3.60, P = <0.001) significantly improved after VRT compared with NAT. No significant difference was found in GOALS score. No significant differences were found between VRT and ST groups. Intraoperative errors were reported as reduced in VRT groups compared with NAT but were not suitable for meta-analysis.ConclusionMeta-analysis suggests that performance measured by OSATS and time to completion of task is improved with VRT compared with NAT for junior trainee in laparoscopic cholecystectomy. However, conclusions are limited by methodological heterogeneity and more research is needed to quantify the potential benefit to surgical training.

Establishing validity evidence for device-assisted advanced laparoscopic suturing tasks using simulation.

Laparoscopic suturing (LS) is an essential technique required for a wide range of procedures, and it is one of the most challenging for surgical trainees to master. We designed and collected validity evidence for advanced LS tasks using an automated suturing device and evaluated the perceived educational value of the tasks. This project was a multicentre prospective study involving McGill University, University of Toronto (UofT), and Louisiana State University (LSU) Health New Orleans. Novice (NS) and experienced (ES) surgeons performed suturing under tension (UT) and continuous suturing (CS) tasks. ES performed the tasks twice to establish proficiency benchmarks, and they were interviewed to develop formative feedback tools (FFT). Participants were assessed on completion time, error, Global Operative Assessment of Laparoscopic Skills (GOALS), and FFT. Data were analyzed using descriptive and inferential statistical methods. Twenty-seven participants (13 ES, 14 NS, median age 34years; 85% male) completed the study. Eight were attending surgeons, 7 fellows, 4 PGY5, 5 PGY4, and 3 PGY3 (18 from McGill, 5 UofT, and 4 LSU). Comparing ES and NS, for UT task, ES had significantly greater task scores (570 [563-648] vs 323 [130-464], p value 0.00036) and GOALS scores (14 [13-16] vs 10 [8-12], p value 0.0038). Similarly, for CS, ES had significantly greater task scores (976 [959-1010] vs 785 [626-856], p value 0.00009) and GOALS scores (16 [12-17] vs 12.5 [8.25-15], p value 0.028). After FFTs were developed, comparing ES and NS, for both UT and CS tasks, ES had significantly greater FFT scores (UT 25 [24-26] vs 17 [14-20], p value 0.0016 and CS 30 [27-32] vs 22[17.2-25.8], p value 0.00061). In conclusion, preliminary validity evidence was provided for the tasks. Once further validity evidence is established, incorporating the tasks into the training curricula could improve trainee skills and help to meet the need for better advanced suturing models.

Pilot Evaluation of a Novel, Low-Cost, Simulation Model for Training and Assessment of Laparoscopic Intracorporeal Continuous Suturing.

Background: Laparoscopic intracorporeal continuous suturing is being employed in a growing number of minimally invasive procedures. However, there is a lack of adequate bench models for gaining proficiency in this complex task. The purpose of this study was to assess a novel simulation model for running suture. Methods: Participants were grouped as novice (LSN) or expert (LSE) at laparoscopic suturing based on prior experience and training level. A novel low-cost bench model was developed to simulate laparoscopic intracorporeal continuous closure of a defect. The primary outcome measured was time taken to complete the task. Videos were scored by independent raters for Global Operative Assessment of Laparoscopic Skills (GOALS). Results: Sixteen subjects (7 LSE and 9 LSN) participated in this study. LSE completed the task significantly faster than LSN (430 ± 107 vs 637 ± 164seconds, P ≤ .05). LSN scored higher on accuracy penalties than LSE (Median 30 vs 0, P ≤ .05). Mean GOALS score was significantly different between the 2 groups (LSE 20.64 ± 2.64 vs LSN 14.28 ± 1.94, P < .001) with good inter-rater reliability (ICC ≥ .823). An aggregate score using the formula: Performance Score = 1200-time(sec)-(accuracy penalties x 10) was significantly different between groups with a mean score of 741 ± 141 for LSE vs 285 ± 167 for LSN (P < .001). Conclusion A novel bench model for laparoscopic continuous suturing was able to significantly discriminate between laparoscopic experts and novices. This low-cost model may be useful for both training and assessment of laparoscopic continuous suturing proficiency.

Resident performance in simulation module is associated with operating room performance for laparoscopic cholecystectomy.

Simulation is an important tool in surgical training. However, the transferability of skills obtained in the simulation setting to the operating room (OR) is uncertain. This study explores the association between resident simulation performance and OR performance in a laparoscopic cholecystectomy (LC) simulation module. A simulation module focused on LC utilizing a virtual reality simulator was completed by general surgery residents. Simulation performance was evaluated using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) and Objective Structured Assessment of Technical Skills (OSATS), as well as a LC-specific simulation assessment form (LC-SIM). Resident subsequent OR performances of LC were measured by the Surgical Training and Assessment Tool (STAT), an online mobile-based evaluation completed by attending surgeons. Twenty-one residents who completed the simulation module and also with STAT data on LC from 2016 to 2020 were included. Higher scores on incision/port placement on LC-SIM is associated with better tissue handling (coefficient 0.20, p = 0.048) and better time & economy of motion on STAT (coefficient 0.22, p = 0.037). However, higher scores on time and motion on OSATS are associated with worse tissue handling (-0.28, p = 0.046), worse time & economy of motion (-0.37, p = 0.009), and worse overall grade (-0.21, p = 0.044). Higher scores on overall performance on OSATS is associated with worse time & economy of motion (-0.80, p = 0.008). Higher scores on depth perception on GOALS are associated with worse tissue handling (-0.28, p = 0.044). We found significant positive and negative associations between resident simulation performance and OR performance, particularly in tissue handling and economy of motion. This could suggest that simulation performance does not reliably predict OR performance. However, this could highlight the concept of excessive caution in the real OR environment and longer operative time which could be interpreted as worse time and economy of motion by the attending surgeons.

Telestration with augmented reality for visual presentation of intraoperative target structures in minimally invasive surgery: a randomized controlled study

AimsIn minimally invasive surgery (MIS), intraoperative guidance has been limited to verbal communication without direct visual guidance. Communication issues and mistaken instructions in training procedures can hinder correct identification of anatomical structures on the MIS screen. The iSurgeon system was developed to provide visual guidance in the operating room by telestration with augmented reality (AR).MethodsLaparoscopic novices (n = 60) were randomized in two groups in a cross-over design: group 1 trained only with verbal guidance first and then with additional telestration with AR on the operative screen and vice versa for group 2. Training consisted of laparoscopic basic training and subsequently a specifically designed training course, including a porcine laparoscopic cholecystectomy (LC). Outcome included time needed for training, performance with Global Operative Assessment of Laparoscopic Skills (GOALS), and Objective Structured Assessment of Technical Skills (OSATS) score for LC, complications, and subjective workload (NASA-TLX questionnaire).ResultsTelestration with AR led to significantly faster total training time (1163 ± 275 vs. 1658 ± 375 s, p < 0.001) and reduced error rates. LC on a porcine liver was performed significantly better (GOALS 21 ± 5 vs. 18 ± 4, p < 0.007 and OSATS 67 ± 11 vs. 61 ± 8, p < 0.015) and with less complications (13.3% vs. 40%, p < 0.020) with AR. Subjective workload and stress were significantly reduced during training with AR (33.6 ± 12.0 vs. 30.6 ± 12.9, p < 0.022).ConclusionTelestration with AR improves training success and safety in MIS. The next step will be the clinical application of telestration with AR and the development of a mobile version for remote guidance.

Watching surgical tapes: video playback as an educational tool for improving resident laparoscopic performance

In this single masked, randomized controlled study, we evaluate if watching video recordings of oneself performing Fundamentals of Laparoscopic Skills (FLS) exercises results in an improvement on the 30-point Global Operative Assessment of Laparoscopic Skills (GOALS) assessment among Obstetrics & Gynecology (Ob/Gyn) residents. Participants were randomized to verbal feedback only (control) or verbal feedback and watching their video recording (intervention). Each participant completed the five FLS exercises using a laparoscopic box trainer while being timed, video recorded and receiving real-time feedback from an Ob/Gyn faculty member. Baseline GOALS assessment was completed by participants and faculty evaluator. Participants randomized to the intervention were allowed to watch their video recording and the faculty evaluator was masked to this randomization. All participants then repeated the FLS exercises while being timed and this was again scored using the GOALS assessment. In addition, all participants completed a pre- and post-test survey to gauge their laparoscopic comfort level. Two-sample t-test was used to assess within-group differences in overall GOALS scores and exercise times while Wilcox Rank Sum test was used for within-group comparisons of survey responses. Between-group analysis for change in GOALS scores, exercise times and survey responses was performed using the Mann-Whitney test. Differences in variables across Post Graduate Year (PGY) levels were evaluated using ANOVA and Kruskal-Wallis tests. Twenty-three Ob/Gyn residents in the 2020-2021 academic year completed the study; 12 were randomized to the intervention and 11 received verbal feedback only. Relative to baseline, there were significant within-group improvements in the faculty (p<0.01) and self-reported GOALS scores (p<0.01). Between the groups, the intervention group improved by 3.2 points more than the control group based on masked faculty evaluation (95% CI 1.4 to 5.0 points, p<0.01). This between-group difference was not significant in resident self-scores. Overall time improved for all participants (15:54 ± 0.21 minutes pre and 13:13 ± 0.14 minutes post) but this difference was not significant between groups. Higher PGY residents reported significantly more comfort performing laparoscopic tasks, earned higher GOALS score (faculty and self-scores), and completed the first set of exercises in less time. A significant interaction between PGY and intervention was detected using 2-way ANOVA with between-group improvement in GOALS score most strongly associated with PGY4 residents (p<0.01). This suggests that while all learners objectively benefit from watching recordings of themselves performing surgical tasks, advanced learners may benefit the most when video recordings are used as an educational tool.

Quantitative influence and performance analysis of virtual reality laparoscopic surgical training system

BackgroundVirtual reality (VR) surgery training has become a trend in clinical education. Many research papers validate the effectiveness of VR-based surgical simulators in training medical students. However, most existing articles employ subjective methods to study the residents’ surgical skills improvement. Few of them investigate how to improve the surgery skills on specific dimensions substantially.MethodsOur paper resorts to physiological approaches to objectively study the quantitative influence and performance analysis of VR laparoscopic surgical training system for medical students. Fifty-one participants were recruited from a pool of medical students. They conducted four pre and post experiments in the training box. They were trained on VR-based laparoscopic surgery simulators (VRLS) in the middle of pre and post experiments. Their operation and physiological data (heart rate and electroencephalogram) are recorded during the pre and post experiments. The physiological data is used to compute cognitive load and flow experience quantitatively. Senior surgeons graded their performance using newly designed hybrid standards for fundamental tasks and Global operative assessment of laparoscopic skills (GOALS) standards for colon resection tasks. Finally, the participants were required to fill the questionnaires about their cognitive load and flow experience.ResultsAfter training on VRLS, the time of the experimental group to complete the same task could drop sharply (p < 0.01). The performance scores are enhanced significantly (p < 0.01). The performance and cognitive load computed from EEG are negatively correlated (p < 0.05).ConclusionThe results show that the VRLS could highly improve medical students' performance and enable the participants to obtain flow experience with a lower cognitive load. Participants' performance is negatively correlated with cognitive load through quantitative physiological analysis. This might provide a new way of assessing skill acquirement.

Subjective vs. objective assessment of simulation performance on laparoscopic cholecystectomy: are we evaluating the right things?

Simulation using virtual reality (VR) simulators is an important tool in surgical training. VR laparoscopic simulators can provide immediate objective performance assessment without observer evaluation. This study aims to explore the correlation between subjective observer evaluation and VR laparoscopic simulator performance metrics in a laparoscopic cholecystectomy (LC) simulation module. A LC simulation module using a VR laparoscopic simulator was completed by PGY2-3 general surgery residents at a single institution. Simulation performance was recorded and evaluated by a trained evaluator using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) form, the Objective Structured Assessment of Technical Skills (OSATS) form, and a LC-specific simulation assessment form (LC-SIM). Objective performance metrics were also obtained from the simulator system. Performance before the curriculum (pre-test) and after the curriculum (post-test) were compared. Fourteen residents were included in the study. There were significant improvements from pre-test to post-test on each component of GOALS, OSATS, and LC-SIM scores (all p values < 0.05). In terms of objective simulator metrics, significant improvements were noted in time to extract gallbladder (481 ± 221 vs 909 ± 366min, p = 0.019), total number of movements (475 ± 264 vs 839 ± 324min, p = 0.012), and total path length (955 ± 475 vs 1775 ± 632cm, p = 0.012) from pre-test to post-test. While number of movements and total path lengths of both hands decreased, speed of right instrument also decreased from 4.1 + 2.7 to 3.0 ± 0.7cm/sec (p = 0.007). Average speed of left instrument was associated with respect for tissue (r = 0.60, p < 0.05) and depth perception (r = 0.68, p < 0.05) on post-test evaluations. Our study demonstrated significant improvement in technical skills based on subjective evaluator assessment as well as objective simulator metrics after simulation. The few correlations identified between the subjective evaluator and the objective simulator assessments suggest the two evaluation modalities were measuring different aspects of the technical skills and should both be considered in the evaluation process.