Assessment Of Surgical Skills Research Articles

Depth over RGB: automatic evaluation of open surgery skills using depth camera.

In this paper, we present a novel approach to the automatic evaluation of open surgery skills using depth cameras. This work is intended to show that depth cameras achieve similar results to RGB cameras, which is the common method in the automatic evaluation of open surgery skills. Moreover, depth cameras offer advantages such as robustness to lighting variations, camera positioning, simplified data compression, and enhanced privacy, making them a promising alternative to RGB cameras. Experts and novice surgeons completed two simulators of open suturing. We focused on hand and tool detection and action segmentation in suturing procedures. YOLOv8 was used for tool detection in RGB and depth videos. Furthermore, UVAST and MSTCN++ were used for action segmentation. Our study includes the collection and annotation of a dataset recorded with Azure Kinect. We demonstrated that using depth cameras in object detection and action segmentation achieves comparable results to RGB cameras. Furthermore, we analyzed 3D hand path length, revealing significant differences between experts and novice surgeons, emphasizing the potential of depth cameras in capturing surgical skills. We also investigated the influence of camera angles on measurement accuracy, highlighting the advantages of 3D cameras in providing a more accurate representation of hand movements. Our research contributes to advancing the field of surgical skill assessment by leveraging depth cameras for more reliable and privacy evaluations. The findings suggest that depth cameras can be valuable in assessing surgical skills and provide a foundation for future research in this area.

Current Advances in Artificial Intelligence in the Field of Aesthetic Surgery and Breast Augmentation: Short Review

Study design: This review is structured as a systematic analysis of the literature to evaluate the impacts of artificial intelligence (AI) on the field of plastic surgery, with a focus on breast augmentation and aesthetic surgical procedures. Key areas of exploration include advancements in machine-learning techniques relevant to plastic surgery, the integration of AI into preoperative planning processes, and the historical evolution of AI in aesthetic surgery. The review also systematically assesses current AI tools specifically developed for breast augmentation, such as 3D imaging and predictive analytics, to understand their efficacy and role in clinical practice. Objective: This article explores the current state and future prospects of AI in plastic surgery, with a focus on breast augmentation and aesthetic procedures. Emphasis is placed on the benefits, challenges, and the imperative for collaborative efforts in the integration of AI technologies. Methods: Advancements in machine-learning algorithms and AI technologies are examined for their potential in automating the assessment and enhancement of surgical skills. The role of AI in facilitating objective evaluations in aesthetic surgery is discussed, addressing challenges such as the lack of standardized training datasets and integration issues. The importance of mitigating potential biases introduced by AI to ensure objectivity in patient assessments is highlighted. Results: The article discusses the historical evolution of AI, from Alan Turing’s conceptualization to contemporary applications in aesthetic surgery. Artificial intelligence’s ability to analyze vast patient datasets is explored, showcasing its potential for offering personalized treatment recommendations and improving accuracy over time. Specific AI tools for breast augmentation, including Canfield Mirror, QuantifiCare LifeViz Infinity Pro, Crisalix, BreastGAN, Arbrea Breast Software (ABS), and Deep Surface AI, are examined in detail, emphasizing their advantages and drawbacks. Conclusions: Evaluation of clinical photography techniques, relying on specific hardware, is presented, with consideration given to the potential of AI-based illumination systems to enhance consistency in preoperative images. The review concludes by envisioning the transformative future of AI in aesthetic surgery, considering its untapped potential in diagnostic imaging, personalized treatment plans, and enhanced surgical precision through integration with virtual and augmented reality. Despite challenges, the promises of AI in personalized treatments, precise patient care, and improved surgical assistance suggest a transformative future for plastic surgery, contingent on addressing current concerns. Collaborative efforts are deemed essential for the successful implementation of AI technologies in the field.

OSABSS: An authentic examination for assessing basic surgical skills in surgical residents

ObjectivesThis study aimed to develop and validate the OSABSS (Objective Structured Assessment of Basic Surgical Skills), a modified Objective Structured Clinical Examination (OSCE), to assess basic surgical skills in residents. DesignA developmental study conducted in two phases. Basic skills were identified through literature review and gap analysis. The OSABSS was then designed as a modified OSCE. SettingThis study took place at Alborz University of Medical Sciences in Iran. InterventionsThe OSABSS was created using Harden's OSCE (Objective Structured Clinical Examination) methodology. Scenarios, checklists, and station configurations were developed through expert panels. The exam was piloted and implemented with residents as participants and faculty as evaluators. Participants32 surgical residents in gynecology, general surgery, orthopedics, and neurosurgery participated. 22 faculty members were evaluators. Primary and secondary outcome measuresThe primary outcome was OSABSS exam scores. Secondary outcomes were written exam scores, and national residency entrance ranks. Main resultsThe mean OSABSS score was 16.59 ± 0.19 across all stations. Criterion validity was demonstrated through correlations between OSABSS scores, written scores and entrance ranks. Reliability was high, with a Cronbach's alpha of 0.87. No significant inter-rater score differences were found. ConclusionsThe rigorous OSABSS development process produced an exam demonstrating strong validity and reliability for assessing basic surgical skills. The comprehensive station variety evaluates diverse technical and non-technical competencies. Further research should expand participant samples across surgical disciplines.

Quantitative assessment and objective improvement of the accuracy of neurosurgical planning through digital patient-specific 3D models.

Neurosurgical patient-specific 3D models have been shown to facilitate learning, enhance planning skills and improve surgical results. However, there is limited data on the objective validation of these models. Here, we aim to investigate their potential for improving the accuracy of surgical planning process of the neurosurgery residents and their usage as a surgical planning skill assessment tool. A patient-specific 3D digital model of parasagittal meningioma case was constructed. Participants were invited to plan the incision and craniotomy first after the conventional planning session with MRI, and then with 3D model. A feedback survey was performed at the end of the session. Quantitative metrics were used to assess the performance of the participants in a double-blind fashion. A total of 38 neurosurgical residents and interns participated in this study. For estimated tumor projection on scalp, percent tumor coverage increased (66.4 ± 26.2%-77.2 ± 17.4%, p = 0.026), excess coverage decreased (2,232 ± 1,322 mm2-1,662 ± 956 mm2, p = 0.019); and craniotomy margin deviation from acceptable the standard was reduced (57.3 ± 24.0 mm-47.2 ± 19.8 mm, p = 0.024) after training with 3D model. For linear skin incision, deviation from tumor epicenter significantly reduced from 16.3 ± 9.6 mm-8.3 ± 7.9 mm after training with 3D model only in residents (p = 0.02). The participants scored realism, performance, usefulness, and practicality of the digital 3D models very highly. This study provides evidence that patient-specific digital 3D models can be used as educational materials to objectively improve the surgical planning accuracy of neurosurgical residents and to quantitatively assess their surgical planning skills through various surgical scenarios.

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review.

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity. A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis. A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes. Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

428 Artificial Intelligence Training Versus In-person Expert Training in Teaching Simulated Tumor Resection Skills - A Cross-Over Randomized Controlled Trial

INTRODUCTION: Neurosurgical simulations equipped with artificial intelligence systems provide an objective quantitative assessment of surgical technical skills and tailored intelligent feedback. These systems allow for repeated practice of complex skills such as subpial brain tumor resection. However, more work is needed to assess the utility of intelligent systems in teaching tumor resection skills, compared to traditional human instruction. METHODS: Twenty-five trainees who are currently enrolled in four Canadian medical schools participated in two training sessions, during which they completed a simulated subpial tumor resection five times. Participants were randomly assigned to two feedback groups: (1) real-time intelligent instruction, and (2) in-person human instruction. They were then assigned to the other feedback group in the second (cross-over) session. A composite-score was given by the intelligent system to assess technical skills during each task. RESULTS: Trainees who received real-time intelligent instruction significantly improved their composite-score in the first and second training sessions (p = .017, p = .005, respectively). Trainees who received in-person human instruction in the first training session had no statistically significant changes in their composite-score from the first to the last task repetition (p = .119). The composite-score decreased significantly in the second training session with in-person human instruction (p = .004). CONCLUSIONS: Real-time intelligent instruction provides an effective way of teaching simulated brain tumor resection skills. Artificial intelligence systems may be a useful addition to current surgical training curricula, providing objective, and tailored assessment and teaching of surgical technical skills in risk-free realistically simulated patient cases. Educators can benefit from these systems to guide their supervision, while trainees can use them for self-guided skill acquisition. Further research may assess the generalizability of these findings to other surgical procedures.

228 Expert Hand Motion During Microvascular Anastomosis Simulation: Big Data Analysis Using Machine Learning Hand Detection

INTRODUCTION: Microanastomosis is a complex surgical technique that demands delicate and coordinated hand movement. Advances in machine learning technology have enabled new methods to study human motion. Convolutional neural networks are algorithms specialized in visual tasks that enable discrete object detection and tracking including the hands of a surgeon. METHODS: A machine learning hand detector was developed using a pre-trained convolutional neural network capable of tracking 21 hand landmarks without sensors attached. En-to-side microanastomosis was simulated using 2 mm polyvinyl-alcohol vessels and 10/0 micro-suture and recorded with a video camera capturing hand movement in the surgical field. Participants (n = 4) were expert cerebrovascular surgeons with significant operative experience in cerebral bypass surgery. Calculation of motion of 14 hand joint angles was performed using a vectorization method from landmark detections. Time series analysis was performed to calculate the absolute angular motion for each joint of the hand. RESULTS: Four minutes of simulation for each surgeon yielded a total of 784.294 angular detections (14 hand joints total). For the dominant hand, the mean (SD) absolute angular motion was expert 1: 178.2 rad (98.3); expert 2: 261.8 rad (164.3); expert 3: 449.9 rad (307); expert 4: 380.1 rad (198.1).For the non-dominant hand, the mean (SD) absolute angular motion was expert 1: 132.9 rad (64.6), expert 2: 149.2 rad (72.3); expert 3: 389.2 rad (243); expert 4: 162.6 rad (87.2). CONCLUSIONS: A machine learning hand detector system can calculate hand joint motion during microvascular anastomosis. The absolute angular motion is a quantitative performance metric providing information about hand stability during microanastomosis simulation. Such studies using powerful artificial intelligence detection algorithms may have educational potential in the assessment of complex and delicate surgical skills and supervision of surgical training.

AIxSuture: vision-based assessment of open suturing skills

PurposeEfficient and precise surgical skills are essential in ensuring positive patient outcomes. By continuously providing real-time, data driven, and objective evaluation of surgical performance, automated skill assessment has the potential to greatly improve surgical skill training. Whereas machine learning-based surgical skill assessment is gaining traction for minimally invasive techniques, this cannot be said for open surgery skills. Open surgery generally has more degrees of freedom when compared to minimally invasive surgery, making it more difficult to interpret. In this paper, we present novel approaches for skill assessment for open surgery skills.MethodsWe analyzed a novel video dataset for open suturing training. We provide a detailed analysis of the dataset and define evaluation guidelines, using state of the art deep learning models. Furthermore, we present novel benchmarking results for surgical skill assessment in open suturing. The models are trained to classify a video into three skill levels based on the global rating score. To obtain initial results for video-based surgical skill classification, we benchmarked a temporal segment network with both an I3D and a Video Swin backbone on this dataset.ResultsThe dataset is composed of 314 videos of approximately five minutes each. Model benchmarking results are an accuracy and F1 score of up to 75 and 72%, respectively. This is similar to the performance achieved by the individual raters, regarding inter-rater agreement and rater variability. We present the first end-to-end trained approach for skill assessment for open surgery training.ConclusionWe provide a thorough analysis of a new dataset as well as novel benchmarking results for surgical skill assessment. This opens the doors to new advances in skill assessment by enabling video-based skill assessment for classic surgical techniques with the potential to improve the surgical outcome of patients.

Correcting for Rater Effects in Operating Room Surgical Skills Assessment.

To estimate and adjust for rater effects in operating room surgical skills assessment performed using a structured rating scale for nasal septoplasty. We analyzed survey responses from attending surgeons (raters) who supervised residents and fellows (trainees) performing nasal septoplasty in a prospective cohort study. We fit a structural equation model with the rubric item scores regressed on a latent component of skill and then fit a second model including the rating surgeon as a random effect to model a rater-effects-adjusted latent surgical skill. We validated this model against conventional measures including the level of expertise and post-graduation year (PGY) commensurate with the trainee's performance, the actual PGY of the trainee, and whether the surgical goals were achieved. Our dataset included 188 assessments by 7 raters and 41 trainees. The model with one latent construct for surgical skill and the rater as a random effect was the best. Rubric scores depended on how severe or lenient the rater was, sometimes almost as much as they depended on trainee skill. Rater-adjusted latent skill scores increased with attending-estimated skill levels and PGY of trainees, increased with the actual PGY, and appeared constant over different levels of achievement of surgical goals. Our work provides a method to obtain rater effect adjusted surgical skill assessments in the operating room using structured rating scales. Our method allows for the creation of standardized (i.e., rater-effects-adjusted) quantitative surgical skill benchmarks using national-level databases on trainee assessments. N/A Laryngoscope, 2024.

Decoding surgical skill: an objective and efficient algorithm for surgical skill classification based on surgical gesture features -experimental studies.

Various surgical skills lead to differences in patient outcomes and identifying poorly skilled surgeons with constructive feedback contributes to surgical quality improvement. The aim of the study was to develop an algorithm for evaluating surgical skills in laparoscopic cholecystectomy based on the features of elementary functional surgical gestures (Surgestures). Seventy-five laparoscopic cholecystectomy videos were collected from 33 surgeons in five hospitals. The phase of mobilization hepatocystic triangle and gallbladder dissection from the liver bed of each video were annotated with 14 Surgestures. The videos were grouped into competent and incompetent based on the quantiles of modified global operative assessment of laparoscopic skills (mGOALS). Surgeon-related information, clinical data, and intraoperative events were analyzed. Sixty-three Surgesture features were extracted to develop the surgical skill classification algorithm. The area under the receiver operating characteristic curve of the classification and the top features were evaluated. Correlation analysis revealed that most perioperative factors had no significant correlation with mGOALS scores. The incompetent group has a higher probability of cholecystic vascular injury compared to the competent group (30.8 vs 6.1%, P =0.004). The competent group demonstrated fewer inefficient Surgestures, lower shift frequency, and a larger dissection-exposure ratio of Surgestures during the procedure. The area under the receiver operating characteristic curve of the classification algorithm achieved 0.866. Different Surgesture features contributed variably to overall performance and specific skill items. The computer algorithm accurately classified surgeons with different skill levels using objective Surgesture features, adding insight into designing automatic laparoscopic surgical skill assessment tools with technical feedback.

Computer vision in colorectal surgery: Current status and future challenges

The shift from open to endoscopic surgeries, including laparoscopic and robot-assisted surgeries, has enabled the storage of a large number of high-quality intraoperative videos. Endoscopic surgery is highly compatible with artificial intelligence (AI), especially deep-learning-based computer vision, as it provides easy access to videos that form the basis of image analysis. Following the self-learning process, wherein surgeons gain an understanding of surgery by repeatedly watching intraoperative videos, numerous efforts have been made to build AI models for surgery using AI input and analyzing a vast amount of information from intraoperative videos. However, whether AI's understanding of surgery increases sufficiently and becomes useful in daily surgical practice remains unclear. Therefore, this review aims to discuss the current status and future challenges of using AI in surgery, particularly in laparoscopic colorectal surgery, and to explore aspects such as surgical phase or step recognition, navigation and surgical automation, and surgical skill assessment.

Case Curve: A Novel Web-Based Platform and Mobile Phone Application to Evaluate Surgical Competence in Graduate Medical Education.

Competency-based medical education is well established, yet methods to evaluate and document acquisition of surgical skill remain underdeveloped. We describe a novel web-based application for competency-based surgical education at a single neurosurgical department over a 3-year period. We used a web-based application to track procedural and cognitive skills acquisition for neurosurgical residents. This process included self-assessment, resident peer evaluations, evaluations from supervising attending physicians, and blinded video reviews. Direct observation by faculty and video recordings were used to evaluate surgical skill. Cases were subdivided into component skills, which were evaluated using a 5-point scale. The learning curve for each skill was continuously updated and compared with expectations. Progress was reviewed at a monthly surgical skills conference that involved discussion and analysis of recorded surgical performances. During an escalating 3-year pilot from 2019 to 2022, a total of 1078 cases in 39 categories were accrued by 17 resident physicians with 10 neurosurgical faculty who participated as evaluators. A total of 16 251 skill performances in 110 categories were evaluated. The most evaluated skills were those that were common to several types of procedures, such as positioning, hemostasis, and wound closure. The concordance between attending evaluations and resident self-evaluations was 76%, with residents underestimating their performance in 17% of evaluations and overestimating in 7%. We developed a method for evaluating and tracking surgical resident skill performance with an application that provides timely and actionable feedback. The data collected from this system could allow more accurate assessments of surgical skills and deeper insights into factors influencing surgical skill acquisition.

Trainee Experience in Levels of Difficulties for Flaps in Plastic Surgery

The competency-based medical education (CBME) approach has introduced medical curricula across the globe, designed to provide flexibility, Responsibility and learner-centredness for medical learners. While traditional surgical skills training in most places has relied on "see one, do one, teach one model", simulation model-based training has been shown to improve surgical interns skills. Safely performing local and regional flaps is often a challenge for surgical residents. Therefore, the aim of this study was to assess the difficulty classification of flaps and establish equivalence between them at each level. A questionnaire and an assessment of surgical skills were sent to all plastic surgery residents and young specialists in Morocco analyzing the theoretical and procedural knowledge of flaps. Skills are assessed using a modified version of an objective structured assessment of the Technical Skills Score.

Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

Laparoscopic surgical skill assessment and machine learning are often inaccessible to low-and-middle-income countries (LMIC). Our team developed a low-cost laparoscopic training system to teach and assess psychomotor skills required in laparoscopic salpingostomy in LMICs. We performed video review using AI to assess global surgical techniques. The objective of this study was to assess the validity of artificial intelligence (AI) generated scoring measures of laparoscopic simulation videos by comparing the accuracy of AI results to human-generated scores. Seventy-four surgical simulation videos were collected and graded by human participants using a modified OSATS (Objective Structured Assessment of Technical Skills). The videos were then analyzed via AI using 3 different time and distance-based calculations of the laparoscopic instruments including path length, dimensionless jerk, and standard deviation of tool position. Predicted scores were generated using 5-fold cross validation and K-Nearest-Neighbors to train classifiers. Surgical novices and experts from a variety of hospitals in Ethiopia, Cameroon, Kenya, and the United States contributed 74 laparoscopic salpingostomy simulation videos. Complete accuracy of AI compared to human assessment ranged from 65-77%. There were no statistical differences in rank mean scores for 3 domains, Flow of Operation, Respect for Tissue, and Economy of Motion, while there were significant differences in ratings for Instrument Handling, Overall Performance, and the total summed score of all 5 domains (Summed). Estimated effect sizes were all less than 0.11, indicating very small practical effect. Estimated intraclass correlation coefficient (ICC) of Summed was 0.72 indicating moderate correlation between AI and Human scores. Video review using AI technology of global characteristics was similar to that of human review in our laparoscopic training system. Machine learning may help fill an educational gap in LMICs where direct apprenticeship may not be feasible.

Evaluation of anesthetic skills acquisition in pre-graduate veterinary students with different grades of anesthetic experience using veterinary simulation exercises.

Anesthetic skills are usually learned through continuous supervision by experienced trainers who observe, advise and challenge students. Current educational techniques rely less on live animal training and include the use of simulations and models for teaching and assessment of surgical and anesthetic skills. To evaluate the development of anesthetic skills of veterinary students having different levels of previous experience using simulation. An additional aim was to evaluate the impact of the simulation training on students with no anesthesia experience. Single group periinterventional and postinterventional study. Initial and final anesthesia simulation training recording were obtained from 53 randomly selected veterinary students. Seven faculty members blinded to previous student anesthesia experience reviewed the simulation recording and scored student performance using a rubric, results were recorded and analyzed. All students participating in an anesthesia and surgery course reached higher proficiency levels on fundamental anesthesia skills regardless of their previous amount of experience with anesthesia. Simulation based learning positively influenced the final score in veterinary students having no previous anesthesia training, suggesting that it is possible for veterinary students to achieve a level of competence in anesthesia skills with simulation-based training. Sample size, group simulation, multiple reviewers bias. Students having no experience with clinical anesthesia demonstrated remarkable improvement in their skills, achieving a score that was similar to students having extensive prior clinical anesthesia experience. Despite this clear improvement students having no prior clinical anesthesia experience required more time to complete all anesthesia tasks and may require more training sessions to acquire the speed demonstrated by peers who had significant prior clinical anesthesia experience. Overall, all participants reached a higher proficiency level performing fundamental anesthesia skills at the end of the course.

Perception of endoscopic endonasal surgery training by French otolaryngology residents: A STROBE analysis of expectations

ObjectivesTo analyze the perception of endoscopic endonasal surgery training by French otolaryngology residents. Material and methodsA multicenter retrospective observational study was conducted from March to April 2023. Otolaryngology residents from 7 French regions filled out a 27-item questionnaire on their training in endoscopic endonasal surgery. ResultsOut of 283 residents contacted, 126 (45%) filled out the questionnaire. Seventy-four (59%) had already partially or completely performed the surgeries specified in their diploma course. The level of mastery of the main steps of endonasal surgery and the level of autonomy were higher in the consolidation stage group than in the basic and advanced stages. Seventy residents (56%) felt they had gaps in their level of training. To improve training, 94 (75%) wished for more dissection sessions, surgical skills assessments each semester and simulation sessions. Eighty-nine (71%) felt they needed to find their own teaching aids and other methods to complete their training. One hundred and thirteen (90%) felt that the lack of funding available for congresses and training courses was detrimental. ConclusionThis study highlighted the overall satisfaction of residents with their training in endoscopic endonasal surgery. They expressed a desire for more dissection, simulation and evaluation.

Learning Skill Characteristics From Manipulations.

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

The application of objective clinical human reliability analysis (OCHRA) in the assessment of basic robotic surgical skills.

Using a validated, objective, and standardised assessment tool to assess progression and competency is essential for basic robotic surgical training programmes. Objective clinical human reliability analysis (OCHRA) is an error-based assessment tool that provides in-depth analysis of individual technical errors. We conducted a feasibility study to assess the concurrent validity and reliability of OCHRA when applied to basic, generic robotic technical skills assessment. Selected basic robotic surgical skill tasks, in virtual reality (VR) and dry lab equivalent, were performed by novice robotic surgeons during an intensive 5-day robotic surgical skills course on da Vinci® X and Xi surgical systems. For each task, we described a hierarchical task analysis. Our developed robotic surgical-specific OCHRA methodology was applied to error events in recorded videos with a standardised definition. Statistical analysis to assess concurrent validity with existing tools and inter-rater reliability were performed. OCHRA methodology was applied to 272 basic robotic surgical skills tasks performed by 20 novice robotic surgeons. Performance scores improved from the start of the course to the end using all three assessment tools; Global Evaluative Assessment of Robotic Skills (GEARS) [VR: t(19) = - 9.33, p < 0.001] [dry lab: t(19) = - 10.17, p < 0.001], OCHRA [VR: t(19) = 6.33, p < 0.001] [dry lab: t(19) = 10.69, p < 0.001] and automated VR [VR: t(19) = - 8.26, p < 0.001]. Correlation analysis, for OCHRA compared to GEARS and automated VR scores, shows a significant and strong inverse correlation in every VR and dry lab task; OCHRA vs GEARS [VR: mean r = - 0.78, p < 0.001] [dry lab: mean r = - 0.82, p < 0.001] and OCHRA vs automated VR [VR: mean r = - 0.77, p < 0.001]. There is very strong and significant inter-rater reliability between two independent reviewers (r = 0.926, p < 0.001). OCHRA methodology provides a detailed error analysis tool in basic robotic surgical skills with high reliability and concurrent validity with existing tools. OCHRA requires further evaluation in more advanced robotic surgical procedures.

Surgical Phase Duration in Robot-Assisted Partial Nephrectomy: A Surgical Data Science Exploration for Clinical Relevance.

(1) Background: Surgical phases form the basic building blocks for surgical skill assessment, feedback, and teaching. The phase duration itself and its correlation with clinical parameters at diagnosis have not yet been investigated. Novel commercial platforms provide phase indications but have not been assessed for accuracy yet. (2) Methods: We assessed 100 robot-assisted partial nephrectomy videos for phase durations based on previously defined proficiency metrics. We developed an annotation framework and subsequently compared our annotations to an existing commercial solution (Touch Surgery, Medtronic™). We subsequently explored clinical correlations between phase durations and parameters derived from diagnosis and treatment. (3) Results: An objective and uniform phase assessment requires precise definitions derived from an iterative revision process. A comparison to a commercial solution shows large differences in definitions across phases. BMI and the duration of renal tumor identification are positively correlated, as are tumor complexity and both tumor excision and renorrhaphy duration. (4) Conclusions: The surgical phase duration can be correlated with certain clinical outcomes. Further research should investigate whether the retrieved correlations are also clinically meaningful. This requires an increase in dataset sizes and facilitation through intelligent computer vision algorithms. Commercial platforms can facilitate this dataset expansion and help unlock the full potential, provided that the phase annotation details are disclosed.

A Deep Learning Approach to Classify Surgical Skill in Microsurgery Using Force Data from a Novel Sensorised Surgical Glove.

Microsurgery serves as the foundation for numerous operative procedures. Given its highly technical nature, the assessment of surgical skill becomes an essential component of clinical practice and microsurgery education. The interaction forces between surgical tools and tissues play a pivotal role in surgical success, making them a valuable indicator of surgical skill. In this study, we employ six distinct deep learning architectures (LSTM, GRU, Bi-LSTM, CLDNN, TCN, Transformer) specifically designed for the classification of surgical skill levels. We use force data obtained from a novel sensorized surgical glove utilized during a microsurgical task. To enhance the performance of our models, we propose six data augmentation techniques. The proposed frameworks are accompanied by a comprehensive analysis, both quantitative and qualitative, including experiments conducted with two cross-validation schemes and interpretable visualizations of the network's decision-making process. Our experimental results show that CLDNN and TCN are the top-performing models, achieving impressive accuracy rates of 96.16% and 97.45%, respectively. This not only underscores the effectiveness of our proposed architectures, but also serves as compelling evidence that the force data obtained through the sensorized surgical glove contains valuable information regarding surgical skill.