Evaluation Of Surgical Skills Research Articles

Neuromechanisms of simulation-based arthroscopic skills assessment: a fNIRS study.

The neural mechanisms underlying differences in the performance of simulated arthroscopic skills across various skill levels remain unclear. Our primary objective is to investigate the learning mechanisms of simulated arthroscopic skills using functional near-infrared spectroscopy (fNIRS). We recruited 27 participants, divided into three groups: novices (n = 9), intermediates (n = 9), and experts (n = 9). Participants completed seven arthroscopic tasks on a simulator, including diagnostic navigation, triangulation, grasping stars, diagnostic exploration, meniscectomy, synovial membrane cleaning, and loose body removal. All tasks were videotaped and assessed via the simulator system and the Arthroscopic Surgical Skill Evaluation Tool (ASSET), while cortical activation data were collected using fNIRS. Simulator scores and ASSET scores were analyzed to identify different level of performance of all participants. Brain region activation and functional connectivity (FC) of different types of participants were analyzed from fNIRS data. Both the expert and intermediate groups scored significantly higher than the novice group (p < 0.001). There were significant differences in ASSET scores between experts and intermediates, experts and novices, and intermediates and novices (p = 0.0047, p < 0.0001, p < 0.0001), with the trend being experts > intermediates > novices. The intermediate group exhibited significantly greater activation in the left primary motor cortex (LPMC) and left prefrontal cortex (LPFC) compared to the novice group (p = 0.0152, p = 0.0021). Compared to experts, the intermediate group demonstrated significantly increased FC between the presupplementary motor area (preSMA) and the right prefrontal cortex (RPFC; p < 0.001). Additionally, the intermediate group showed significantly increased FC between the preSMA and LPFC, RPFC and LPFC, and LPMC and LPFC compared to novices (p = 0.0077, p = 0.0285, p = 0.0446). Cortical activation and functional connectivity reveal varying levels of activation intensity in the PFC, PMC, and preSMA among novices, intermediates, and experts. The intermediate group exhibited the highest activation intensity.

Hypospadias Reconstruction Training: Development of an Ex-Vivo Model for Objective Evaluation of Surgical Skills

Hypospadias Reconstruction Training: Development of an Ex-Vivo Model for Objective Evaluation of Surgical Skills

Classification of subtask types and skill levels in robot-assisted surgery using EEG, eye-tracking, and machine learning

BackgroundObjective and standardized evaluation of surgical skills in robot-assisted surgery (RAS) holds critical importance for both surgical education and patient safety. This study introduces machine learning (ML) techniques using features derived from electroencephalogram (EEG) and eye-tracking data to identify surgical subtasks and classify skill levels.MethodThe efficacy of this approach was assessed using a comprehensive dataset encompassing nine distinct classes, each representing a unique combination of three surgical subtasks executed by surgeons while performing operations on pigs. Four ML models, logistic regression, random forest, gradient boosting, and extreme gradient boosting (XGB) were used for multi-class classification. To develop the models, 20% of data samples were randomly allocated to a test set, with the remaining 80% used for training and validation. Hyperparameters were optimized through grid search, using fivefold stratified cross-validation repeated five times. Model reliability was ensured by performing train-test split over 30 iterations, with average measurements reported.ResultsThe findings revealed that the proposed approach outperformed existing methods for classifying RAS subtasks and skills; the XGB and random forest models yielded high accuracy rates (88.49% and 88.56%, respectively) that were not significantly different (two-sample t-test; P-value = 0.9).ConclusionThese results underscore the potential of ML models to augment the objectivity and precision of RAS subtask and skill evaluation. Future research should consider exploring ways to optimize these models, particularly focusing on the classes identified as challenging in this study. Ultimately, this study marks a significant step towards a more refined, objective, and standardized approach to RAS training and competency assessment.

Can a Low-Fidelity Arthroscopic Simulator Improve Technical Expertise in Performing Temporomandibular Joint Arthroscopy?

BackgroundChallenges in temporomandibular joint (TMJ) arthroscopy training include the cost of operating room time, surgical risks, and the accessibility of cadavers and high-fidelity simulators. A low-fidelity simulator (LFS) was developed for initial TMJ arthroscopy training. PurposeThe aim of this study was to evaluate improvement in TMJ arthroscopic skills after training with an LFS using the Arthroscopic Surgical Skill Evaluation Tool (ASSET) global rating scale. Study design, setting, sampleA prospective randomized controlled study was conducted. Subjects included Oral and Maxillofacial Surgery residents in post-graduate year (PGY) 1–5 at Thomas Jefferson University. Chief residents actively performing TMJ arthroscopy were excluded. Predictor variableThe predictor variable was participation in LFS training. Subjects were randomly assigned to the LFS training or no LFS training group. Main outcome variablesThe primary outcome variable was simulated surgical skill measured by total ASSET score (maximum score of 35). The secondary outcome variables were simulated surgical skill measured by each ASSET domain score and time to completion (TTC). CovariatesThe covariates included sex, age, handedness, PGY, endoscopic experience, open TMJ surgery experience, and experience with musical instruments and sports. AnalysesData analyses included paired T-tests to determine changes in outcome variables after TMJ arthroscopy training between experimental groups, and ANOVA and chi-square tests to identify associations between covariates. Inter-rater reliability of the blinded examiners was evaluated using Cronbach’s alpha correlation. A P-value < .05 was considered significant. ResultsThe sample was composed of 10 residents: 5 (100%) males in the no LFS group, 3 (60%) males and 2 (40%) females in the LFS group (P = .4). There was no significant difference between the groups in pre-TMJ arthroscopy training ASSET scores (P = .3). After training, the total ASSET score improved by 3.40 ± 6.87 in the no LFS group and by 6.27 ± 2.68 in the LFS group (P = .03). Conclusions and relevanceStudy results showed that low-fidelity simulation can improve fundamental arthroscopic skills. Future research should be conducted on a larger scale to further validate this model and show the transfer of skill improvements to cadaver and live patient settings.

A surgical instrument motion measurement system for skill evaluation in practical laparoscopic surgery training.

This study developed and validated a surgical instrument motion measurement system for skill evaluation during practical laparoscopic surgery training. Owing to the various advantages of laparoscopic surgery including minimal invasiveness, this technique has been widely used. However, expert surgeons have insufficient time for providing training to beginners due to the shortage of surgeons and limited working hours. Skill transfer efficiency has to be improved for which there is an urgent need to develop objective surgical skill evaluation methods. Therefore, a simple motion capture-based surgical instrument motion measurement system that could be easily installed in an operating room for skill assessment during practical surgical training was developed. The tip positions and orientations of the instruments were calculated based on the marker positions attached to the root of the instrument. Because the patterns of these markers are individual, this system can track multiple instruments simultaneously and detect exchanges. However due to the many obstacles in the operating room, the measurement data included noise and outliers. In this study, the effect of this decrease in measurement accuracy on feature calculation was determined. Accuracy verification experiments were conducted during wet-lab training to demonstrate the capability of this system to measure the motion of surgical instruments with practical accuracy. A surgical training experiment on a cadaver was conducted, and the motions of six surgical instruments were measured in 36 cases of laparoscopic radical nephrectomy. Outlier removal and smoothing methods were also developed and applied to remove the noise and outliers in the obtained data. The questionnaire survey conducted during the experiment confirmed that the measurement system did not interfere with the surgical operation. Thus, the proposed system was capable of making reliable measurements with minimal impact on surgery. The system will facilitate surgical education by enabling the evaluation of skill transfer of surgical skills.

Developing a Training Model for Local Flaps Using Fresh Human Skin Excised During Tummy Tuck Procedures

Introduction: The proficient execution of local flaps poses a challenge for surgical residents. This study aimed to assess a training model for local flaps using fresh human skin excised during tummy tuck procedures. Methods: A questionnaire and a surgical skills evaluation, examining theoretical and procedural knowledge of local flaps, were administered before and after the training. All surgical procedures were performed on a simulation model utilizing freshly excised human skin. Skills were evaluated using a modified version of the Objective Structured Assessment of Technical Skills Score. SPSS version 21 was employed for comparing pre- and post-training results. Results: In the pretraining assessment, residents faced difficulties in flap design accuracy and wound coverage adequacy, indicating a need for surgical training beyond the operating theater. Post-training, there was a significant improvement in procedural skills, as reflected in the modified Objective Structured Assessment of Technical Skills score. The mean cumulative pretraining score was 26.81 ± 5.41, while the posttraining score was 43.59 ± 5.72 (P = 0.008). Theoretical knowledge also significantly improved in the posttraining evaluation, except for the indication of a Z-plasty (P = 0.257). The training model received positive feedback and was deemed highly useful and recommendable. Discussion: The presented model demonstrated improved surgical handling and understanding of tissue rotation under realistic conditions. The simulation model, using fresh human skin, is cost-effective and versatile for a broad range of flap procedures, advocating for its further adoption. Conclusions: The training model significantly enhanced surgical handling and comprehension of tissue rotation, replicating realistic conditions. The cost-effective simulation model, based on fresh human skin, offers versatility for various flap procedures, emphasizing the need for its wider implementation.

Reliability in performance assessment creates a potential application of artificial intelligence in veterinary education: evaluation of suturing skills at a single institution.

To evaluate suturing skills of veterinary students using 3 common performance assessments (PAs) and to compare findings to data obtained by an electromyographic armband. 16 second-year veterinary students. Students performed 4 suturing tasks on synthetic tissue models 1, 3, and 5 weeks after a surgical skills course. Digital videos were scored by 4 expert surgeons using 3 PAs (an Objective Structured Clinical Examination [OSCE]- style surgical binary checklist, an Objective Structured Assessment of Technical Skill [OSATS] checklist, and a surgical Global Rating Scale [GRS]). Surface electromyography (sEMG) data collected from the dominant forearm were input to machine learning algorithms. Performance assessment scores were compared between experts and correlated to task completion times and sEMG data. Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC). Inter-rater agreement was calculated using percent agreement with varying levels of tolerance. Reliability was moderate for the OSCE and OSATS checklists and poor for the GRS. Agreement was achieved for the checklists when moderate tolerance was applied but remained poor for the GRS. sEMG signals did not correlate well with checklist scores or task times, but features extracted from signals permitted task differentiation by routine statistical comparison and correct task classification using machine learning algorithms. Reliability and agreement of an OSCE-style checklist, OSATS checklist, and surgical GRS assessment were insufficient to characterize suturing skills of veterinary students. To avoid subjectivity associated with PA by raters, further study of kinematics and EMG data is warranted in the surgical skills evaluation of veterinary students.

Current update on multimodal analgesia and nonopiate surgical pain management

Based on the Competency Assessment Tool, herein we developed an assessment instrument suitable to evaluate the implantation of central intravenous devices.Surgical assessment is based mainly on the subjective impressions of the teacher. Based on the “Competency Assessment Tool” (CAT) developed for the evaluation of technical surgical skills in minimally invasive colorectal resection, we designed an assessment tool suitable to evaluate the implantation of central venous access devices performed by junior surgical trainees.Four major assessments during the different steps of the intervention were used in this evaluation. Each of these tasks was divided into four sub-domains according to surgical skill. In addition to the CAT score, the apprentices’ skills were evaluated using a visual assessment that was quantified using an analogue scale (value from 1 to 10). The candidates were classified into junior and senior trainees depending on the number of procedures they had already performed and on their surgical experience.71 procedures were evaluated during the study period. Seven senior trainees conducted 43 procedures and five junior trainees performed 28 interventions. The senior trainees had significantly higher CAT scores than junior candidates, and the scores fluctuated according to surgical experience, usually reaching their peak after 10 procedures.The CAT model is well suited for the assessment of surgical trainees during central venous access device implantation. It enables a close assessment of the learning process and the technical skills of trainees, which helps them improving in a safe, standardized manner.

Evaluation of single-stage vision models for pose estimation of surgical instruments.

Multiple applications in open surgical environments may benefit from adoption of markerless computer vision depending on associated speed and accuracy requirements. The current work evaluates vision models for 6-degree of freedom pose estimation of surgical instruments in RGB scenes. Potential use cases are discussed based on observed performance. Convolutional neural nets were developed with simulated training data for 6-degree of freedom pose estimation of a representative surgical instrument in RGB scenes. Trained models were evaluated with simulated and real-world scenes. Real-world scenes were produced by using a robotic manipulator to procedurally generate a wide range of object poses. CNNs trained in simulation transferred to real-world evaluation scenes with a mild decrease in pose accuracy. Model performance was sensitive to input image resolution and orientation prediction format. The model with highest accuracy demonstrated mean in-plane translation error of 13mm and mean long axis orientation error of 5[Formula: see text] in simulated evaluation scenes. Similar errors of 29mm and 8[Formula: see text] were observed in real-world scenes. 6-DoF pose estimators can predict object pose in RGB scenes with real-time inference speed. Observed pose accuracy suggests that applications such as coarse-grained guidance, surgical skill evaluation, or instrument tracking for tray optimization may benefit from markerless pose estimation.

Effectiveness of the Eyesi Surgical Simulator for ophthalmology trainees: systematic review and meta-analysis

ObjectifLe simulateur chirurgical Eyesi (VRmagic, Mannheim, Allemagne) est le simulateur de réalité virtuelle le plus souvent utilisé en chirurgie ophtalmologique. Si des études ont révélé que des outils de formation fondés sur la simulation améliorent l'acquisition des compétences, aucune revue de synthèse systématique avec méta-analyse n'a encore été réalisée à l’égard du simulateur chirurgical Eyesi. Notre étude visait donc à effectuer une méta-analyse des études qui comparaient le simulateur chirurgical Eyesi à d'autres méthodes d'enseignement des compétences techniques auprès d’étudiants. MéthodesNotre recherche a porté sur 3 bases de données (MEDLINE, Embase et CENTRAL). Les articles inclus comparaient le simulateur chirurgical Eyesi à d'autres formes d'enseignement des compétences techniques auprès d’étudiants et de résidents dans des facultés de médecine. On a réalisé une méta-analyse par paire qui reposait sur l'inverse de la variance à effets aléatoires. Les différences moyennes standardisées (DMS) ont servi de principal paramètre de mesure afin de tenir compte des différences entre les outils d’évaluation des compétences chirurgicales. RésultatsDes 207 citations évaluées, on a retenu 8 études, totalisant 108 étudiants qui ont eu recours au simulateur chirurgical Eyesi et de 123 étudiants qui ont utilisé une autre forme d'apprentissage. L'amélioration des scores de compétences techniques était significativement supérieure dans le groupe Eyesi (3 études; DMS = 2,02; intervalle de confiance [IC] à 95 % : 1,47–2,57; p < 0,001; I2 = 0 %). On a noté une baisse significative du nombre d’étudiants du groupe Eyesi qui ont fait des erreurs techniques après leur formation (5 études : rapport de cotes = 0,43; IC à 95 % : 0,20–0,90; p = 0,03; I2 = 30 %). Aucune différence significative n'a été enregistrée entre les groupes en ce qui a trait au temps consacré pour exécuter une tâche après la formation (3 études; DMS = 1,96; IC à 95 % : –1,96–5,88; p = 0,33; I2 = 97 %). ConclusionsLe simulateur chirurgical Eyesi pourrait améliorer l'acquisition des compétences techniques (faible degré de certitude des données probantes) et réduire le nombre d'erreurs techniques (très faible degré de certitude des données probantes) lors de l'apprentissage de la chirurgie de la cataracte chez des étudiants. Il y a donc lieu de réaliser d'autres études prospectives sur le module Eyesi qui comporteraient un système de notation standardisée.

Craniotomy Simulator with Force Myography and Machine Learning-Based Skills Assessment.

Craniotomy is a fundamental component of neurosurgery that involves the removal of the skull bone flap. Simulation-based training of craniotomy is an efficient method to develop competent skills outside the operating room. Traditionally, an expert surgeon evaluates the surgical skills using rating scales, but this method is subjective, time-consuming, and tedious. Accordingly, the objective of the present study was to develop an anatomically accurate craniotomy simulator with realistic haptic feedback and objective evaluation of surgical skills. A CT scan segmentation-based craniotomy simulator with two bone flaps for drilling task was developed using 3D printed bone matrix material. Force myography (FMG) and machine learning were used to automatically evaluate the surgical skills. Twenty-two neurosurgeons participated in this study, including novices (n = 8), intermediates (n = 8), and experts (n = 6), and they performed the defined drilling experiments. They provided feedback on the effectiveness of the simulator using a Likert scale questionnaire on a scale ranging from 1 to 10. The data acquired from the FMG band was used to classify the surgical expertise into novice, intermediate and expert categories. The study employed naïve Bayes, linear discriminant (LDA), support vector machine (SVM), and decision tree (DT) classifiers with leave one out cross-validation. The neurosurgeons' feedback indicates that the developed simulator was found to be an effective tool to hone drilling skills. In addition, the bone matrix material provided good value in terms of haptic feedback (average score 7.1). For FMG-data-based skills evaluation, we achieved maximum accuracy using the naïve Bayes classifier (90.0 ± 14.8%). DT had a classification accuracy of 86.22 ± 20.8%, LDA had an accuracy of 81.9 ± 23.6%, and SVM had an accuracy of 76.7 ± 32.9%. The findings of this study indicate that materials with comparable biomechanical properties to those of real tissues are more effective for surgical simulation. In addition, force myography and machine learning provide objective and automated assessment of surgical drilling skills.

MP26-09 EYE MOVEMENT BEHAVIOR ASSOCIATES WITH EXPERTISE LEVEL IN ROBOT-ASSISTED SURGERY

You have accessJournal of UrologyCME1 Apr 2023MP26-09 EYE MOVEMENT BEHAVIOR ASSOCIATES WITH EXPERTISE LEVEL IN ROBOT-ASSISTED SURGERY Mehdi Seilanian Toussi, Saeed Shadpour, Kristopher Attwood, Qian Liu, Camille Gutierrez, james L. Mohler, and Somayeh B. Shafiei Mehdi Seilanian ToussiMehdi Seilanian Toussi More articles by this author , Saeed ShadpourSaeed Shadpour More articles by this author , Kristopher AttwoodKristopher Attwood More articles by this author , Qian LiuQian Liu More articles by this author , Camille GutierrezCamille Gutierrez More articles by this author , james L. Mohlerjames L. Mohler More articles by this author , and Somayeh B. ShafieiSomayeh B. Shafiei More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003254.09AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: An objective method for assessing robot-assisted surgery (RAS) skills that can be used in clinical settings has not been established. This study aimed to compare visual metrics at three different skill levels—inexperienced, competent, or experienced— to determine whether eye gaze data differed among RAS surgeons. METHODS: Eleven right-handed participants who performed coagulation subtasks (monopolar or bipolar cautery) during hysterectomy, cystectomy, or nephrectomy on live pigs with the Da Vinci robot had their eye gaze data captured using TobiiPro2 eyeglasses. Visual metrics were extracted using eye gaze data (Table 1). An expert RAS surgeon evaluated each participant's performance and level of expertise after watching operation videos using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. Each metric was compared across the three skill levels—novice, competent, or experienced—using ANOVA. RESULTS: When participants performed coagulation using the dominant hand, the entropy of the right eye’s pupil diameter differed among skill levels (p-value = 0.01). When this subtask was completed with the non-dominant hand, the rates of change of the gaze directions of the left eye in the horizontal and vertical dimensions and the right eye in the vertical dimension differed (p-values = 2x10-6, 6x10-6, and 6x10-4, respectively) across skill levels. Table 2 shows the significance level for the difference in visual metrics between skill level pairs. CONCLUSIONS: Visual metrics differed significantly across RAS skill levels. The ease and efficacy of recording and evaluating visual data without interfering with the surgeon's work suggest a new objective surgical skill evaluation method that could be applied to clinical settings. Source of Funding: National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award number R01EB029398, the Alliance Foundation of Roswell Park Comprehensive Cancer Center, and the National Cancer Institute (NCI) grant P30CA016056 involving the use of Roswell Park Comprehensive Cancer Center’s Laboratory Animal Shared Resource and Applied Technology Laboratory for Advanced Surgery © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e355 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Mehdi Seilanian Toussi More articles by this author Saeed Shadpour More articles by this author Kristopher Attwood More articles by this author Qian Liu More articles by this author Camille Gutierrez More articles by this author james L. Mohler More articles by this author Somayeh B. Shafiei More articles by this author Expand All Advertisement PDF downloadLoading ...

Deep neural network architecture for automated soft surgical skills evaluation using objective structured assessment of technical skills criteria.

Classic methods of surgery skills evaluation tend to classify the surgeon performance in multi-categorical discrete classes. If this classification scheme has proven to be effective, it does not provide in-between evaluation levels. If these intermediate scoring levels were available, they would provide more accurate evaluation of the surgeon trainee. We propose a novel approach to assess surgery skills on a continuous scale ranging from 1 to 5. We show that the proposed approach is flexible enough to be used either for scores of global performance or several sub-scores based on a surgical criteria set called Objective Structured Assessment of Technical Skills (OSATS). We established a combined CNN+BiLSTM architecture to take advantage of both temporal and spatial features of kinematic data. Our experimental validation relies on real-world data obtained from JIGSAWS database. The surgeons are evaluated on three tasks: Knot-Tying, Needle-Passing and Suturing. The proposed framework of neural networks takes as inputs a sequence of 76 kinematic variables and produces an output float score ranging from 1 to 5, reflecting the quality of the performed surgical task. Our proposed model achieves high-quality OSATS scores predictions with means of Spearman correlation coefficients between the predicted outputs and the ground-truth outputs of 0.82, 0.60 and 0.65 for Knot-Tying, Needle-Passing and Suturing, respectively. To our knowledge, we are the first to achieve this regression performance using the OSATS criteria and the JIGSAWS kinematic data. An effective deep learning tool was created for the purpose of surgical skills assessment. It was shown that our method could be a promising surgical skills evaluation tool for surgical training programs.

Virtual reality simulation training improve diagnostic knee arthroscopy and meniscectomy skills: a prospective transfer validity study

PurposeLimited data exist on the actual transfer of skills learned using a virtual reality (VR) simulator for arthroscopy training because studies mainly focused on VR performance improvement and not on transfer to real word (transfer validity). The purpose of this single-blinded, controlled trial was to objectively investigate transfer validity in the context of initial knee arthroscopy training.MethodsFor this study, 36 junior resident orthopaedic surgeons (postgraduate year one and year two) without prior experience in arthroscopic surgery were enrolled to receive standard knee arthroscopy surgery training (NON-VR group) or standard training plus training on a hybrid virtual reality knee arthroscopy simulator (1 h/month) (VR group). At inclusion, all participants completed a questionnaire on their current arthroscopic technical skills. After 6 months of training, both groups performed three exercises that were evaluated independently by two blinded trainers: i) arthroscopic partial meniscectomy on a bench-top knee simulator; ii) supervised diagnostic knee arthroscopy on a cadaveric knee; and iii) supervised knee partial meniscectomy on a cadaveric knee. Training level was determined with the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.ResultsOverall, performance (ASSET scores) was better in the VR group than NON-VR group (difference in the global scores: p < 0.001, in bench-top meniscectomy scores: p = 0.03, in diagnostic knee arthroscopy on a cadaveric knee scores: p = 0.04, and in partial meniscectomy on a cadaveric knee scores: p = 0.02). Subgroup analysis by postgraduate year showed that the year-one NON-VR subgroup performed worse than the other subgroups, regardless of the exercise.ConclusionThis study showed the transferability of the technical skills acquired by novice residents on a hybrid virtual reality simulator to the bench-top and cadaveric models. Surgical skill acquired with a VR arthroscopy surgical simulator might safely improve arthroscopy competences in the operating room, also helping to standardise resident training and follow their progress.Level of evidence2

An investigation into gender bias in the evaluation of orthopedic trainee arthroscopic skills.

Women surgeons receive lower compensation, hold fewer academic positions, and hold fewer leadership positions than men, particularly in orthopedic surgery. Gender bias at the trainee level has been demonstrated in various surgical subspecialties, but there is a lack of information on gender bias within the orthopedic training environment. This study investigated whether implicit gender bias is present in the subjective evaluation of orthopedic trainee arthroscopic skills. After institutional review board approval, a web-based survey was sent to American Shoulder and Elbow Surgeons (ASES) members via the society's email listserve. Study participants were informed that the study was being done to develop a systematic evaluation method for trainees. The survey randomized participants to view and evaluate a prefellowship and a postfellowship video of "Rachel" (she/her) or "Charles" (he/him) performing a 15-point diagnostic shoulder arthroscopy. The videos for Rachel and Charles were identical except for the pronouns used in the video. Participants evaluated the trainee's skill level using questions from the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Blinded and deidentified additional comments regarding the trainee's skill were classified as positive, negative, or neutral. Statistical analyses were used to compare scores and comments between Rachel and Charles. Of 1115 active ASES members, 181 ASES members started the survey and 106 watched both videos and were included in the analysis. Of the 106 participants completing the survey, 96 (91%) were men and 10 (9%) were women with a median (interquartile range) age of 44 (38-51). A teaching role was reported by 84 of 106 participants (79%). There was no significant difference between prefellowship scores (P=.87) or between postfellowship scores (P=.84) for the woman and man fellow. The numbers of comments classified as positive, negative, or neutral were not significantly different between the man and woman fellow (P=.19). Participants in teaching roles gave significantly lower scores to both fellows at both time points (P=.04), and participants who had fellow trainees were more likely to give negative comments to both fellows (P=.02). Trainee gender did not influence the ratings and comments participants gave for trainee arthroscopic skills, suggesting that gender bias may not play a major role in the evaluation of arthroscopic skill during orthopedic training.

Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgery.

Surgical activity recognition and prediction can help provide important context in many Robot-Assisted Surgery (RAS) applications, for example, surgical progress monitoring and estimation, surgical skill evaluation, and shared control strategies during teleoperation. Transformer models were first developed for Natural Language Processing (NLP) to model word sequences and soon the method gained popularity for general sequence modeling tasks. In this paper, we propose the novel use of a Transformer model for three tasks: gesture recognition, gesture prediction, and trajectory prediction during RAS. We modify the original Transformer architecture to be able to generate the current gesture sequence, future gesture sequence, and future trajectory sequence estimations using only the current kinematic data of the surgical robot end-effectors. We evaluate our proposed models on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) and use Leave-One-User-Out (LOUO) cross validation to ensure generalizability of our results. Our models achieve up to 89.3% gesture recognition accuracy, 84.6% gesture prediction accuracy (1 second ahead) and 2.71mm trajectory prediction error (1 second ahead). Our models are comparable to and able to outperform state-of-the-art methods while using only the kinematic data channel. This approach can enabling near-real time surgical activity recognition and prediction.

Evaluation of surgical skill among otology residents during cadaveric temporal bone dissection using an objective assessment tool

BackgroundThe acquisition and refinement of technical skills by the surgical residents are central to surgical teaching; hence, there is increasing interest in the objective assessment of surgical competence. In the field of otology, as of now, there are limited studies for the assessment of surgical competence, also, various subjective methods are being used to assess this vital aspect of training. This study aimed to validate and use an objective assessment tool for the valuation of surgical skills in a tertiary care teaching institute in the Indian subcontinent. MethodsSurgical competence of the residents in temporal bone dissection was assessed on a Likert scale by using a grading scale developed by Wan et al., after obtaining necessary permissions. ResultsJunior residents in ENT who had completed one year of residency in otolaryngology were asked to perform temporal bone surgery (cortical mastoidectomy) and were marked according to the proforma by two senior experienced otologists. The Cronbach's alpha value was .86 which is an indicator of good technical validity. ConclusionsIn our study, we have validated a grading scoring scale used by Wan et al. to objectively assess the ability of an otology resident to perform temporal bone surgery. It is recommended for use in Indian scenario due to its good technical validity. The use of a Likert scale to individually rate each competency makes the evaluation precise and simple.

An Evaluation of Peer-Rated Surgical Skill and its Relationship With Detrusor Muscle Sampling in Transurethral Resection of Bladder Tumor

ObjectiveTo assess the reliability of peer-review of TURBT videos as a means to evaluate surgeon skill and its relationship to detrusor sampling. MethodsUrologists from an academic health system submitted TURBT videos in 2019. Ten blinded peers evaluated each surgeon's performance using a 10-item scoring instrument to quantify surgeon skill. Normalized composite skill scores for each surgeon were calculated using peer ratings. For surgeons submitting videos, we retrospectively reviewed all TURBT pathology results (2018-2019) to assess surgeon-specific detrusor sampling. A hierarchical logistic regression model was fit to evaluate the association between skill and detrusor sampling, adjusting for patient and surgeon factors. ResultsSurgeon skill scores and detrusor sampling rates were determined for 13 surgeons performing 245 TURBTs. Skill scores varied from -6.0 to 5.1 [mean: 0; standard deviation (SD): 2.40]. Muscle was sampled in 72% of cases, varying considerably across surgeons (mean: 64.5%; SD: 30.7%). Among 8 surgeons performing >5 TURBTs during the study period, adjusted detrusor sampling rate was associated with sending separate deep specimens (odds ratio [OR]: 1.97; 95% confidence interval [CI]: 1.02-3.81, P = .045) but not skill (OR: 0.81; 95% CI: 0.57-1.17, P = .191). ConclusionSurgeon skill was not associated with detrusor sampling, suggesting there may be other drivers of variability of detrusor sampling in TURBT.

A three-dimensional (3D) printed simulator as a feasible assessment tool for evaluating hip arthroscopy skills

PurposeThe aims of this study were (1) to develop a three-dimensional (3D) printed simulator that facilitates the simulation of surgical skills for portal placement, intra-articular identification of anatomical structures and arthroscope navigation for hip arthroscopy and (2) to concurrently examine the feasibility of using this simulator as an assessment tool to evaluate trainees’ surgical competencies.MethodsA simulator was developed using a combination of medical imaging, computer-aided design, and 3D printing. A cross-sectional study was conducted with 29 participants divided into 3 subgroups (novice, intermediate and experienced). All participants performed related skills on the simulator, and their performance was evaluated using different assessment parameters. The participants’ qualitative feedback regarding the simulator was also collected. The data collated from each group of participants were subsequently compared.ResultsSignificant differences were observed between the three subgroups of participants with regard to the total checklist score (F2,26 = 11.3), total Arthroscopic Surgical Skill Evaluation score (F2,26 = 92.1), overall final global rating scale score (F2,26 = 49), number of times the participants used fluoroscopy (F2,26 = 7.4), and task completion times (F2,26 = 23.5). The participants’ performance in the simulated operation was correlated with their prior clinical experience. There was mainly positive feedback with regard to the fidelity and utility of the simulator in relation to the surgeons’ prior clinical experience.ConclusionsThis study demonstrated that a reliable hip arthroscopic simulator can be developed for use by orthopedic surgeons to evaluate their hip arthroscopic skills before performing actual surgical operations.Level of evidenceLevel III.

Beyond Efficiency: Surface Electromyography Enables Further Insights into the Surgical Movements of Urologists

Introduction: Surgical skill evaluation while performing minimally invasive surgeries is a highly complex task. It is important to objectively assess an individual's technical skills throughout surgical training to monitor progress and to intervene when skills are not commensurate with the year of training. The miniaturization of wireless wearable platforms integrated with sensor technology has made it possible to noninvasively assess muscle activations and movement variability during performance of minimally invasive surgical tasks. Our objective was to use electromyography (EMG) to deconstruct the motions of a surgeon during robotic suturing (RS) and distinguish quantifiable movements that characterize the skill of an experienced expert urologic surgeon from trainees. Methods: Three skill groups of participants, novice (n = 11), intermediate (n = 12), and expert (n = 3), were enrolled in the study. A total of 12 wireless wearable sensors consisting of surface EMGs and accelerometers were placed along upper extremity muscles to assess muscle activations and movement variability, respectively. Participants then performed a RS task. Results: EMG-based parameters, total time, dominant frequency, and cumulative muscular workload, were significantly different across the three skill groups. We also found nonlinear movement variability parameters such as correlation dimension, Lyapunov exponent trended differently across the three skill groups. Conclusions: These findings suggest that economy of motion variables and nonlinear movement variabilities are affected by surgical experience level. Wearable sensor signal analysis could make it possible to objectively evaluate surgical skill level periodically throughout the residency training experience. Clinical Trial Registration Number: HS# 2018-4407.