Assessment Of Surgical Skills Research Articles

Development and validation of an objective scoring tool to evaluate surgical dissection: Dissection Assessment for Robotic Technique (DART).

Evaluation of surgical competency has important implications for training new surgeons, accreditation, and improving patient outcomes. A method to specifically evaluate dissection performance does not yet exist. This project aimed to design a tool to assess surgical dissection quality. Delphi method was used to validate structure and content of the dissection evaluation. A multi-institutional and multi-disciplinary panel of 14 expert surgeons systematically evaluated each element of the dissection tool. Ten blinded reviewers evaluated 46 de-identified videos of pelvic lymph node and seminal vesicle dissections during the robot-assisted radical prostatectomy. Inter-rater variability was calculated using prevalence-adjusted and bias-adjusted kappa. The area under the curve from receiver operating characteristic curve was used to assess discrimination power for overall DART scores as well as domains in discriminating trainees (≤100 robotic cases) from experts (>100). Four rounds of Delphi method achieved language and content validity in 27/28 elements. Use of 3- or 5-point scale remained contested; thus, both scales were evaluated during validation. The 3-point scale showed improved kappa for each domain. Experts demonstrated significantly greater total scores on both scales (3-point, p< 0.001; 5-point, p< 0.001). The ability to distinguish experience was equivalent for total score on both scales (3-point AUC= 0.92, CI 0.82-1.00, 5-point AUC= 0.92, CI 0.83-1.00). We present the development and validation of Dissection Assessment for Robotic Technique (DART), an objective and reproducible 3-point surgical assessment to evaluate tissue dissection. DART can effectively differentiate levels of surgeon experience and can be used in multiple surgical steps.

A new basic thoracoscopic surgical skill training and assessment system using automatic scoring techniques.

We report a new thoracoscopic surgical skill training and assessment system with automatic scoring techniques, the Huaxi Intelligent Thoracoscopic Skill Training and Assessment (HITSTA) system. We also evaluated the discriminative ability of this system compared to our conventional scoring method at our institution. We retrospectively collected training data of thoracic board-certified thoracic surgeons at West China Hospital, Sichuan University from January 1, 2018 to January 1, 2019. Surgeons were assessed by HITSTA system and human examiners simultaneously. Total scores were summed from 3 tasks (grasping with delivery, pattern cutting, and suture with knot). Bland-Altman analysis was used to test agreement of scores made by HITSTA system (automatic scoring) and human examiners (manual scoring). Differentiation ability was also compared between the two scoring methods. Thirty-nine surgeons were recruited. Scores made by HITSTA system and human examiners were not consistent. For suture with knot, automatic scoring method could detect the score differences between different training status (trained: 26.92 ± 12.04, untrained: 19.85 ± 11.12; p = 0.026) and training duration (< 10h: 20.67 ± 15.23, ≥ 10h: 31.92 ± 5.56; p = 0.003). For total scores, automatic scoring approach could discriminate between different training status (trained: 71.90 ± 12.63; untrained: 61.41 ± 13.87; p = 0.016) and training duration (< 10h: 65.23 ± 15.31; ≥ 10h 77.23 ± 6.94; p = 0.046). HITSTA system could discriminate the different levels of thoracoscopic surgical skills better than the traditional manual scoring method. Larger prospective studies are warranted to validate the differentiation ability of HITSTA system.

Correlation of surgical case volume and fellowship training with performance on simulated procedural tasks

High-volume and fellowship-trained surgeons have superior outcomes. However, in gynecology, a large proportion of cases are performed by low-volume surgeons. Simulation has been shown to be useful in assessing surgical skill and may be a useful tool in hospital credentialing and maintenance of privileges. To determine the correlation between a surgical case volume and fellowship training with performance on simulated procedural tasks. A total of 108 obstetricians and gynecologists with laparoscopic privileges at 2 academic institutions completed a pre-test survey and performed 3 tasks on the LapSim laparoscopic virtual reality simulator. The pre-test survey inquired about the monthly laparoscopic case volume and prior training. Simulations included a basic skills task (peg transfer) followed by a procedural task (salpingectomy) of 2 difficulty levels (low and moderate). Spearman correlation and Wilcoxon tests were used to determine correlations between the survey responses and performance metrics. Participants included 67 generalists (62%) and 41 fellowship-trained specialists (38%). There was an observed weak correlation among surgical volume (more than 6 cases per month), time to completion, and the amount of blood loss when performing the low-difficulty level salpingectomy (r=-0.32, P=.0007 and r=-0.29, P=.002, respectively). The economy of movement (instrument path length) was correlated to high surgical volume (r=-0.35, P=.0002). Compared with generalists, surgeons with fellowship training performed tasks faster (410.8 seconds [interquartile range, 309.7-595.2]) vs 530.2 seconds (interquartile range, 406.2-605.0; P=.0009), more efficiently at 6.1 m (interquartile range, 4.8-7.3) vs 8.1 m (interquartile range, 5.8-10.7; P=.0003), and with less blood loss at 21.7 mL (interquartile range, 11.8-37.7) vs 42.9 mL (interquartile range, 18.1-70.6; P=.002). Regarding the case volume and fellowship background, there was no difference in ovarian diathermy damage. In addition, there was no difference among most performance parameters for the peg transfer task and the moderate-difficulty salpingectomy procedure. Surgical experience obtained through higher case volume and fellowship training correlate with higher performance scores during simulated procedural tasks. In a previous study, we found a similar correlation with simulated basic skills tasks. The current study is a continuation of an ongoing quality initiative to establish a summative assessment of laparoscopic surgical skills using virtual reality simulator for the maintenance of credentials among obstetrical and gynecologic surgeons. Future studies will compare the performance metrics from laparoscopic procedures performed on virtual reality simulator with the performance in the operating room and clinical outcomes.

Crowd-Sourced Assessment of Surgical Skills of Urology Resident Applicants: Four-Year Experience

To determine a) if surgical skills among urology resident applicants could be reliably assessed via crowdsourcing and b) to what extent surgical skills testing impacts resident selection. Interviewees completed the following surgical skills tasks during their interview day: open knot tying (OKT), laparoscopic peg transfer (LPT), and robotic suturing (RS). Urology faculty and crowd-workers evaluated each applicant's video-recorded performance using validated scoring and were assessed for agreement using Cronbach's alpha. Applicants' USMLE scores, interview scores, and Jefferson Scale of Physician Empathy (JSPE-S) scores were assessed for correlation with skills testing scores and match rank. Additionally, a survey was distributed to interviewees assessing match outcomes. University of California Irvine Department of Urology, Surgical Skills Laboratory PARTICIPANTS: All 94 urology residency interviewees at the University of California Irvine Department of Urology from 2015-2018 were invited to complete the three surgical skills tasks on their interview day. Survey responses were received from all 94 interviewees (100%). Crowd and expert agreement was good (α=0.88), fair (α=0.67), and poor (α=0.32) for LPT, RS, and OKT scores, respectively. The skills testing scores did not correlate with match rank, USMLE score, or JSPE-S score. On multivariate analysis, only interview score (r= -0.723; p<0.001) and faculty LPT score (r=-0.262; p=0.001) were significant predictors of match rank. Interviewees who reported matching into a top 3 residency choice had significantly higher faculty LPT scores than those who did not (11.9 vs. 9.7, p=0.03). Surgical skills overall did not significantly impact match rank. Expert assessment of laparoscopic peg transfer skills and interview performance among urology resident applicants correlated with match rank.

Obstetric fistula scoring tool for surgical trainees.

To examine the ability of a previously published risk score to predict incontinence at discharge in women with vesicovaginal fistulas (VVF) and to examine how the score correlates with an independent assessment of surgical skill. This is a retrospective cohort study including cases from January to June 2018. We evaluated operative records for factors associated with incontinence at hospital discharge, as well as relationships between a risk score cut-point of 20 or more and surgical skill level. All women with VVF undergoing vaginal repair were included. A total of 176 individuals underwent repair; 23 were performed by beginner, 85 by intermediate, 47 by advanced, and 21 by expert surgeons. Factors found significantly associated with incontinence at hospital discharge included Goh classification, fistula size, circumferential fistula, and vaginal scarring. A score of 20 or more predicted residual incontinence with a negative predictive value of 92% (odds ratio 7.75, 95% confidence interval 2.95-22.34). Applying the score cut-point of 20 or more, we found an increased proportion of "high-risk" cases allocated to surgeons with an increasing level of expertise. The correlation we observed between a risk score cut-point of 20 or more, continence status, and an independent assessment of surgical skill is promising. Although the risk score is not meant to replace clinical judgment, it may provide a surgical trainee with an objective method of determining whether to operate or refer for optimal outcomes.

Surgical skill assessment using machine learning algorithms

Abstract Objective Surgical skill is correlated with clinical outcomes. Therefore, the assessment of surgical skill is of major importance to improve clinical outcomes and increase patient safety. However, surgical skill assessment often lacks objectivity and reproducibility. Furthermore, it is time-consuming and expensive. Therefore, we developed an automated surgical skill assessment using machine learning algorithms. Methods Surgical skills were assessed in videos of laparoscopic cholecystectomy using a three-step machine learning algorithm. First, a three-dimensional convolutional neural network was trained to localize and classify the instruments within the videos. Second, movement patterns of the instruments were recorded over time and extracted. Third, the movement patterns were correlated with human surgical skill ratings using a linear regression model to predict surgical skill ratings automatically. Machine ratings were compared against human ratings of four board certified surgeons using a score ranging from 1 (poor skills) to 5 (excellent skills). Results Human raters and machine learning algorithms assessed surgical skills in 242 videos. Inter-rater reliability for human raters was excellent (79%, 95%CI 72-85%). Instrument detection showed an average precision of 78% and average recall of 82%. Machine learning algorithms showed an 87% accuracy in predicting good or poor surgical skills, when compared to human raters. Conclusion Machine learning algorithms can be trained to distinguish good and poor surgical skills with high accuracy. This work was published in Sci Rep 11, 5197 (2021). https://doi.org/10.1038/s41598-021-84295-6

Non-Technical Skill Assessment and Mental Load Evaluation in Robot-Assisted Minimally Invasive Surgery.

BACKGROUND: Sensor technologies and data collection practices are changing and improving quality metrics across various domains. Surgical skill assessment in Robot-Assisted Minimally Invasive Surgery (RAMIS) is essential for training and quality assurance. The mental workload on the surgeon (such as time criticality, task complexity, distractions) and non-technical surgical skills (including situational awareness, decision making, stress resilience, communication, leadership) may directly influence the clinical outcome of the surgery. METHODS: A literature search in PubMed, Scopus and PsycNet databases was conducted for relevant scientific publications. The standard PRISMA method was followed to filter the search results, including non-technical skill assessment and mental/cognitive load and workload estimation in RAMIS. Publications related to traditional manual Minimally Invasive Surgery were excluded, and also the usability studies on the surgical tools were not assessed. RESULTS: 50 relevant publications were identified for non-technical skill assessment and mental load and workload estimation in the domain of RAMIS. The identified assessment techniques ranged from self-rating questionnaires and expert ratings to autonomous techniques, citing their most important benefits and disadvantages. CONCLUSIONS: Despite the systematic research, only a limited number of articles was found, indicating that non-technical skill and mental load assessment in RAMIS is not a well-studied area. Workload assessment and soft skill measurement do not constitute part of the regular clinical training and practice yet. Meanwhile, the importance of the research domain is clear based on the publicly available surgical error statistics. Questionnaires and expert-rating techniques are widely employed in traditional surgical skill assessment; nevertheless, recent technological development in sensors and Internet of Things-type devices show that skill assessment approaches in RAMIS can be much more profound employing automated solutions. Measurements and especially big data type analysis may introduce more objectivity and transparency to this critical domain as well. SIGNIFICANCE: Non-technical skill assessment and mental load evaluation in Robot-Assisted Minimally Invasive Surgery is not a well-studied area yet; while the importance of this domain from the clinical outcome’s point of view is clearly indicated by the available surgical error statistics.

Machine learning: principles and applications for thoracic surgery.

Machine learning (ML) has experienced a revolutionary decade with advances across many disciplines. We seek to understand how recent advances in ML are going to specifically influence the practice of surgery in the future with a particular focus on thoracic surgery. Review of relevant literature in both technical and clinical domains. ML is a revolutionary technology that promises to change the way that surgery is practiced in the near future. Spurred by an advance in computing power and the volume of data produced in healthcare, ML has shown remarkable ability to master tasks that had once been reserved for physicians. Supervised learning, unsupervised learning and reinforcement learning are all important techniques that can be leveraged to improve care. Five key applications of ML to cardiac surgery include diagnostics, surgical skill assessment, postoperative prognostication, augmenting intraoperative performance and accelerating translational research. Some key limitations of ML include lack of interpretability, low quality and volumes of relevant clinical data, ethical limitations and difficulties with clinical implementation. In the future, the practice of cardiac surgery will be greatly augmented by ML technologies, ultimately leading to improved surgical performance and better patient outcomes.

Should Robotic Surgery Simulation Be Introduced in the Core Surgical Training Curriculum?

Introduction: The focus of this research is to qualitatively analyse the literature and address the knowledge gap between robotic surgery simulation (RoSS) and core surgical training curriculum. It will compare the effectiveness and the benefits of using robotic simulators in training as compared to the current standard training methods.Materials and Methods: A qualitative research of literature was carried out with the use of critical analysis formatting to expand the search. The inclusion criteria entailed selecting academic resources that focused on Robotic Surgery Simulation (RoSS) and core surgical curriculum. The Online databases used in the search took into account information retrieval from stakeholders.Evidence Synthesis: In this article, we compiled and scrutinized the available relevant literature comparing performance assessments, surgical skills transfer and assessment tools between robotic surgery simulation (RoSS) and current training platforms in open and minimal access surgery. Data that has been published underpins the authenticity of robotic Surgery Simulation (RoSS), based on a combination of observational evaluation and simulation scores.Conclusion: The introduction of robotic surgery simulation (RoSS) has the potential to bring major improvements in the surgical training curriculum. RoSS platforms are more robust in terms of ensuring rapid surgical skills transfer/ acquisition, assessment is standardized, unbiased and the training covers non-technical skills aspects.

Automation of surgical skill assessment using a three-stage machine learning algorithm

Surgical skills are associated with clinical outcomes. To improve surgical skills and thereby reduce adverse outcomes, continuous surgical training and feedback is required. Currently, assessment of surgical skills is a manual and time-consuming process which is prone to subjective interpretation. This study aims to automate surgical skill assessment in laparoscopic cholecystectomy videos using machine learning algorithms. To address this, a three-stage machine learning method is proposed: first, a Convolutional Neural Network was trained to identify and localize surgical instruments. Second, motion features were extracted from the detected instrument localizations throughout time. Third, a linear regression model was trained based on the extracted motion features to predict surgical skills. This three-stage modeling approach achieved an accuracy of 87 ± 0.2% in distinguishing good versus poor surgical skill. While the technique cannot reliably quantify the degree of surgical skill yet it represents an important advance towards automation of surgical skill assessment.

Surgical Hand Gesture Recognition Utilizing Electroencephalogram as Input to the Machine Learning and Network Neuroscience Algorithms.

Surgical gestures detection can provide targeted, automated surgical skill assessment and feedback during surgical training for robot-assisted surgery (RAS). Several sources including surgical videos, robot tool kinematics, and an electromyogram (EMG) have been proposed to reach this goal. We aimed to extract features from electroencephalogram (EEG) data and use them in machine learning algorithms to classify robot-assisted surgical gestures. EEG was collected from five RAS surgeons with varying experience while performing 34 robot-assisted radical prostatectomies over the course of three years. Eight dominant hand and six non-dominant hand gesture types were extracted and synchronized with associated EEG data. Network neuroscience algorithms were utilized to extract functional brain network and power spectral density features. Sixty extracted features were used as input to machine learning algorithms to classify gesture types. The analysis of variance (ANOVA) F-value statistical method was used for feature selection and 10-fold cross-validation was used to validate the proposed method. The proposed feature set used in the extra trees (ET) algorithm classified eight gesture types performed by the dominant hand of five RAS surgeons with an accuracy of 90%, precision: 90%, sensitivity: 88%, and also classified six gesture types performed by the non-dominant hand with an accuracy of 93%, precision: 94%, sensitivity: 94%.

Validity of scoring systems for the assessment of technical and non-technical skills in ophthalmic surgery-a systematic review.

Evaluation and recommendation of the scoring systems for technical skills (TS) and non-technical skills (NTS) assessments in ophthalmic surgery. A literature search was performed between December 2019 and May 2020. Studies describing the development or validation of TS or NTS scoring systems in ophthalmic surgery were included. Only scoring systems for completion by hand were included. The primary outcome was the validity and reliability status for each scoring system. The secondary outcome was recommendation based on modified Oxford Centre for Evidence-Based Medicine guidelines. Nineteen and five scoring systems were identified for TS and NTS respectively. TS scoring systems exist for cataract surgery (including the steps of phacoemulsification and paediatric cataract surgery) ptosis, strabismus, lateral tarsal strip, vitrectomy, and intraocular surgery in general. NTS scoring systems apply to cataract surgery or ophthalmic surgery in general. No single scoring system satisfied all validity and reliability measures. The recommended TS scoring systems are 'International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubrics' (ICO-OSCAR) for phacoemulsification, strabismus and paediatric cataract surgery, and 'Objective Structured Assessment of Cataract Surgical Skill' (OSACSS). Non-Technical Skills for Surgeons (NOTSS), Observational Teamwork Assessment for Surgery (OTAS) and Anaesthetists Non-Technical Skills (ANTS) are recommended for NTS. There is a paucity of NTS scoring systems. Further research is required to validate all scoring systems to consistent standards. Limitations of the assessment tools included infrequent quantification of face and content validity, and inconsistency in terminology and statistical methods between studies.

6. Assessing the Validity and Reliability of the Zwisch Scale in Plastic Surgery Residency

Body: Background: The Zwisch scale is well recognized as an important simple model for assessing surgical competency but has never been studied specifically in plastic surgery. We look to assess the validity and reliability of the Zwisch scale for the use in plastic surgery residency. Methods: This study was a prospective, pilot study which occurred over a 6-month period with several attending surgeons in 3 different teaching hospitals, evaluating residents of various PGY levels performing procedures of varying complexities. Two samples were obtained. One sample, all procedures performed by a participating faculty member and a participating resident during the given period were included. The second sample, a subset of the first that included 15 procedures. The procedures were observed in person by 2 additional surgeons who rated the resident’s performance using two Zwisch scores and a third surgeon validated the operative skill assessment tool- Ottawa Surgical Competency OR Evaluation (O-SCORE). All raters were blinded to each other’s rating. Results: A total of 6 residents participated with 5 attendings, with a total number of encounters of 126 for 184 procedures. The association between the Q1 (Complexity of Procedure- Easiest, Average, Hardest) and Q2 (Zwisch Level) was estimated with Spearman correlation and tested using Chi-square test. ICC and Spearman’s rank correlation coefficient demonstrated validity of Zwisch scores in plastic surgery procedures with good reliability amongst attending surgeons with Resident self-rating decreased score reliability (ICC Without self-rating 0.817, with self-rating 0.752). Conclusion: When comparing plastic surgery to general surgery we see more individuality among surgeons with addition of more complexity and longer operative times. This makes it more difficult to use a standard scoring system to evaluate operative skills of plastic surgery residents. We found that Zwisch scoring is valid and has good inter-rater reliability in plastic surgery. Compared to other surgical skill assessment tools, it is quick and efficient. Therefore, current research is a good steppingstone to further research with potential for developing a plastic surgery-specific, modified-Zwisch scoring system that will give excellent reliability.

A Review on Deep Learning in Minimally Invasive Surgery

In the last five years, deep learning has attracted great interest in computer-assisted systems for Minimally Invasive Surgery. The straightforward accessibility to images in surgical interventions makes deep neural networks enormously powerful for solving classification problems in complex surgical scenarios. The objective of this work is to provide readers a survey on deep learning models applied to minimally invasive surgery, identifying the different architectures used depending on the application, the results achieved until now, and the publicly available surgical datasets that can be used for validating new studies. A total of 85 publications have been extracted from manual research from four databases (IEEE Xplorer, Springer Link, Science Direct, and ACM Digital Library). After analyzing all these studies, they have been classified into four applications: surgical image analysis, surgical task analysis, surgical skill assessment, and automation of surgical tasks. This work provides a technical description of these works and a comparison among them. Finally, promising research directions to advance in this field are identified.

Resident Training and the Assessment of Orthopaedic Surgical Skills.

Background:Medical knowledge and technical skills are foundations of surgical competency. The American Board of Orthopaedic Surgery (ABOS) and the Resident Review Committee for Orthopaedic Surgery recently mandated simulation training to improve surgical skills, listing 17 surgical skills modules to improve residents' technical skills. However, there is no established tool to measure the effectiveness of these modules. The Global Index for Technical Skills (GRITS) tool has been previously validated for evaluating general surgery residents. The aim of this study was to determine whether the GRITS tool is valid, practical, and reliable in evaluating the skills of orthopaedic residents in a simulation setting, whether the outcomes correlate to performance in the operating room, and to what extent these simulation modules are valued by residents.Methods:Simulation performance was assessed longitudinally on 5 residents using the GRITS assessment through postgraduate years (PGY) 1 to 5 (n = 25 evaluations) in a simulated volar forearm approach using cadaveric specimens. An additional 20 PGY-1 residents were evaluated cross-sectionally in this same time frame. Written, open-ended feedback on the simulation experience was sought and analyzed via a thematic analysis. For correlative data, evaluations (n = 65 evaluations) of a variety of authentic surgical procedures were compiled on PGY-2 through PGY-5 orthopaedic residents and compared with the simulated experiences.Results:GRITS scores were averaged for each group of residents, and validity and reliability were assessed using R-software. PGY-1 residents' mean GRITS evaluation score (expressed as a value from 1 to 5) was 3.4. Longitudinally, this mean score increased over the PGY years 2-5 to 4.4, 4.7, 4.9, and 4.8, respectively. Of the parameters measured by GRITS, the lowest average scores were “flow of operation” and “time and motion” across all levels, although these did improve over PGY years 2 to 5. Findings were consistent between simulation and “real-world” procedures. Open-ended responses evaluating the module were positive.Conclusions:Our study suggests that the GRITS tool shows promise as an effective and reliable method for assessing orthopaedic resident's technical skills based on an ABOS module system.

16937 Evaluation of construct validity and inter-rater reliability of the Objective Structured Assessment of Technical Skills (OSATS) tool for assessment of surgical skills of dermatology trainees

Background: Assessment of procedural skills is an integral component of residency training. The OSATS tool, comprised of the global rating scale (GRS) and a suture-task checklist (STC), has been validated to evaluate surgical performance among surgical trainees. We assessed construct validity and inter-rater reliability of the OSATS tool in evaluating video recordings of dermatology residents throughout different stages of training.

The influence of reviewer's occupation on the skill assessment of urethrovesical anastomosis in robot-assisted radical prostatectomy.

In recent years, crowd-sourced assessments have been reported as a timesaving, cost-efficient, and practicable method of surgical skill evaluation. However, the differences in the assessment of surgical skills by the individual reviewers cannot be further examined in terms of characteristics of the reviewers because they are usually anonymously and randomly selected. This study aimed to reveal the effects of reviewers' occupations on their assessment of a surgeon's skill. In total, 42 urologists, 19 paramedics, 73 medical students, and 28 non-medical personnel used the Global Evaluative Assessment of Robotic Skills (GEARS) validated robotic surgery rating tool to assess the surgical skill of surgeons in nine edited video clips of complete urethrovesical anastomosis during a robot-assisted radical prostatectomy. The total GEARS scores of the four groups of reviewers were compared, and the similarities and the differences between the ratings of the urologists group and those of the other three groups were subsequently investigated. The rankings of video clips in the order of GEARS scores were very similar in each group, and a strong positive correlation (R2 values >0.8) was observed between the scores assigned by the urologists group and those assigned by the other three groups. Our findings indicate that the crude evaluation of robot-assisted urethrovesical anastomosis is not affected by the reviewers' occupations. Non-medical personnel may be able to provide a rudimentary screening evaluation of surgical skill.

Establishing an Orthopedic Program–Specific, Comprehensive Competency–Based Education Program

BackgroundCompetency-based education (CBE) seeks to determine resident proficiency in the knowledge, skills, and behaviors required for independent patient care. Multiple assessment instruments evaluate technical skills or direct patient care in the clinic setting, but there are few reports incorporating both within an orthopedic specialty rotation. This study reports a residency program's comprehensive CBE initiative using formative assessments in the clinic and operating room during a sports medicine rotation. Materials and methodsThe sports medicine rotation used validated formative assessments to evaluate resident performance during clinic encounters and program-defined surgical entrustable professional activities (EPAs). Junior resident (postgraduate year [PGY] 1-2) EPAs included basic knee/shoulder arthroscopic procedures. Senior resident (PYG 5) EPAs comprised anterior cruciate ligament reconstruction, biceps tenodesis, shoulder stabilization, and rotator cuff repair. Assessment scores were compared between individuals and PGY groups. ResultsSixty-six clinical skills (CS) and 106 surgical skills assessments were conducted for 22 residents in one academic year. Surgical skills assessments demonstrated significant differences between each PGY group (P < 0.01). All PGY2 and PGY5 residents achieved independence on the evaluated EPAs. PGY5s earned higher scores in CS assessments than the other classes (P < 0.01). PGY2 residents scored higher than PGY1s in 7 of 9 CS domains. CS independence was achieved by 21 of 22 residents by the end of the rotation. ConclusionsThe CBE program effectively quantified expected differences in resident performance by PGY for clinic and surgical assessments on a sports medicine rotation. Assessments built an environment where feedback was more structured and standardized, creating a culture to improve resident education.

The computerized objective assessment of surgical skills: Considerations for counting the number of movements.

Motion capture and analysis techniques are emerging in the surgical education and surgical education research literature as viable ways to augment the assessment of technical skills. In particular, these methods provide an opportunity to reveal objective information about the efficiency of surgical procedures, above and beyond the accuracy of procedural outcomes. One assessment that is very prevalent in the literature are counts of the number of movements a surgeon makes in completing a technical performance. In this commentary, the number of movements metric is explored from kinesiology and engineering perspectives; two disciplines that have contributed heavily to the development of rigorous motion analysis methods. Furthermore, the assumption that skill efficiency improves linearly as a learner progresses along the continuum of expertise is challenged. While movement efficiency does certainly improve, this assumption does not necessarily capture the way that learners flexibly prioritize particular aspects of performance in the intermediate stages of skill learning. By way of this commentary, important a priori decisions that should proceed effective motion capture and analysis are highlighted, a call for the standardization of procedures is made, and an opportunity to better understand the way that computerized movement analysis techniques may contribute (or be detrimental) to competency constructs in surgical education and assessment is realized.

Clearness of operating field: a surrogate for surgical skills on in vivo clinical data.

Automatic surgical skill assessment is an emerging field beneficial to both efficiency and quality of surgical education and practice. Prior works largely evaluate skills on elementary tasks performed in the simulation laboratory, which cannot fully reflect the variety of intraoperative circumstances in the real operating room. In this paper, we attempt to fill this gap by expanding surgical skill assessment onto a clinical dataset including fifty-seven in vivo surgeries. To tackle the workflow and device constraints in the clinical setting, we propose a robust and non-interruptive surrogate for surgical skills, namely the clearness of operating field (COF), which shows strong correlation with overall skills and high inter-annotator consistency on our clinical data. Then, an automatic model based on neural networks is developed to regress surgical skills through the surrogate of COF using only video as input. The automatic model achieves 0.595 Spearman's correlation with the ground truth of overall technical skill, which even exceeds the human performance of junior surgeons. Moreover, an exploratory study is conducted to validate the skill predictions against the clinical outcomes of patients. Our results demonstrate that the surrogate of COF is promising and the approach is potentially applicable to clinical practice.