Modified Objective Structured Assessment Research Articles

How to integrate robotic training in surgical residency? An example of a 2-week robotic rotation.

General surgery residents should be proficiently trained in robotic surgery. However, there is currently no standardized robotic training curriculum. We aimed to evaluate two approaches to a robotic curriculum and how implementing a virtual reality (VR) simulation curriculum improves trainee robotic performance. From 2019 to 2022, two models of a robotic training curriculum were examined: an in-unit rotation (IUR) and a 2-week curriculum (2WR). The VR curriculum was completed using the da Vinci® Skill Simulator. The curriculum used a pre/post-test design. Residents completed a pre-test that consisted of 4 VR exercises (graded 0-100%) and 3 inanimate box trainer exercises (graded using modified Objective Structured Assessment of Technical Skills). Then, residents completed a VR curriculum of 23 modules. Following the curriculum, residents were given a post-test with the same pre-test exercises. Time necessary to complete the curriculum and compliance were recorded. Of the 11 residents who participated in the IUR, 4 completed the VR curriculum. Comparatively, 100% (n = 23) of residents in the 2WR completed the curriculum. Average time to complete the VR curriculum was 3.8h. After completion of the 2WR curriculum, resident performance improved from pre-test to post-test: VR test scores increased (160% vs 223%, p < 0.001), OSATS scores increased (15.0 vs 21.0, p < 0.001), and time to complete inanimate exercises decreased (1083 vs 756s, p = 0.001). Residents who mastered all modules had higher post-test VR scores (241% vs 214%, p = 0.024). General surgery residents demonstrated improved compliance with the 2WR. The VR curriculum improved resident robotic performance in both virtual and inanimate domains.

“Seeing inside out”: revealing the effectiveness of otoscopy training in virtual reality enhanced practical exams - a randomized controlled trial

BackgroundThe study aimed to assess the impact of different training modalities on otoscopy performance during a practical exam using a high-fidelity simulator and to determine if objective evaluation of otoscopy is feasible using a simulator that records insertion depth and tympanic membrane coverage.MethodsParticipants were assigned to one of four groups: control and three intervention groups with varying training approaches. Participants received otoscopy training and then were assessed through a practical exam on a high-fidelity simulator that uses virtual reality to visualize the ear canal and middle ear. Performance was evaluated using a modified Objective Structured Assessment of Technical Skills checklist and Integrated Procedural Performance Instrument checklist. Insertion depth, tympanic membrane coverage, and correct diagnosis were recorded. Data were tested for normal distribution using the Shapiro-Wilk test. One-way ANOVA and, for non-normally distributed data, Kruskal-Wallis test combined with Dunn’s test for multiple comparisons were used. Interrater reliability was assessed using Cohen’s κ and Intraclass correlation coefficient.ResultsAll groups rated their training sessions positively. Performance on the OSATS checklist was similar among groups. IPPI scores indicated comparable patient handling skills. The feedback group examined larger tympanic membrane areas and had higher rates of correct diagnosis. The correct insertion depth was rarely achieved by all participants. Interrater reliability for OSATS was strong. IPPI reliability showed good correlation.ConclusionRegardless of training modality, participants perceived learning improvement and skill acquisition. Feedback improved examination performance, indicating simulator-guided training enhances skills. High-fidelity simulator usage in exams provides an objective assessment of performance.

Developing a Training Model for Local Flaps Using Fresh Human Skin Excised During Tummy Tuck Procedures

Introduction: The proficient execution of local flaps poses a challenge for surgical residents. This study aimed to assess a training model for local flaps using fresh human skin excised during tummy tuck procedures. Methods: A questionnaire and a surgical skills evaluation, examining theoretical and procedural knowledge of local flaps, were administered before and after the training. All surgical procedures were performed on a simulation model utilizing freshly excised human skin. Skills were evaluated using a modified version of the Objective Structured Assessment of Technical Skills Score. SPSS version 21 was employed for comparing pre- and post-training results. Results: In the pretraining assessment, residents faced difficulties in flap design accuracy and wound coverage adequacy, indicating a need for surgical training beyond the operating theater. Post-training, there was a significant improvement in procedural skills, as reflected in the modified Objective Structured Assessment of Technical Skills score. The mean cumulative pretraining score was 26.81 ± 5.41, while the posttraining score was 43.59 ± 5.72 (P = 0.008). Theoretical knowledge also significantly improved in the posttraining evaluation, except for the indication of a Z-plasty (P = 0.257). The training model received positive feedback and was deemed highly useful and recommendable. Discussion: The presented model demonstrated improved surgical handling and understanding of tissue rotation under realistic conditions. The simulation model, using fresh human skin, is cost-effective and versatile for a broad range of flap procedures, advocating for its further adoption. Conclusions: The training model significantly enhanced surgical handling and comprehension of tissue rotation, replicating realistic conditions. The cost-effective simulation model, based on fresh human skin, offers versatility for various flap procedures, emphasizing the need for its wider implementation.

Adapting to a Robotic Era: The Transferability of Open and Laparoscopic Skills to Robotic Surgery

The learning curve of robotic surgical skills is poorly understood. There is a lack of data on the transferability of skills from open and laparoscopic training to robotic surgery. In this retrospective cohort study, we investigated the impact of training acquired during intern year on the development of robotic skills in general surgery residents, prior to formal robotic training. Between 2019 and 2021, novice general surgery residents underwent robotic skill assessment using 3 validated inanimate drills before starting intern year. After completing basic open and laparoscopic proficiency-based curricula, they completed an identical robotic skill assessment at the end of intern year. Pre and post intern year robotic performances were recorded and analyzed by 2 blinded graders. Video-based assessment included completion time, errors, and the modified Objective Structured Assessment of Technical Skills (mOSATS) score. The total time needed to complete all 3 robotic drills decreased from a mean of 26 to 17 minutes after intern year (p < 0.001). The number of errors decreased from a mean of 2.16 to 0.56 errors per subject (p < 0.001). The aggregated mOSATS score increased by an average of 41% (p < 0.001), with a greater increase in technical skill domains compared to the knowledge-based domain. The interrater intraclass correlation coefficient was 0.91. Baseline robotic surgical skills are limited without formal training. Our findings suggest that acquiring basic open and laparoscopic skills, such as knot tying, needle driving, and tissue handling results in improved performance on the robotic platform, prior to formal robotic training. Therefore, requiring trainees to complete fundamental open and laparoscopic training prior to robotic training may be an efficient and effective strategy within a surgical residency curriculum.

High-Fidelity Cleft Simulation Maintains Improvements in Performance and Confidence: A Prospective Study

High-fidelity simulation has a growing role in plastic surgical education. This study tests the hypothesis that cleft lip repair simulation followed by structured debriefing improves performance and self-confidence and that gains are maintained. Prospective, single-blinded interventional study with repeated measures. Trainees performed cleft lip repair on a high-fidelity simulator followed by debriefing, immediately completed a second repair, and returned 3 months later for a third session. Anonymized simulation videos were rated using the modified Objective Structured Assessment of Technical Skills (OSATS) and the Unilateral Cleft Lip Repair competency assessment tool (UCLR). Self-assessed cleft lip knowledge/confidence and procedural self-confidence were surveyed after each simulation. Boston Children's Hospital, a tertiary care academic hospital in Boston, MA, USA. All trainees rotating through the study setting were eligible. Twenty-six participated; 21 returned for follow-up. Significant improvements (p < 0.05) occurred between the first and second simulations for OSATS, UCLR, and procedural self-confidence. Significant improvement occurred between the second and third simulations cleft lip knowledge/confidence. Compared to the first simulation, improvements were maintained at the third simulation for all variables. Training level moderately correlated with score for UCLR for the first simulation (r = 0.55, p < 0.01), deteriorated somewhat with the second (r = 0.35, p = 0.08), and no longer corelated by the third (r = 0.02, p = 0.92). Objective performance and subjective self-assessed knowledge and confidence improve with high-fidelity simulation plus structured debriefing and improvement is maintained. Differences in procedure-specific performance seen with increasing training level are reduced with simulation, suggesting it may accelerate knowledge and skill acquisition.

Evidence supporting performance measures of laparoscopic appendectomy through a novel surgical proficiency assessment tool and low-cost laparoscopic training system.

Laparoscopic training remains inaccessible for surgeons in low- and middle-income countries, limiting its widespread adoption. We developed a novel tool for assessment of laparoscopic appendectomy skills through ALL-SAFE, a low-cost laparoscopy training system. This pilot study in Ethiopia, Cameroon, and the USA assessed appendectomy skills using the ALL-SAFE training system. Performance measures were captured using the ALL-SAFE verification of proficiency tool (APPY-VOP), consisting of a checklist, modified Objective Structured Assessment of Technical Skills (m-OSATS), and final rating. Twenty participants, including novice (n = 11), intermediate (n = 8), and expert (n = 1), completed an online module covering appendicitis management and psychomotor skills in laparoscopic appendectomy. After viewing an expert skills demonstration video, participants recorded their performance within ALL-SAFE. Using the APPY-VOP, participants rated their own and three peer videos. We used the Kruskal-Wallis test and a Many-Facet Rasch Model to evaluate (i) capacity of APPY-VOP to differentiate performance levels, (ii) correlation among three APPY-VOP components, and (iii) rating differences across groups. Checklist scores increased from novice (M = 21.02) to intermediate (M = 23.64) and expert (M = 28.25), with differentiation between experts and novices, P = 0.005. All five m-OSATS domains and global summed, total summed, and final rating discriminated across all performance levels (P < 0.001). APPY-VOP final ratings adequately discriminated Competent (M = 2.0), Borderline (N = 1.8), and Not Competent (M = 1.4) performances, Χ2 (2,85) = 32.3, P = 0.001. There was a positive correlation between ALL-SAFE checklist and m-OSATS summed scores, r(83) = 0.63, P < 0.001. Comparison of ratings suggested no differences across expertise levels (P = 0.69) or location (P = 0.66). APPY-VOP effectively discriminated between novice and expert performance in laparoscopic appendectomy skills in a simulated setting. Scoring alignment across raters suggests consistent evaluation, independent of expertise. These results support the use of APPY-VOP among all skill levels inside a peer rating system. Future studies will focus on correlating proficiency to clinical practice and scaling ALL-SAFE to other settings.

A synthetic model simulator for intracranial aneurysm clipping: validation of the UpSurgeOn AneurysmBox.

In recent decades, the rise of endovascular management of aneurysms has led to a significant decline in operative training for surgical aneurysm clipping. Simulation has the potential to bridge this gap and benchtop synthetic simulators aim to combine the best of both anatomical realism and haptic feedback. The aim of this study was to validate a synthetic benchtop simulator for aneurysm clipping (AneurysmBox, UpSurgeOn). Expert and novice surgeons from multiple neurosurgical centres were asked to clip a terminal internal carotid artery aneurysm using the AneurysmBox. Face and content validity were evaluated using Likert scales by asking experts to complete a post-task questionnaire. Construct validity was evaluated by comparing expert and novice performance using the modified Objective Structured Assessment of Technical Skills (mOSATS), developing a curriculum-derived assessment of Specific Technical Skills (STS), and measuring the forces exerted using a force-sensitive glove. Ten experts and eighteen novices completed the task. Most experts agreed that the brain looked realistic (8/10), but far fewer agreed that the brain felt realistic (2/10). Half the expert participants (5/10) agreed that the aneurysm clip application task was realistic. When compared to novices, experts had a significantly higher median mOSATS (27 vs. 14.5; p < 0.01) and STS score (18 vs. 9; p < 0.01); the STS score was strongly correlated with the previously validated mOSATS score (p < 0.01). Overall, there was a trend towards experts exerting a lower median force than novices, however, these differences were not statistically significant (3.8 N vs. 4.0 N; p = 0.77). Suggested improvements for the model included reduced stiffness and the addition of cerebrospinal fluid (CSF) and arachnoid mater. At present, the AneurysmBox has equivocal face and content validity, and future versions may benefit from materials that allow for improved haptic feedback. Nonetheless, it has good construct validity, suggesting it is a promising adjunct to training.

368 Development, Outcome and Costs of a Simulation-Based Neurosurgery Bootcamp at the National Level

INTRODUCTION: With a growing interest in simulation-based training to develop clinical proficiency, bootcamps have been utilized for imparting basic skills to medical trainees. Some research on the topic has been published from high income countries, with considerable costs of the course. No such neurosurgical teaching standards have been employed in Low-and Middle-Income Countries METHODS: It was a cross-sectional study. Twenty two participants hailing from 6 institutes attended 12 lectures, and practiced 12 hands-on skills. Burr-holes and craniotomies were done on 3D printed skulls. Lumbar drain insertion was practiced on a lumbar puncture mannequin. For laminectomy, we used an indigenously designed simulator. Knowledge was assessed via a pre and post study design using a validated tool, while the modified Objective Structured Assessment of Technical Skills tool was utilized for skills Assessment. Feedback from faculty and residents was collected via a standard 5-point Likert scale. RESULTS: Only one participant (4.55%) had previously attended a neurosurgical skills workshop. Comparison of knowledge assessment scores of the participants on topics taught showed significant improvement (p value = 0.034). Comparison of outcomes on 1st and 3rd attempts of cranial and spinal skills showed a significant improvement in all 14 domains assessed. Positive feedback was received ranging from 3.9 up to 4.8 on a 5 point Likert scale. Overall cost per participant culminated to $145, significantly lower than previously reported data. CONCLUSIONS: Our findings report the effectiveness of sustainable, low-cost training models which can be easily reproduced elsewhere. This indigenously designed inexpensive simulator can be modified for variable difficulty level and serve as an effective educational strategy in improving learners’ skills, knowledge and confidence.

Development, outcome and costs of a simulation-based neurosurgery bootcamp at the national level

IntroductionWith a growing interest in simulation-based training to develop clinical proficiency, bootcamps have been utilized for imparting basic skills to medical trainees. While considerable research on the topic exists in high income countries, no such neurosurgical teaching standards have been employed in Low- and Middle-Income Countries.MethodsWe conducted a cross-sectional study to explore the effectiveness of first low-cost, multi-center regional neurosurgery bootcamp in South Asia. Twenty-two participants attended the bootcamp and practiced 12 hands-on skills over the course of 2 days. Burr-holes and craniotomies were done on 3D printed skulls. Lumbar drain insertion was practiced on a purpose-built lumbar puncture mannequin. For laminectomy, we used an in-house designed simulation. The modified Objective Structured Assessment of Technical Skills tool was utilized for skills Assessment. Feedback from faculty and residents was collected via a standard 5-point Likert scale.ResultsOnly one participant (4.55%) had previously attended a neurosurgical skills workshop. Comparison of outcomes on 1st and 3rd attempts of cranial and spinal skills showed a significant improvement in all 14 domains assessed (p <0.05). Positive feedback was received ranging from 3.9 up to 4.8 on a 5-point Likert scale. Overall cost per participant culminated to $145, significantly lower than previously reported data.ConclusionOur findings report the effectiveness of sustainable, low-cost training models which can be easily reproduced elsewhere. These indigenously designed simulators can be modified for variable difficulty level and serve as an effective educational strategy in improving learners’ skills, knowledge and confidence.

High fidelity simulation of the endoscopic transsphenoidal approach: Validation of the UpSurgeOn TNS Box.

Endoscopic endonasal transsphenoidal surgery is an established technique for the resection of sellar and suprasellar lesions. The approach is technically challenging and has a steep learning curve. Simulation is a growing training tool, allowing the acquisition of technical skills pre-clinically and potentially resulting in a shorter clinical learning curve. We sought validation of the UpSurgeOn Transsphenoidal (TNS) Box for the endoscopic endonasal transsphenoidal approach to the pituitary fossa. Novice, intermediate and expert neurosurgeons were recruited from multiple centres. Participants were asked to perform a sphenoidotomy using the TNS model. Face and content validity were evaluated using a post-task questionnaire. Construct validity was assessed through post-hoc blinded scoring of operative videos using a Modified Objective Structured Assessment of Technical Skills (mOSAT) and a Task-Specific Technical Skill scoring system. Fifteen participants were recruited of which n = 10 (66.6%) were novices and n = 5 (33.3%) were intermediate and expert neurosurgeons. Three intermediate and experts (60%) agreed that the model was realistic. All intermediate and experts (n = 5) strongly agreed or agreed that the TNS model was useful for teaching the endonasal transsphenoidal approach to the pituitary fossa. The consensus-derived mOSAT score was 16/30 (IQR 14-16.75) for novices and 29/30 (IQR 27-29) for intermediate and experts (p < 0.001, Mann-Whitney U). The median Task-Specific Technical Skill score was 10/20 (IQR 8.25-13) for novices and 18/20 (IQR 17.75-19) for intermediate and experts (p < 0.001, Mann-Whitney U). Interrater reliability was 0.949 (CI 0.983-0.853) for OSATS and 0.945 (CI 0.981-0.842) for Task-Specific Technical Skills. Suggested improvements for the model included the addition of neuro-vascular anatomy and arachnoid mater to simulate bleeding vessels and CSF leak, respectively, as well as improvement in materials to reproduce the consistency closer to that of human tissue and bone. The TNS Box simulation model has demonstrated face, content, and construct validity as a simulator for the endoscopic endonasal transsphenoidal approach. With the steep learning curve associated with endoscopic approaches, this simulation model has the potential as a valuable training tool in neurosurgery with further improvements including advancing simulation materials, dynamic models (e.g., with blood flow) and synergy with complementary technologies (e.g., artificial intelligence and augmented reality).

1006 Supersized-Simulation vs Traditional Cardiothoracic Surgery Education: A Prospective, Randomised Controlled Trial

Abstract Aim Exposure to cardiothoracic surgery is limited during medical school in the United Kingdom. This study evaluated the effectiveness of a supersized simulation model (SSM) compared to traditional methods (TM) in developing coronary anastomosis surgical skills. Method This was a prospective, randomised controlled trial with a 1:1 allocation ratio. Undergraduate medical students with no prior cardiothoracic surgical skills training were eligible for inclusion. Both groups received a guided didactic lecture followed by modified Objective Structured Assessment of Technical Skills (OSATS) assessment for baseline. The control group then received traditional teaching and the intervention group received additional teaching with SSM before being reassessed. Modified OSATS score were the primary outcome, adjudicated by two senior cardiothoracic registrars. Performance time (PT), self-assessment (SA), confidence (CON) and global rating scale (GRS) were secondary outcomes. Results 48 novice medical students were randomised. OSATS scores were significantly higher in SSM group compared to TM group (13.2 vs 9.8, p&amp;lt;0.01). The secondary outcomes PT (17.7 minutes vs 22.2 minutes, p&amp;lt;0.01) and CON (3.04 vs 1.73, p&amp;lt;0.01) were also significantly different between groups, favouring SSM. SSM participants were more likely to want to pursue a career in cardiothoracic surgery (12 participants vs 2 participants). All participants agreed that SSM improved visualisation and comprehension of surgical procedure. SSM cost £0.75 per model. Conclusions Supersized simulation is effective for coronary anastomosis surgical training with technical skills successfully transferred to a clinical task. Supersized models can accelerate the novice trainee's surgical skill progression and represents a valuable resource in one's educational armamentarium.

Validation and reliability testing of a rating scale for objective assessment of performance in laparoscopic appendicectomy surgery

BackgroundsTo achieve a competency‐based training paradigm, the ability to obtain reliable and valid quantitative assessments of intraoperative performance is required. Through this, weaknesses can be identified and practiced, and competency assessed. This study aimed to determine the validity and reliability an objective evaluation tool for assessment of performance in laparoscopic appendicectomy (LA).MethodsA prospective single‐blinded observational study design was used. Videos of inexperienced (performed <10 LAs) and experienced (performed >100 LAs) surgeons performing LA surgery were collected. Surgical performance during each recording was rated by two independent, blinded expert surgeons using the LA Rating Scale (LARS) and the modified Objective Structured Assessment of Technical Skill (OSATS) scale.ResultsThe intraclass correlation coefficient (ICC) for LARS was 0.95 (95%CI 0.83–0.98). The ICC for each step ranged from 0.48 to 0.90, and the test–retest ICC for LARS was 0.91 (95%CI 0.69–0.98). Significant differences (P < 0.001) between median performance scores as rated by LARS were observed between the inexperienced and experienced surgeons. A Spearman's correlation coefficient of 0.87 (P < 0.001) was observed between LARS performance scores and modified OSATS scores.ConclusionLARS demonstrated excellent inter‐rater and test–retest reliability, and construct and concurrent validity and can be used to quantitatively evaluate performance during LA. This can potentially allow specific weaknesses to be identified and improved upon through deliberate practice. Progress can be tracked through re‐evaluation and scores of expert surgeons can be used as performance goals for credentialing in LA.

Telestration in the Teaching of Basic Surgical Skills: A Randomized Trial

To evaluate how an affordable course using telestration with augmented reality can be compared to the traditional teaching of basic surgical skills. Prospective, randomized and blinded study. Faculty of Medicine of PortoUniversity. Twenty medical students without any experience in basic surgical skills were randomized into two different learning groups: telestration and traditional teaching (on-site mentoring) groups. Five different types of sutures were taught: the single interrupted, the cruciate mattress, the horizontal mattress, the vertical mattress and the simple continuous sutures. Data was obtained on the time taken to learn each of the techniques and to perform each exercise without any support from the faculty, tension of the suture, quality of the procedure using a modified Objective Structured Assessment of Technical Skills and participants' answers to a Likert questionnaire in terms of their learning experience, confidence, and self-evaluation. Trainees in the telestration group were globally faster when performing independently (1393.40 [SD 288.89] vs 1679.00 [SD 328.22] seconds, p = 0.04) particularly during the cruciate mattress suture (235.50 [SD 61.81] vs 290.00 [SD 68.77] seconds, p = 0.05) and the simple continuous suture (492.40 [SD 87.49] vs 630.30 [SD 132.34] seconds, p = 0.01).Time needed for students to learn the procedures was similar between the groups. There were also no statistically significant differences in terms of the quality of the surgical gesture, tension of the suture, self-evaluation or confidence. A basic surgical skills course using telestration through a head-mounted device with augmented reality capabilities can be a viable alternative to traditional teaching, considering time and quality of the gesture. Though costs can discourage from using this technology in basic procedures, the use of free software may turn it into an affordable option in the context of distant learning.

Surgical skill and oncological outcome of laparoscopic radical hysterectomy: JGOG1081s-A1, an ancillary analysis of the Japanese Gynecologic Oncology Group Study JGOG1081

ObjectivesWe investigated whether surgical skill and procedure were related to oncological outcomes in cervical cancer patients who underwent Laparoscopic Radical Hysterectomy (LRH). MethodsWe previously assessed data of LRH from 251 patients with FIGO stage (2009) IA2, IB1and IIA1 cervical cancer collected for JGOG 1081s study. 1) The JGOG 1081s cohort study was re-examined to refine the surgical details and extend the follow-up period as chart review. 2) Unedited videos for recurrent cases and matched non-recurrent control cases were newly compared by experts for various surgical skills and surgical procedures using the modified Objective Structured Assessment of Technical Skills (OSATS) tool, without awareness of the recurrence status as video review. ResultsAfter a median follow-up of 46 months, tumors had recurred in 31 of the 251 patients. The five-year Recurrence-Free Survival rate was 86.9% (81.8–90.6) and five-year Overall Survival rate was 93.7% (87.5–96.8). Multivariate analysis from chart reviews found that an experience with LRH of less than 20 cases per institution was an independent prognostic factor for recurrence (Hazard Ratio (HR) 2.49, 95%CI 1.12–5.53, p = 0.025). For the surgical video review, we compared 23 videos of recurrent cases with 23 background-matched non-recurrent controls. Lower modified OSATS scores from the video review were consistently trended to have a higher risk of recurrence. ConclusionsOur new study has found that LRH surgical experience and skill trended to have better oncological outcomes.

The role of motion tracking in assessing technical skill acquisition using a neonatal 3D-printed thoracoscopic esophageal atresia/tracheo-esophageal fistula simulator

IntroductionAcquiring the technical skills required for thoracoscopic repair of esophageal atresia with tracheo-esophageal fistula (EA/TEF) is challenging. A high-fidelity 3D-printed pediatric thoracoscopic EA/TEF simulator has been developed to address this issue. This study explored motion-tracking as an assessment tool to distinguish between surgeons of different expertise using the simulator. MethodsParticipants performed a single intracorporeal suture between the esophageal ends in EA with TEF. Total relative path lengths of the right and left surgical instruments were recorded during the task. Each video-recorded attempt was assessed by a blinded pediatric surgeon using a modified Objective Structured Assessment of Technical Skills (OSATS) score. Data recorded as median (range) and statistical significance as p<0.05. ResultsThe task was performed by 17 participants. The median OSATS scores identified a significant difference between experts and novices. A difference between left- and right-hands was only found in the mid-skill level group. Right-hand path length was greatest in novices and lowest in experts. Left-hand path length was greatest in novices and the mid-skill level group compared to experts. ConclusionExperts had the lowest total path length for either hand, suggesting they had the greatest efficiency of movement. The similar high path lengths in both hands for novices indicate their relatively low level of skill with either hand. The difference between right- and left-hand path lengths in the mid-skill level group likely reflects the improved right-handed technical skills in contrast to the still developing left hand. Further focus on the left hand during simulation training may improve left-handed economy of movement.

Improving Neurosurgery Resident Microdissection Through Placental Simulation

ABSTRACT Prior work has shown that the human placenta is an available and realistic model for microdissection simulation. We sought to find a measurable improvement in the technical skills of neurosurgical residents with deliberate practice of microdissection tasks using a placental model. Postgraduate year (PGY) 1 to 3 neurosurgery residents were consented. A 1-min video of each participant's baseline skills skeletonizing placental vessels was recorded. Participants underwent 10 practice sessions with intermittent informal feedback for 30-60 min over 18 mo. Another 1-min video was recorded following the 10th dissection. The videos were blinded and assessed by 3 board eligible or certified microsurgical neurosurgeons using a modified Objective Structured Assessment of Aneurysm Clipping Skills. Performance was compared via t-testing among four domains: instrument handling, time flow and forward planning, quality of dissection, and respect for tissue. Microdissection, instrument handling, and quality of dissection were significantly improved after deliberate practice with the placental simulator (P &lt; .05). Improvement was seen in time flow and forward planning and respect for tissue; however, this failed to be significant. Subjectively, residents expressed enjoyment performing the exercise. They also expressed a desire for demonstrations or videos to watch before practice sessions. The placental simulation model provides microsurgical skill development with minimal deliberate practice sessions. Practice exercises are favorably regarded and interest in continuing them is strong by residents. Residents expressed a desire to make the dissection more deliberate with demonstration, breakdown of steps, and mimicry, which could improve the effectiveness and enjoyment of the skills session.

Development of a Low-Fidelity Laparoscopic Sacrocolpopexy Simulation Model and Evaluation of Curriculum.

Minimally invasive sacrocolpopexy (SCP) is a compelling surgical procedure that requires advanced laparoscopic suturing and dissection skills and knowledge of pelvic retroperitoneal anatomy. The aim of this study was to develop a low-cost laparoscopic SCP model and educational curriculum to improve dissection and suturing skills along with anatomic knowledge to avoid complications. The pelvic SCP model was developed with easily available material that was placed on the commercial pelvic bone model to construct a vagina, major vessels, ureter, peritoneum, and areolar tissue. A comprehensive curriculum encompassing didactic and technical skills components on the pelvic model was used to teach laparoscopic SCP. Participants completed precurriculum and postcurriculum multiple-choice questionnaires to evaluate the didactic component. A modified Objective Structured Assessment of Technical Skills (OSATS) tool was used to measure technical skills before and after technical skills curriculum. Among the 10 senior residents, 6 (60%) were studying at their fourth postgraduate year, and 3 (30%) were studying at their third postgraduate year. Postcurriculum test scores (14.5 vs 10.6) for assessment of cognitive knowledge were significantly improved (P < 0.001). The median postcurriculum OSATS scores (26.5; range, 18-30) were significantly improved (P = 0.005). Laparoscopic SCP OSATS scores increased from median 21 to median 26.5 after cognitive and technical curriculum, showing a 21.4% improvement. The majority (70%) of residents believed that the SCP model would be useful to enhance skills acquisition before performing the skill in the operating room. A low-fidelity laparoscopic SCP curriculum showed improvement in cognitive knowledge and technical skills.

Construct validation of a 3D printed neonatal thoracoscopic simulator: Can it measure expertise?

Backgroundacquiring technical expertise for neonatal thoracoscopy is challenging. To address this, we designed a fully synthetic thoracoscopic simulator for which we established its construct validity. Methodsthree thoracoscopic tasks were assessed: ring transfer, needle pass and incision of a blind upper esophageal pouch (EA cut). Participants watched instructional videos with accompanying written instructions for each task before having their attempt video recorded. All tasks were marked by three blinded pediatric surgeons using a modified Objective Structured Assessment of Technical Skills (OSATS). Scores were assessed using appropriate statistical analysis and inter-rater reliability was analyzed by interclass correlation coefficient (ICC). Results23 participants completed the ring transfer and needle pass and 21 the EA cut: 5 experts (consultant surgeons), 5 intermediate (registrars on a training program) and 13 novices (medical students, house surgeons or non-training registrars). All three tasks distinguished between novice and intermediate/expert (ring transfer p = 0.00001, needle pass p = 0.0004 and EA cut p = 0.0014, respectively). Interrater reliability was good for ring transfer and needle pass but poor for EA cut. Conclusionthe tasks distinguished between novice and intermediate/expert but not between expert and intermediate. In needle pass and EA cut, there was a trend for the experts to score higher than intermediate participants. Ring transfer and needle pass tasks achieved construct validity, had good interrater reliability and were found to be useful in assessing a novice surgeon's progression towards the intermediate level. Distinguishing between intermediate and expert may require assessment of more complex tasks such as intracorporeal suturing and tying. Level of evidenceII

Evaluating the effectiveness of the Imperial Femoral Intramedullary Nailing Cognitive Task Analysis (IFINCTA) tool in a real-time simulation setting (Distributed Interactive Simulation): a randomized controlled trial

IntroductionWorking time regulations, senior led service delivery and increasing complexity of surgical technology has led to significant strains in surgical training. Additionally, the current COVID-19 pandemic has placed substantial limitations on surgical training worldwide. Contact free, remote, web-based, validated learning tools which are easily accessible and allows repeated, sustained practice are the need of the hour. Cognitive Task Analysis (CTA) have been used extensively to train pilots and military personnel and has shown excellent early results within orthopaedic training. We designed a femoral nailing CTA tool which showed objective benefits in the enhancement of cognitive knowledge in medical students. The aim of this study was to evaluate the effectiveness of this CTA tool to enhance practical skills in orthopaedic trainees in a real time interactive simulation setting (Distributed Interactive Simulation (DIS)). MethodsThis was a double blinded, randomized controlled trial. 14 junior orthopaedic residents who met the inclusion criteria were recruited in the study. They were randomized into two equal groups. The intervention group were given the CTA learning tool, the control group were given a standard operative technique manual used for antegrade femoral intramedullary nailing. The participants were assessed on a high-fidelity phantom femur model with actual femoral nailing instruments in a simulation mobile operating theatre where the candidate had a simulation patient, an acting anesthetist and a scrub nurse (DIS). They were assessed using the modified Objective Structured Assessment of Technical Skills (OSATS) rating scale which has been validated for orthopaedic trauma. ResultsThe median OSATS score in the intervention group was 49 (±4.93, range 39-55) compared to 17 in the control group (±14.98, range 12-51). The median improvement was by 32 points (p = 0.02). The ICC between the two raters was 0.977. ConclusionsThis study has demonstrated objective benefits of a novel femoral nailing CTA tool in the enhancement of practical skills for junior trainees in the DIS setting. This adds to the growing evidence supporting the use of CTA in orthopaedic training. This tool can be accessed remotely, is contact free and allows repeated sustained practice which is key in simulation training.

Crowdsourced Assessment of Surgical Skill Proficiency in Cataract Surgery

OBJECTIVE:To test whether crowdsourced lay raters can accurately assess cataract surgical skills.DESIGN:Two-armed study: independent cross-sectional and longitudinal cohorts.SETTING:Washington University Department of Ophthalmology.PARTICIPANTS AND METHODS:Sixteen cataract surgeons with varying experience levels submitted cataract surgery videos to be graded by 5 experts and 300+ crowdworkers masked to surgeon experience. Cross-sectional study: 50 videos from surgeons ranging from first-year resident to attending physician, pooled by years of training. Longitudinal study: 28 videos obtained at regular intervals as residents progressed through 180 cases. Surgical skill was graded using the modified Objective Structured Assessment of Technical Skill (mOSATS). Main outcome measures were overall technical performance, reliability indices, and correlation between expert and crowd mean scores.RESULTS:Experts demonstrated high interrater reliability and accurately predicted training level, establishing construct validity for the modified OSATS. Crowd scores were correlated with (r = 0.865, p < 0.0001) but consistently higher than expert scores for first, second, and third-year residents (p < 0.0001, paired t-test). Longer surgery duration negatively correlated with training level (r = −0.855, p < 0.0001) and expert score (r = −0.927, p < 0.0001). The longitudinal dataset reproduced cross-sectional study findings for crowd and expert comparisons. A regression equation transforming crowd score plus video length into expert score was derived from the cross-sectional dataset (r2 = 0.92) and demonstrated excellent predictive modeling when applied to the independent longitudinal dataset (r2 = 0.80). A group of student raters who had edited the cataract videos also graded them, producing scores that more closely approximated experts than the crowd.CONCLUSIONS:Crowdsourced rankings correlated with expert scores, but were not equivalent; crowd scores overestimated technical competency, especially for novice surgeons. A novel approach of adjusting crowd scores with surgery duration generated a more accurate predictive model for surgical skill. More studies are needed before crowdsourcing can be reliably used for assessing surgical proficiency.