Imperial College Surgical Assessment Device Research Articles

Development and Validation of a Laryngeal Microsurgery Simulation Training System for Otolaryngology Residents.

This study aims to create a synthetic laryngeal microsurgery simulation model and training program; to assess its face, content, and construct validity; and to review the available phonomicrosurgery simulation models in the literature. Nonrandomly assigned control study. Simulation training course for the otolaryngology residency program at Pontificia Universidad Católica de Chile. Resident (postgraduate year 1 [PGY1]/PGY2) and expert groups were recruited. A laryngeal microsurgery synthetic model was developed. Nine tasks were designed and assessed through a set of programmed exercises with increasing difficulty, to fulfill 5 surgical competencies. Imperial College Surgical Assessment Device sensors applied to the participants'hands measured time and movements. The activities were video-recorded and blindly assessed by 2laryngologists using a specific and global rating scale (SRSand GRS). A 5-point Likert survey assessing validity was completed by experts. Eighteen participants were recruited (14 residentsand 4 experts). Experts performed significantly better than residents in the SRS (p = .003), and GRS (p = .004). Internal consistency was demonstrated for the SRS (α = .972, p < .001). Experts had a shorter execution time (p = .007), and path length with the right hand (p = .04). The left hand did not show significant differences. The survey assessing validity resulted in a median 36 out of 40 points score for face validity; and 43 out of 45 points score, for global content validity. The literature review revealed 20 available phonomicrosurgery simulation models, only 6 with construct validity. The face, content, and construct validity of the laryngeal microsurgery simulation training programwere established. It could be replicated and incorporated into residents' curricula.

Training Program for Orthopedic Residents in Forefoot Osteotomy Skills: Transference From a Simulator to a Cadaveric Surgical Scenario.

An effective simulation program allows both the acquisition of surgical skills on the simulated model and the transfer of these skills to a surgical scenario. We designed a forefoot osteotomy training program and sought to determine the transferability to a cadaveric surgical scenario. Eleven orthopedic residents and 2 foot and ankle surgeons were included. A foot simulator was used. All residents were instructed on the surgical techniques of Chevron, Akin, and triple Weil osteotomies. Eight junior residents (trainees) were enrolled in a supervised simulation program. Baseline assessment was performed on the simulator with the Objective Structured Assessment of Technical Skills (OSATS) and the Imperial College Surgical Assessment Device (ICSAD). After baseline, trainees completed a training program and had a final evaluation of proficiency on the simulator and on cadaveric specimens. Three senior residents with no simulated training (controls) and experts were assessed for comparison. All trainees improved from a baseline OSATS score of 11 points (9-20) to a final score of 35 points (33-35) in the simulator and 34 points (32-34) in the cadaveric specimen ( P < 0.01). Compared with baseline, the ICSAD results improved in path length (391 [205-544] to 131 [73-278] meters, P < 0.01) and number of movements (2756 [1258-3338] to 992 [478-1908], P < 0.01). The final OSATS and ICSAD scores did not differ from experts ( P = 0.1) and were significantly different from untrained residents ( P = 0.02). Simulated training of Chevron, Akin, and triple Weil osteotomies in orthopedic residents improved procedural proficiency, enabling successful skill transfer to a surgical scenario in cadavers. II (Prospective Cohort Study).

Acquisition of microvascular suturing techniques is feasible using objective measures of performance outside of the operating room

The aim of this study was to determine the effectiveness of a microsurgical simulation course on a group of novice surgeons. The study also aimed to see whether a device to measure manual dexterity (Imperial College Surgical Assessment Device, ICSAD), a global rating scale and an end product assessment tool could be used to objectively measure performance. Thirty surgeons were recruited to take part and were evenly split into novice, intermediate, and expert groups. The novice group took part in a one-week microsurgical course. The assessment of performance was done with the surgeon anastomosing a harvested rat aorta. They were assessed using ICSAD, which records number of hand movements, distance moved by hands and time. The assessment was also video recorded and later watched by two assessors. The procedure performance was scored using a global rating scale and the final anastomosis was scored using an end product assessment tool. The novice surgeons took the assessment at the beginning and at the end of the course. The intermediate and expert group performed the assessment once to use as a comparison to the novice group. By the end of the course the novices performed significantly better in all parameters in comparison to their pre course assessment. The novice group performed similarly to the expert group in their end of course assessment in four out of the five assessment parameters. Simulation using objective measures of performance can be used to effectively to train a group of novice microsurgeons.

Validation of the imperial college surgical assessment device for spinal anesthesia

BackgroundTraditionally, technical proficiency for spinal anesthesia has been assessed using observational scales such as global rating scales or task specific checklists. However more objective metrics are required in order to improve novice’s training programs. The aim of this study is to validate the hand motion analysis of the Imperial College Surgical Assessment Device (ICSAD) in a simulated model of spinal anesthesia.MethodsThree groups of physicians with different levels of experience were video recorded performing a spinal anesthesia in a simulated lumbar puncture torso. Participants’ technical performance was assessed with ICSAD, a Global Rating Scale (GRS) and a specific Checklist. Differences between the 3 groups were determined by Kruskal-Wallis test with post hoc Dunn’s correction for multiple comparisons. Spearman correlation coefficient between ICSAD variables and the scores of the observational scales were calculated to establish concurrent validity.ResultsThirty subjects participated in the study: ten novice (first year residents), 10 intermediate (third year residents) and 10 experts (attending anesthesiologists). GRS scores were significantly higher in experts, than intermediates and novices. Regarding total path length, number of movements and procedural time measured with ICSAD, all groups had significant differences between them (p = 0.026, p = 0.045 and p = 0.005 respectively). Spearman correlation coefficient was −0,46 (p = 0.012) between total path length measured with ICSAD and GRS scores.ConclusionsThis is the first validation study of ICSAD as an assessment tool for spinal anesthesia in a simulated model. Using ICSAD can discriminate proficiency between expert and novices and correlates with previously validated GRS. Its use in the assessment of spinal anesthesia proficiency provides complementary data to existing tools. Our results could be used to design future training programs with reliable goals to accomplish.

Simulation-based training program with deliberate practice for ultrasound-guided jugular central venous catheter placement.

Current evidence supports the utility of simulation training for bedside procedures such as ultrasound-guided jugular central venous catheter (CVC) insertion. However, a standardized methodology to teach procedural skills has not been determined yet. The aim of this study was to evaluate the effectiveness of a simulation-based training program for improving novice technical performance during ultrasound-guided internal jugular CVC placement. Postgraduate year 1 (PGY-1) residents from anesthesiology, emergency medicine, cardiology, ICU, and nephrology specialties were trained in four deliberate practice sessions. Learning objectives included principles of ultrasound (US), preparation (gown, glove, draping), procedural skills I (US scanning and puncture), and procedural skills II (catheter insertion). CVC technical proficiency was tested pre- and post-training using hand-motion analysis with the Imperial College Surgical Assessment Device (ICSAD) and a global rating scale (GRS). Thirty-five PGY-1 residents successfully completed the program. These novices' GRS scores improved significantly after the training (P < 0.001). Total path length measured with the ICSAD decreased significantly after the training (P = 0.008). Procedural time decreased significantly after training from 387 (310-501) seconds to 200 (157-261) seconds (median and interquartile range) (P = 0.029). This simulation-training program based on deliberate practice significantly increased the technical skills of residents in US-guided short-axis, out-of-plane internal jugular CVC placement. Data also confirm the validity of the ICSAD as an assessment tool for ultrasound-guided internal jugular CVC placement learning.

Assessment of central venous catheterization in a simulated model using a motion-tracking device: an experimental validation study.

BackgroundCentral venous catheterization (CVC) is a basic requirement for many medical specialties. Simulated training in CVC may allow the acquisition of this competency but few reports have established a valid methodology for learning and acquiring procedural skills for CVC. This study aims to validate the use of a tracking motion device, the imperial college surgical assessment device (ICSAD), by comparing it with validated global rating scales (GRS) to measure CVC performance in a simulated torso.MethodsSenior year medical students, first and last year residents (PGY1, LYR), and expert anesthesiologists performed a jugular CVC assessment in a simulated model (Laerdal IV Torso). A validated GRS for objective assessment of technical skills and motion analysis by ICSAD was used. Statistical analysis was performed through Mann–Whitney and Kruskal–Wallis tests for construct validity and Spearman correlation coefficients between the ICSAD and GRS scores for concurrent validity between both.Results32 subjects were recruited (10 medical students, 8 PGY1, 8 LYR and 8 experts). Total path length measured with ICSAD and GRS scores were significantly different between all groups, except for LYR compared to experts (p = 0.664 for GRS and p = 0.72 for ICSAD). Regarding jugular CVC procedural time, LYR and experts were faster than PGY1 and MS (p < 0.05). Spearman correlation coefficient was −0.684 (p < 0.001) between ICSAD and GRS scores.ConclusionsICSAD is a valid tool for assessment of jugular CVC since it differentiates between expert and novice subjects, and correlates with a validated GRS for jugular CVC in a simulated torso.

A Randomized Controlled Study to Evaluate the Impact of Instrument and Laparoscope Length on Performance and Learning Curve in Single-Incision Laparoscopic Surgery.

The proximity of instrumentation in single-incision laparoscopic surgery (SILS) creates ergonomic challenges. An innovative method to reduce external collisions between instruments and handles is to use instruments of different lengths. This study evaluated the impact of instrument and laparoscope length on simulated SILS performance. Performance was assessed using peg transfer (PEG) and pattern cutting (CUT) tasks from the Fundamentals of Laparoscopic Surgery (FLS) curriculum. Following baseline testing, surgeons were randomized into 3 trial arms: Control--standard length instruments and standard length laparoscope; group 1--one long instrument, one standard length instrument and standard length laparoscope; and group 2--standard length instruments and long laparoscope. Two phases were undertaken using a validated SILS-modified FLS box trainer: phase 1--25 repetitions of PEG and phase 2--5 repetitions of CUT. FLS scoring parameters measured performance and the Imperial College Surgical Assessment Device (ICSAD) captured motion analysis of hands. Twenty-three surgeons were recruited--control (n = 7), group 1 (n = 9), and group 2 (n = 7). No significant differences were observed in operative experience or baseline skills performance. Phase 1: Peak FLS score was significantly higher in group 1 compared with control (P = .009). Comparison of learning curves revealed learning plateau was significantly higher in group 1 compared with control (P = .010). Phase 2: Group 1 revealed a trend toward higher peak FLS scores over the control (P = .067). No significant differences in motion analysis of hands were demonstrated using ICSAD. This study demonstrates that using instruments of different lengths can improve simulated SILS performance.

A Randomized Controlled Study to Establish the Effect of Articulating Instruments on Performance in Single-Incision Laparoscopic Surgery

In single-incision laparoscopic surgery (SILS), operating through 1 incision presents ergonomic challenges. No consensus exists on whether articulating instruments (ARTs) may help. This study evaluated their effect on simulated SILS, hypothesizing that they would affect performance and workload. Surgeons were randomized to 2 straight instruments (STRs), 1 ART and 1 STR, or 2 ARTs. After baseline testing, 25 repetitions of the Fundamentals of Laparoscopic Surgery (FLS) peg-transfer (PEG) task and 5 repetitions of the short-hand for the FLS pattern-cutting task (CIRCLE) were performed. Primary outcomes were maximum FLS PEG scores, CIRCLE times and errors, and Imperial College Surgical Assessment Device hand motion analysis. National Aeronautics and Space Administration (NASA) Raw Task Load Index (RTLX) questionnaires evaluated a secondary outcome--workload. The trial took place in a simulated operating theater within the Academic Surgical Unit at St Mary's Hospital, London, UK. Eligible surgeons had completed at least 5 laparoscopic cases as a primary operator. Surgeons were stratified by laparoscopic experience into intermediate (less than 25 previous procedures as primary operator) or advanced (25 procedures or more). A total of 21 surgeons were recruited and randomized; 7 of them to each instrument combination group. All surgeons completed PEG, and 5 from each group completed CIRCLE. Groups' baseline PEG scores were similar (p = 0.625). STR-ART achieved higher maximum PEG scores than STR or ART did (median = 236 vs 198 vs 193, respectively, p = 0.002). Fastest CIRCLE times were similar (median = 190s vs 130s vs 186s, p = 0.129) as were minimum errors (median = 1 vs 2 vs 3, p = 0.101). For PEG, Imperial College Surgical Assessment Device demonstrated similar total path lengths (median = 12.3m vs 12.3m vs 16.0m, p = 0.545) and total numbers of movements (median = 89.6 vs 86.4 vs 171, p = 0.080). Groups' NASA Raw Task Load Index scores were similar (p = 0.708). Combining 1 STR and 1 ART improved SILS performance in the PEG task. Therefore, this may be the optimum instrument configuration for use within some clinical SILS applications.

A Cadaveric Porcine Model for Assessment in Laparoscopic Bariatric Surgery—a Validation Study

Laparoscopic Roux-en-Y gastric bypass (LRYGBP) is the most effective surgical therapy for morbid obesity. It is an advanced laparoscopic surgical procedure and has a protracted learning curve. Therefore, it is important to develop innovative ways of training and assessing surgeons. The aim of this study is to determine if a cadaveric porcine jejuno-jejunostomy model is an accurate way of assessing a surgeon's technical skills by determining if a correlation exists with how he performs in the operating room. Eight surgeons of varying experience performed a side-to-side stapled jejuno-jejunostomy on a cadaveric bench model before proceeding to perform the procedure on a real patient scheduled for LRYGBP. Performance was assessed using a motion tracking device, the Imperial College Surgical Assessment Device. Each procedure was recorded in video and scored by two blinded expert surgeons using procedure-specific rating scales. The cadaveric bench model demonstrated concurrent validity with significant correlations between performance on the cadaveric model and patient for dexterity measures. Left-hand path length, r = 0.857 (median, 27, 41.3; P = 0.007), right-hand path length, r = 0.810 (median, 31.5, 60; P = 0.015) and total number of movements, r = 0.743 (median, 422, 637; P = 0.035). This correlation in performance was also demonstrated in the video rating scales, r = 0.727 (median, 13.2, 14.8; P = 0.041). No correlation was found in operative time (median, 541, 742; P = 0.071). This study demonstrates the concurrent validity of the cadaveric porcine model, showing similar performances in surgeons completing a jejuno-jejunostomy on the cadaveric model and the patient.

Is motion analysis a valid tool for assessing laparoscopic skill?

The use of simulation for laparoscopic training has led to the development of objective tools for skills assessment. Motion analysis represents one area of focus. This study was designed to assess the evidence for the use of motion analysis as a valid tool for laparoscopic skills assessment. Embase, MEDLINE and PubMed were searched using the following domains: (1) motion analysis, (2) validation and (3) laparoscopy. Studies investigating motion analysis as a tool for assessment of laparoscopic skill in general surgery were included. Common endpoints in motion analysis metrics were compared between studies according to a modified form of the Oxford Centre for Evidence-Based Medicine levels of evidence and recommendation. Thirteen studies were included from 2,039 initial papers. Twelve (92.3 %) reported the construct validity of motion analysis across a range of laparoscopic tasks. Of these 12, 5 (41.7 %) evaluated the ProMIS Augmented Reality Simulator, 3 (25 %) the Imperial College Surgical Assessment Device (ICSAD), 2 (16.7 %) the Hiroshima University Endoscopic Surgical Assessment Device (HUESAD), 1 (8.33 %) the Advanced Dundee Endoscopic Psychomotor Tester (ADEPT) and 1 (8.33 %) the Robotic and Video Motion Analysis Software (ROVIMAS). Face validity was reported by 1 (7.7 %) study each for ADEPT and ICSAD. Concurrent validity was reported by 1 (7.7 %) study each for ADEPT, ICSAD and ProMIS. There was no evidence for predictive validity. Evidence exists to validate motion analysis for use in laparoscopic skills assessment. Valid parameters are time taken, path length and number of hand movements. Future work should concentrate on the conversion of motion data into competency-based scores for trainee feedback.

Does previous laparoscopic experience improve ability to perform single-incision laparoscopic surgery?

Single-site laparoscopic surgery (SSLS) has been suggested as a safe and less invasive alternative to standard laparoscopic surgery (LAP). It is not clear whether previous laparoscopic experience influences the ability to perform SSLS. This study aimed to assess the impact of laparoscopic experience on the performance of SSLS. For this study, 18 surgeons were recruited including 12 novice surgeons, four experienced laparoscopic surgeons (intermediate) and two experienced SSLS surgeons (expert). All these subjects completed four tasks from the validated Fundamentals of laparoscopic surgery (FLS) curriculum. The tasks were performed using both a LAP and an SSLS approach with a randomized crossover design. Assessment of the tasks was performed with standardized FLS metrics and dexterity analysis using the Imperial college surgical assessment device. The novice group performed two tasks (precision cutting and intracorporeal suture) significantly better with a LAP approach than with an SSLS approach in all parameters (P < 0.05). The two other tasks (peg transfer and endoloop) were performed significantly better with LAP than with SSLS in terms of time and dexterity only (P < 0.05) but not in terms of error score. The intermediate and expert groups demonstrated no significant difference between their LAP and SSLS performances for any of the tasks in any parameter. Intergroup analysis of performance demonstrated construct validity of the SSLS tasks, with significant differences between novice and intermediate performances for three tasks (peg transfer, endoloop, and intracorporeal suture) (P < 0.05) and between novice and expert performances for three tasks (peg transfer, precision cutting, and intracorporeal suture) (P < 0.05). This study demonstrated that previous laparoscopic experience improves ability to perform SSLS tasks. Some SSLS tasks do not show construct validity due to the complexity of the SSLS technique. It also is implied that current LAP technical skills training curricula are insufficient for teaching SSLS.

Clarifying Assumptions about Intraoperative Stress during Surgical Performance: More Than a Stab in the Dark: Reply

We thank Dr. Ali for his concise annotation of our efforts to validate a tool that evaluates mental workload in surgery [1, 2]. Unlike other safety critical domains, the field of surgery has been slow to acknowledge the impact of intraoperative stress on surgical performance, but recently a sea change has been triggered by authorities in the field of surgical education [3]. We agree with Ali that stress is not by default detrimental to performance. Our aim was to develop a diagnostic tool that identifies the factors that contribute to disrupted performance, should it occur. Indeed, studies of the effects of acute stress on operating performance have shown considerable variability, ranging from no effect to either facilitative or debilitative effects [3–5]. The Yerkes-Dodson law emerged from the earliest attempts to explain the relationship between physiological arousal and performance, but it has been criticized for treating stress as a unitary construct, influenced solely by physiological factors [6]. More recently, Catastrophe Theory has been invoked to model the relationship, using both physiological and psychological (cognitive anxiety) components of stress [7]. The model proposes that physiological arousal displays a mild inverted-U relationship with performance when cognitive anxiety is low, but that catastrophic declines in performance can occur if both physiological arousal and cognitive anxiety are high. Recent surgical literature has elucidated the complexity of stress as a construct and signaled the need for a multidimensional approach to the examination of stress in surgery [3]. The importance of investigating both subjective evaluations of stress and objective measures of performance under intraoperative conditions is evident when the workload measures from our study [2] are compared with the performance and stress measures that we used in a previous study [5]. In the previous study, we investigated laparoscopic peg transfer performance under time pressure, multitasking, and evaluation apprehension conditions using a multidimensional approach. That is, we included objective measures of motor performance (Imperial College Surgical Assessment Device; ICSAD) and both subjective (state anxiety) and objective (heart rate) measures of stress. Time pressure was perceived as most stressful, yet performance was maintained at baseline levels. In contrast, multitasking was not perceived as stressful, yet performance was disrupted [5]. Mental workload is an extremely useful construct when the intention is to examine the mechanisms underpinning such effects, as performance is governed by interacting demands of the task, circumstances in which the task is completed, and the skills, propensities, and perceptions of the individual performing that task [2]. Consequently, while multitasking was not perceived as stressful in our earlier study, it was rated subjectively as causing the highest task complexity, mental demands, and distraction on our SURG-TLX measure [2]. Based on these findings, it is not enough simply to reveal that a particular intraoperative condition caused a surgeon objectively to perform worse (as suggested by Ali); without clues to why performance was affected, prevention of potential catastrophes in the future is a “stab in the dark.” Ali makes a valid point that individuals react differently to stress. For example, a surgeon who feels that he or she has sufficient resources to meet the demands of a situation may view stress in a positive, “challenge” state, but a surgeon who feels dispossessed of the resources required to meet the demands of the situation may view stress in a negative, “threat” state [8]. We have previously shown that individual differences in trait anxiety [9] or the propensity for reinvestment [10, 11] have a significant influence on how efficiently an individual performs a motor task when under evaluative pressure, and recently we have begun to examine these questions in the surgical performance context.

Hand Motion Analysis Using the Imperial College Surgical Assessment Device

The Imperial College Surgical Assessment Device (ICSAD) has been validated in various settings as an objective tool to measure technical performance. We sought to establish (1) the construct validity of the ICSAD as an assessment tool in ultrasound-guided supraclavicular block by determining its ability to discriminate between operators of different experience level and (2) the concurrent validity of the ICSAD by correlating it with a task-specific checklist and a global rating scale. We compared 30 performances of ultrasound-guided supraclavicular block by junior residents with 30 performances by highly experienced consultant anesthesiologists. We also studied 10 anesthesiologists undertaking a 1-year regional anesthesia fellowship and compared a performance in their first month to one in their last 3 months. We used the ICSAD to measure 3 dexterity parameters during the scanning and needling phases of each block: time taken, number of movements, and path length traveled by each hand. Two blinded expert observers evaluated video recordings of each block using a 30-item task-specific checklist and a 7-item global rating scale. Consultants (experts) performed significantly better than residents (novices) on all ICSAD parameters in both scanning and needling phases. Fellows demonstrated improvement in all ICSAD parameters between their early and late performance, reflecting their transition from novice to expert. The task-specific checklist and global rating scale were also highly discriminating between novice and expert performances. There was excellent correlation between all 3 measurement tools, thereby establishing their concurrent validity. The ICSAD is both valid and useful in assessing performance of ultrasound-guided supraclavicular block.

Effect of sleep deprivation on labour epidural catheter placement

Epidural catheter insertion for labour analgesia is an invasive procedure with potential serious complications, often performed by a sleep-deprived clinician. The aim of this study was to examine the effects of sleep deprivation on physicians of variable levels of experience performing this procedural skill in the clinical setting. After institutional review board approval, anaesthetists of three levels of experience were recruited: novice residents (<30 epidurals, n=9), experienced residents (>100 epidurals, n=8), and attending anaesthetists (>500 epidurals, n=12). All participants were measured twice, rested and sleep deprived in a random order while performing a labour epidural for analgesia. Our primary outcome measures were scores achieved on the Imperial College Surgical Assessment Device (ICSAD) (measuring path length, number of movements, and time), task-specific checklist (CL), and global rating scale (GRS). Sleep deprivation was documented by the ActiGraph and Epworth sleepiness scale. Subjects were adequately sleep deprived for their sleep deprivation observation. Data were analysed with a two-way mixed design analysis of variance. No significant difference in the effect of sleep deprivation on performance was detected between the groups on the ICSAD measures of movement (P=0.86), path length (P=0.79), and time (P=0.80), or for the CL (P=0.65), and GRS (P=0.86). The performance of this procedural skill in a clinical setting does not seem to be affected by sleep deprivation irrespective of the level of experience.

Validation of the Imperial College Surgical Assessment Device (ICSAD) for labour epidural placement

Technical proficiency in anesthesia has historically been determined subjectively. The purpose of this study was to establish the construct validity for the Imperial College Surgical Assessment Device (ICSAD), a measure of hand motion efficiency, as an objective assessment tool for technical skill performance, by examining its ability to distinguish between operators of different levels of experience performing a labour epidural. Concurrent validity for the ICSAD was investigated by comparison to a validated task specific checklist (CL) and global rating scale (GRS). A single blinded, prospective, controlled study design compared three groups of subjects: novice residents (<30 epidurals), experienced residents (>100 epidurals), and staff anesthesiologists (>500 epidurals). Performance was measured using the ICSAD (number of movements, path length, time) and scores from a CL and GRS graded by examiners blinded to the level of training. Data were analyzed by multivariate analysis of variance (MANOVA). Twenty-nine subjects were recruited. Novice residents had longer path lengths compared to experienced residents (P = 0.031) and staff anesthesiologists (P = 0.0004), made more movements (P = 0.012) and took more time than staff (P = 0.009). Novice residents scored significantly worse on the GRS compared to experienced residents (P = 0.029) and staff (P = 0.01) and had significantly lower CL scores compared to staff (P = 0.003). Construct and concurrent validity for the ICSAD was established for a regional anesthesia technique by demonstrating that it can distinguish between operators of different levels of experience and by comparing it to the current standards of technical skill assessment.

Skills Acquisition and Assessment after a Microsurgical Skills Course for Ophthalmology Residents

To assess the impact of a skills course on microsurgical skills acquisition and to investigate the validity of a video-based modified Objective Structured Assessment of Technical Skill (OSATS) assessment tool that has not previously been applied to ophthalmic surgery. Prospective longitudinal cohort study. Fourteen residents were recruited from 20 attendees at the Moorfields Eye Hospital microsurgical skills course for residents. Each resident performed a standardized microsurgical task consisting of the placement of a 10-0 nylon corneal suture into a model eye using an operating microscope with standardized equipment in a standardized environment. Objective measurements were made using the Imperial College Surgical Assessment Device (ICSAD). This is a motion-tracking device returning 3 parameters for economy of movement: total path length, time, and number of individual hand movements. A concurrent video recording was made of each task by 2 independent observers who were masked to the time of the recording relative to the course and the identity of the resident. Video footage was marked in accordance with the OSATS video scoring template. Each resident had motion-tracking analysis performed during corneal suturing before and after the course (total path length, time, and number of individual hand movements), along with concurrent OSATS video scores. Skills improvement after the course was found to be statistically significant for all 3 ICSAD economy of movement parameters: path length, P = 0.001; hand movements, P = 0.012; and time, P = 0.009. Differences in the combined OSATS scores of the 2 raters before and after the course were found to be significant (P = 0.039). Interrater reliability of OSATS scorers was 0.78 (alpha Cronbach). Correlations between the OSATS scores and each of the ICSAD parameters were found to be significant (P<0.001). A video-based OSATS scoring system has significant correlation with the ICSAD motion-tracking parameters, demonstrating concurrent validity between the 2 assessment tools. These data also demonstrate that surgical skill, as measured by a validated motion-tracking system, is significantly improved after a 1-day microsurgical skills course. The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Simulation in laparoscopic surgery: a concurrent validity study for FLS

Current assessments using the fundamentals of laparoscopic surgery (FLS) tasks are labour intensive and depend heavily on expert raters. Hand motion analysis may offer an alternative method of objective evaluation of FLS performance. The aim of this study was to assess whether a correlation exists between the expert rated assessments of the FLS tasks and computer-based assessment of motion efficiency using the Imperial College surgical assessment device (ICSAD). We recruited 26 volunteer subjects who were stratified into three experience groups: juniors (PGY1-3) (N = 13), seniors (PGY4,5) (N = 7) and staff surgeons (N = 6). All subjects performed four FLS tasks: (1) peg transfer, (2) pattern cut, (3) endoloop and (4) intracorporeal suturing. Performance was assessed by both standard FLS expert rating and motion analysis using ICSAD. Group differences were analyzed using the Kruskal-Wallis test, and Spearman coefficient analyses were employed to compare FLS and ICSAD scores. FLS expert-derived scores discriminated effectively between experience groups for all tasks (p < 0.05). Motion efficiency scores discriminated between experience groups for tasks 1, 3 and 4 for number of movements (p < 0.05), tasks 1 and 4 for total distance (p < 0.05) and tasks 1, 2, 3 and 4 for total time (p < 0.005). There was a significant correlation between total FLS expert scores and the motion efficiency metrics of total distance, number of movements and total time (Spearman coefficient and p values of 0.81, < 0.001; 0.76; < 0.001; and 0.93, < 0.001, respectively). There is a high correlation between FLS standard scoring and motion efficiency metrics. The use of ICSAD for the objective assessment of FLS tasks may in the future offer an adjunctive method of evaluation. ICSAD metrics are potentially less labour intensive due to the instant and fully automated computerized scoring that it provides.

Validation of orthopaedic bench models for trauma surgery

The aim of this study was to validate the use of three models of fracture fixation in the assessment of technical skills. We recruited 21 subjects (six experts, seven intermediates, and eight novices) to perform three procedures: application of a dynamic compression plate on a cadaver porcine model, insertion of an unreamed tibial intramedullary nail, and application of a forearm external fixator, both on synthetic bone models. The primary outcome measures were the Objective Structural Assessment of technical skills global rating scale on video recordings of the procedures which were scored by two independent expert observers, and the hand movements of the surgeons which were analysed using the Imperial College Surgical Assessment Device. The video scores were significantly different for the three groups in all three procedures (p < 0.05), with excellent inter-rater reliability (alpha = 0.88). The novice and intermediate groups specifically were significantly different in their performance with dynamic compression plate and intramedullary nails (p < 0.05). Movement analysis distinguished between the three groups in the dynamic compression plate model, but a ceiling effect was demonstrated in the intramedullary nail and external fixator procedures, where intermediates and experts performed to comparable standards (p > 0.6). A total of 85% (18 of 21) of the subjects found the dynamic compression model and 57% (12 of 21) found all the models acceptable tools of assessment. This study has validated a low-cost, high-fidelity porcine dynamic compression plate model using video rating scores for skills assessment and movement analysis. It has also demonstrated that Synbone models for the application of and intramedullary nail and an external fixator are less sensitive and should be improved for further assessment of surgical skills in trauma. The availability of valid objective tools of assessment of surgical skills allows further studies into improving methods of training.

Objective Assessment of a Bimanual Technique for Access to the Nasal Cavity and Anterior Skull Base

Introduction: Access to the nasal cavity and anterior skull base is often achieved using an endoscopic technique. The current “orthodox” technique preferred by many otolaryngologists is one-handed: the surgeon holds the endoscope in one hand and uses an instrument in the other hand. This technique does have its limitations, particularly when bleeding may cause obstruction of the operative view. Authors have described the use of two surgeons or a camera holder to enable bimanual surgery in the nasal cavity and anterior skull base. A novel method of measuring surgical efficiency has been developed by the Imperial College London, using real-time dexterity motion analysis to compare the two operative methods with statistical significance. The Imperial College Surgical Assessment Device (ICSAD) had never previously been used in any skull base or otolaryngology procedure.

Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills

Background Teaching of technical surgical skills to undergraduate medical students in a laboratory setting away from the patient is not common practice. Because of the large volume of students and shortage of available teaching faculty new methods of teaching must be developed for this group of trainees. In this study we examined the effectiveness of computer-based video training, different types of computer-based motion efficiency feedback (with and without expert criteria), and expert feedback on learning of a basic technical skill in medical students. Methods Forty-five junior medical students were randomized into 3 groups and learned suturing and knot-tying skills. Group A received computer-generated feedback about the economy of their movements. Group B received the same motion economy feedback, as well as expert reference values. Group C received verbal feedback from an expert. All groups were pre-tested, allowed 18 practice trials, and post-tested, and their skill retention was retested after 1 month. Performance was assessed by expert analysis using an objective structured analysis of technical skill and by computer analysis (Imperial College Surgical Assessment Device [ICSAD]). Results All groups showed improvement from pre-test to post-test. However, only group C showed retention of skill on delayed performance testing. Conclusions Verbal feedback from an expert instructor led to lasting improvements in technical skills performance. Providing information about motion efficiency did not lead to similar improvements.