Task-specific Checklist Research Articles

Systematic instruction of arthroscopic knot tying with the ArK Trainer: an objective evaluation tool

A Proficiency Formula was introduced as an objective self-evaluation method for evaluating basic arthroscopic knot tying in a laboratory setting. The correlation between the Proficiency Formula and gold standard pass/fail dichotomy was demonstrated, as well as with other popular evaluation tools--task-specific checklist (TSC) and global rating scale (GRS). A step-by-step video tutorial was used to instruct 35 medical students on how to tie an arthroscopic Samsung Medical Center (SMC) knot secured by three half hitches. Participants were video recorded performing arthroscopic knot tying and assessed on their success tying an SMC knot, pass or fail, and through three outcome tools: the Proficiency Formula, GRS and the TSC. Independent samples t test was used to compare the GRS, TSC and Proficiency Formula scores, between those who were passed or failed by the evaluators. Correlation between the measurement scales was tested using Spearman's rho correlation coefficient. Participants received a mean proficiency score of 195 (140-249). The mean Proficiency score for those that passed was 323 (95 % CI 272-374), for those that failed, 87 (95 % CI 26-148, p < 0.001). We found strong linear correlation between the Proficiency Formula and GRS and TSE (0.83 and 0.78, respectively). The Proficiency Formula has high correlation with gold standard GRS and TSC measurements when used to assess arthroscopic knot tying skills on a model. It has the added advantage of being able to be self-assessed.

Simulation-based training to teach open abdominal aortic aneurysm repair to surgical residents requires dedicated faculty instruction

We assessed the impact of abdominal aortic aneurysm (AAA)-specific simulation training on resident performance in simulated open AAA repair (SOAAAR) and determined whether simulation training required dedicated faculty instruction. We randomized 18 residents (postgraduate years 3-5) to an AAA simulation course consisting of two mandatory practice sessions proctored either by a surgical skills lab coordinator (Group A, n = 8) or by a vascular surgery faculty instructor (Group B, n = 10). All residents received a detailed manual and video demonstrating the technique of open AAA repair. Using a validated tool, vascular faculty who were blinded to resident identity, level of training, and randomization status graded SOAAAR performance via videos that were recorded before and after the course. Characteristics and baseline scores between Groups A and B were not different. Postcourse, there was a no significant improvement in performance in Group A. Group B performance was improved significantly from baseline with regard to task-specific checklist scores (44.1 ± 6.3 vs 34.9 ± .5; P = .02), global rating scores (28.4 ± .6 vs 25.3 ± 5.0; P = .049), and overall assessment of operative competence (P = .02). Time to complete SOAAAR improved in both groups (P = .02). Baseline performance varied significantly with year of training as measured by task-specific checklist scores, global rating scores, final product analysis, time to complete repair, and overall operative competence. Improvement varied inversely with year of training (P < .05) and postcourse scores were equivalent for postgraduate year 3-5 residents. An AAA-specific simulation training course improved resident performance in simulated open AAA repair. Dedicated faculty instruction during the simulation training was required for significant improvement in resident performance. The impact of simulation training was greatest in more junior residents. Procedure-specific simulation training with dedicated faculty can be used to effectively teach simulated open AAA repair.

Quantitative Assessment of Surgical Competence: The Arthroscopic Learning Curve (SS-58)

To assess orthopaedic trainees performing consecutive knee and shoulder arthroscopies and evaluate procedural competence using a novel statistical method. Subjects included orthopaedic trainees in post-graduate year (PGY) 1 through 5 (n = 22) performing diagnostic knee or shoulder arthroscopy during a 3-month sports rotation. Intraoperative evaluations were completed by supervising staff physicians using a validated 10- (knee) or 14-point (shoulder) Task-Specific Checklist (TSCL) and 50-point Global Rating Scale (GRS). For the Cumulative Summation Test for Learning Curve (LC-CUSUM) analysis, a TSCL < 8 (knee) or 11 (shoulder) and GRS < 40 were considered a failure with a maximum of 10% poor procedures defined as adequate performance. Performance limits included acceptable deviation of 10% from adequate performance, a true discovery rate of 90% and a false discovery rate of 90%. The LC-CUSUM was then plotted to assess the learning phase for individual trainees. A total of 433 (340 knee, 93 shoulder) consecutive procedures were performed during the 12-month study period. Competency in knee arthroscopy for the TSCL was achieved by 8 trainees (n = 17) requiring 13 to 24 procedures and a single trainee for the GRS requiring 14 procedures. All trainees (n = 5) failed to demonstrate competency for the TSCL and GRS in shoulder arthroscopy after a mean of 18.6 procedures. Threshold adjusted curves stratified by PGY level enabled multiple trainees to achieve competency in both knee and shoulder diagnostic arthroscopy. The LC-CUSUM can be successfully applied to knee and shoulder arthroscopy to provide an individualized assessment of performance and quantitatively demonstrate competency. Clinical Relevance: The LC-CUSUM is an effective method to evaluate procedure competence in arthroscopic training and can provide objective feedback and benchmarks in the learning phase.

A Comparison of Orthopaedic Resident Performance on Surgical Fixation of an Ulnar Fracture Using Virtual Reality and Synthetic Models

Surgical trainees develop surgical skills using various techniques, with simulators providing a safe learning environment. Fracture fixation is the most common procedure in orthopaedic surgery, and residents may benefit from simulated fracture fixation. The performance of residents on a virtual simulator that allows them to practice the surgical fixation of fractures by providing a sense of touch (haptics) has not yet been compared with their performance using other methods of practicing fracture fixation, such as a Sawbones simulator model. The purpose of this study was to assess whether residents performed similarly on a newly developed virtual simulator compared with a Sawbones simulator fracture fixation model. A stratified, randomized controlled study involving twenty-two orthopaedic surgery residents was performed. The residents were randomized to first perform surgical fixation of the ulna on either the virtual or the Sawbones simulator, after which they performed the same procedure on the other simulator. Their performance was evaluated by examiners experienced in fracture fixation who completed a task-specific checklist, global rating scale (GRS) form, and time-to-completion record for each participant on each simulator. Both simulators distinguished between differing experience levels, demonstrating construct validity; for the Sawbones simulator, the Cohen d value (effect size) was &gt;0.90, and for the virtual simulator, d was &gt;1.10 (p &lt; 0.05 for both). The participants achieved significantly better scores on the virtual simulator compared with the Sawbones simulator (p &lt; 0.05) for all measures except time to completion. The GRS scores showed a high level of internal consistency (Cronbach α, &gt;0.80). However, Pearson product-moment correlation analysis showed no significant correlations between the results on the two simulators; therefore, concurrent validity was not achieved. The newly developed virtual ulnar surgical fixation simulator, which incorporates haptics, shows promise for helping surgical trainees learn and practice basic skills, but it did not attain the same standards as the current standard Sawbones simulator. The procedural measures used to assess resident performance demonstrated good reliability and validity, and both the Sawbones and the virtual simulator showed evidence of construct validity.

Assessment of improvement of trainee surgical skills in the operating room for tonsillectomy

To determine the feasibility, reliability, and validity of an objective surgical assessment tool designed to measure the development of tonsillectomy skills by resident trainees in the operating room. A tonsillectomy evaluation instrument created previously serves as a basis for this tool, which incorporates many similar concepts but was optimized to maximize feasibility and reliability with simplified anchors and inclusion of global and task-specific sections. Prospective longitudinal validation study. Faculty input via modified Delphi technique was used to develop a new objective structured assessment of technical skills-based instrument for tonsillectomy. The task-specific checklist (TSC) and global rating scale (GRS) measured tonsillectomy technical skills and overall surgical performance, respectively. Twenty-one otolaryngology-head and neck residents (ranging from postgraduate year 1 to 6) were evaluated for a period of 3 years by 11 faculty members. Eighty-three evaluations were completed showing strong correlation between both instruments (r = 0.95, P < .001). Our tool demonstrated construct validity for both TSC and GRS, showing higher scores with increasing surgical experience. Both instruments showed high interitem reliability with Cronbach α coefficients of 0.97 for both parts. This assessment tool is a feasible, reliable, and valid instrument for the assessment of surgical competency in tonsillectomy. It is effective in providing structured feedback at the end of each procedure, which encourages specific, targeted development.

Multimedia-based training on Internet platforms improves surgical performance: a randomized controlled trial

BackgroundSurgical procedures are complex motion sequences that require a high level of preparation, training, and concentration. In recent years, Internet platforms providing surgical content have been established. Used as a surgical training method, the effect of multimedia-based training on practical surgical skills has not yet been evaluated. This study aimed to evaluate the effect of multimedia-based training on surgical performance.MethodsA 2 × 2 factorial, randomized controlled trial with a pre- and posttest design was used to test the effect of multimedia-based training in addition to or without practical training on 70 participants in four groups defined by the intervention used: multimedia-based training, practical training, and combination training (multimedia-based training + practical training) or no training (control group). The pre- and posttest consisted of a laparoscopic cholecystectomy in a Pelvi-Trainer and was video recorded, encoded, and saved on DVDs. These were evaluated by blinded raters using a modified objective structured assessment of technical skills (OSATS). The main evaluation criterion was the difference in OSATS score between the pre- and posttest (ΔOSATS) results in terms of a task-specific checklist (procedural steps scored as correct or incorrect).ResultsThe groups were homogeneous in terms of demographic parameters, surgical experience, and pretest OSATS scores. The ΔOSATS results were highest in the multimedia-based training group (4.7 ± 3.3; p < 0.001). The practical training group achieved 2.5 ± 4.3 (p = 0.028), whereas the combination training group achieved 4.6 ± 3.5 (p < 0.001), and the control group achieved 0.8 ± 2.9 (p = 0.294).ConclusionMultimedia-based training improved surgical performance significantly and thus could be considered a reasonable tool for inclusion in surgical curricula.

The use of an online three-dimensional model improves performance in ultrasound scanning of the spine: a randomized trial

The use of ultrasound for neuraxial blockade is a new application of technology that is rapidly becoming accepted as a standard of care. This new skill has shown to improve success, but it is a challenge to teach. To assist with teaching the use of ultrasound in regional anesthesia of the lumbar spine, we have developed an interactive educational model ( http://pie.med.utoronto.ca/vspine or http://www.usra.ca/vspine.php ). In this study, we aimed to determine whether use of this model for a two-week period would improve the performance of novice operators in determining defined landmarks during real-time ultrasound imaging of the lumbar spine. We evaluated the educational benefit of the ultrasound module by randomly assigning 16 postgraduate first-year (PGY1) anesthesia residents to either a control group with password-protected access to only the lumbar anatomy module or to an intervention group with access to the complete module. All residents had access to the module for two weeks following a full-day workshop that is part of the university teaching program which consists of a didactic lecture on ultrasound-facilitated neuraxial anesthesia, mentored teaching on cadaveric spine dissections, and hands-on ultrasound scanning of live models. At the end of the two weeks, the performance of the residents was evaluated using a 12-item task-specific checklist while carrying out a scout scan on a live model. The control group had a median score of 5.5 (25(th) percentile: 4, 75(th) percentile: 18), while the intervention group had a median score of 11.5 (25(th) percentile: 8, 75(th) percentile: 12) in the task-specific checklist, with a significant difference of 6 (confidence interval 1.5 to 10.5) between groups (P = 0.021). Our results show superior performance by the residents who had access to both components of the module, indicating that access to the interactive ultrasound spine module improves knowledge and skills prior to clinical care.

Reliability of results produced through objectively structured assessment of technical skills (OSATS) for endotracheal intubation (ETI).

To estimate the reliability of an objectively structured assessment of technical skills (OSATS) in patients for Endotracheal Intubation (ETI), using task-specific checklists during administration of general anaesthesia. Co-relational reliability study. Department of Anaesthesia and Medical Education, Pakistan Institute of Medical Sciences, Islamabad, from September to November 2010. Ten first year residents of the department of anaesthesia who had successfully completed a one-day course of ETI on mannequins and had performed 25 ETI in actual patients under supervision, were asked to perform ETI on anaesthetized patients in the operation theater after consent. The procedure was directly observed by two expert observers (ER1 and ER2) and one inexperienced observer (novice, NR) using a validated task oriented checklist. Each trainee repeated the procedure on another patient and was reassessed using the same checklist. Inter-rater for test and retest, and expert to novice reliabilities along with internal consistency were calculated. The inter-rater reliability was 0.92 in test and was 0.98 in retest (Ebel's calculation). Spearman Rank Correlation Coefficient (SRCC) for ER 1 and NR was 0.76 in test and 0.89 in retest. Cronbach's alpha for internal consistency was 0.93 for first test and 0.97 for retest. The OSATS for ETI in real patients demonstrated excellent inter-rater test reliability, inter-rater retest reliability, expert-novice reliability and internal consistency. This instrument, therefore, proves promising as a tool for competency-based evaluations.

Preliminary evaluation of a virtual reality-based simulator for learning spinal anesthesia

Study ObjectiveTo evaluate the influence of a simulation-based program on the initial performance of dural puncture by medical interns, and to refine the design of simulator-based teaching and competence assessment. DesignProspective interventional study. SettingAcademic medical center. Subjects27 medical interns inexperienced in the technique of spinal anesthesia or dural puncture and within 12 months of graduating from medical school, were randomly assigned to a conventional or a simulator-based teaching course of spinal anesthesia: 13 were recruited to the Conventional Group (CG) and 14 to the Simulator Group (SG). MeasurementsA SenseGraphic Immersive workbench and a modified Phantom desktop with shutter glasses were used to create a teaching environment. Outcomes of teaching were assessed in two phases within three weeks of the teaching course: Phase I consisted of a written examination followed by assessment on the simulator. A global rating scale and a task-specific checklist were used. Phase II (for those participants for whom a suitable opportunity arose to perform spinal anesthesia under supervision within three wks of the teaching course) consisted of structured observation of clinical performance of the procedure in the operating room. Participants were assessed by independent, study-blinded experts. Student's two-tailed impaired t-tests were used to compare the parametric outcomes (P < 0.05 was considered significant). Main ResultsAll participants completed the written test successfully with no difference between groups. Ten participants from CG and 13 from SG completed the simulator-based testing performing similarly in terms of the global rating scale. Five participants in CG and 6 in SG proceeded to clinical testing. On the global rating scale, interns in SG scored higher than those in CG. They performed similarly according to the task-specific checklist. ConclusionsOverall, no difference was measured between those taught with traditional methods and those, by a simulator based program in regard to the performance of spinal anesthesia.

Validation of a task‐specific scoring system for a microvascular surgery simulation model

Simulation models can help develop procedural skills outside the clinical setting while also providing a means for evaluation of trainees. Objective Structured Assessment of Technical Skills (OSATS) have been developed for several procedures. The purpose of this study was to demonstrate the construct validity of an OSATS for microvascular anastomosis performed on a simulation model using chicken thigh vessels. Validation study. An expert panel constructed a task-specific checklist for an OSATS for microvascular anastomosis. Twenty surgical staff and trainees performed a microvascular anastomosis of a chicken ischiatic artery. Training level and microsurgical experience were assessed by questionnaire. The performances were recorded and scored by two experts using the task-specific and global scales of the OSATS. Analysis of variance revealed a significant effect of training and microvascular experience for both the task-specific score and global rating scale score (P < .005). Interrater reliability was 0.7. Experience level demonstrated a logarithmic relationship with task time. The microvascular OSATS applied to the chicken thigh simulator model differentiated between levels of microvascular experience. It demonstrated construct validity and reliability for the assessment of procedural competence using a cost-effective and easily accessible model.

A curriculum to teach and evaluate resident skills in the management of postpartum hemorrhage

The aim of this study was the development of a curriculum for the management of postpartum hemorrhage and an objective assessment of technical skills (OSATS) for uterine compression sutures. Twenty-two residents participated in the study. Evaluations included the global rating scale, task-specific checklist, and pass/fail rating for the OSATS and task-specific checklist for the hemorrhage drill. The reliability and validity of the evaluation tools were calculated. The inter-rater reliability was 0.98 for the hemorrhage drill checklist, 0.77 for the global rating scale, and 0.93 for the uterine suture checklist. Construct validity evaluation revealed senior residents performed superiorly to junior residents on the global rating scale (P=0.006) and on the uterine suture checklist (P=0.04). There was an improvement in performance on the post-test in comparison to the pretest (P=0.001). We present an inexpensive, reliable, and valid curriculum to teach and evaluate the medical and surgical management of postpartum hemorrhage.

Simulating Various Levels of Clinical Challenge in the Assessment of Clinical Procedure Competence

Immersive simulation is increasingly used for competency assessment of emergency physicians. This group's concept of hybrid simulation (HS) (combining simulated patients and part-task trainers (a simulator that simulates a limited component of a clinical procedure) to create a multimodal clinical context) requires clinicians to integrate technical and nontechnical skills in a holistic clinical performance for assessment. It also offers the potential to provide authentic simulation of a given clinical procedure across multiple levels of challenge. The aims of this study are to systematically design and validate 2 patient-focused HS scenarios (each combining a simulated patient with a part-task simulator) for assessment of the management of a commonly encountered problem in an emergency department (ED) at different levels of clinical challenge, and to explore the effect of level of challenge of the HS scenario on physicians' performance. A simple (HS1) and a complex (HS2) HS scenario (based on the management of a patient with a traumatic skin laceration within the ED) was developed according to expert opinion through cognitive task analysis. Interns and emergency medicine residents (stratified into expert and novice groups according to experience) were recruited to participate in both scenarios. Participants were randomized to perform either the HS1 or HS2 scenario first. Participants completed a questionnaire for face validity (realism of simulation) and content validity (comprehensiveness of simulation). Performances were assessed by 2 independent raters using validated rating tools modified to the needs of this study: the Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, the Objective Structured Assessment of Technical Skills-Global Rating Score, and the Direct Observation of Procedural Skills. Ten novice and 10 expert clinicians completed both scenarios. Mean face and content validity ratings were high for both HS1 (mean 4.4 [SD 0.52] and 4.2 [SD 0.48], respectively) and HS2 scenarios (mean 4.5 [SD 0.35] and 4.3 [SD 0.43], respectively). In HS1, no difference was found between experts' and novices' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, Objective Structured Assessment of Technical Skills-Global Rating Score, and Direct Observation of Procedural Skills ratings. Experts performed significantly better than novices in HS2 in terms of the 3 tools' ratings. Novices' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist and Direct Observation of Procedural Skills ratings were significantly worse in HS2 compared with HS1, but no difference was found with the Objective Structured Assessment of Technical Skills-Global Rating Score. No statistical difference was found in experts' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, Objective Structured Assessment of Technical Skills-Global Rating Score, and Direct Observation of Procedural Skills ratings between HS2 and HS1 scenarios. Recreating clinical challenge is an important consideration in the design of simulation-based assessment of procedural skills of clinicians. In this study, we have demonstrated a systematic approach to developing HS scenarios, which may be able to recreate various levels of clinical challenge for purpose of assessment of procedural skills.

1989 INTRA-OPERATIVE ASSESSMENT OF MICROSURGICAL SKILL USING HAND MOTION ANALYSIS ESTABLISHING LEARNING CURVES OF SURGICAL COMPETENCE DURING VASECTOMY REVERSAL

You have accessJournal of UrologyInfertility: Therapy1 Apr 20121989 INTRA-OPERATIVE ASSESSMENT OF MICROSURGICAL SKILL USING HAND MOTION ANALYSIS ESTABLISHING LEARNING CURVES OF SURGICAL COMPETENCE DURING VASECTOMY REVERSAL Ethan Grober, Majid Fanipour, Matthew Roberts, and Uma Victor Ethan GroberEthan Grober Toronto, Canada More articles by this author , Majid FanipourMajid Fanipour Toronto, Canada More articles by this author , Matthew RobertsMatthew Roberts Ottawa, Canada More articles by this author , and Uma VictorUma Victor Toronto, Canada More articles by this author View All Author Informationhttps://doi.org/10.1016/j.juro.2012.02.2150AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail INTRODUCTION AND OBJECTIVES Within urology and other surgical disciplines, intra-operative assessment of technical skill remains largely unvalidated and subjective. In the training laboratory, measures of economy of hand motion have demonstrated promise as reliable, valid and objective measures of technical competence. The primary objective of the current study was to validate live, intra-operative assessments of economy of hand motion as an objective measure of microsurgical skill and to use hand-motion analysis to establish competency-based surgical learning curves. METHODS Economy of hand-motion (number of hand movements and hand travel distance) was serially evaluated on a standardized, live-patient, intra-operative microsurgical suturing task (microsurgical vasovasostomy). Urology residents, clinical fellows and experienced microsurgeon were simultaneously evaluated. Hand-motion analyses were correlated with blinded, case-matched assessments of technical skill using previously validated global rating scales and task-specific checklists applied to unedited surgical videos. Serial hand-motion data from the urology residents, clinical fellows and experienced surgeons acquired over 19 consecutive cases were compared and used to establish competency-based learning curves over time. RESULTS 12 urology residents, 4 fellows and 3 faculty were evaluated. Intra-operative measures of economy of hand-motion correlated significantly with case-matched checklist scores and global ratings of surgical performance (r > 0.75, p < 0.001). Total number of hand movements, hand travel distance and operating time improved significantly over time for the urology residents (p=0.03) and fellows (p=0.02) but remained stable for the experienced surgeons (p=0.5). Meaningful learning curves can be generated to establish competency standards and provide feedback to surgical trainees over time (Figure 1 - Total Hand Movements). CONCLUSIONS Hand motion analysis represents a feasible, objective and valid measure of microsurgical skill while operating on real patients and can be used to establish competency-based surgical learning curves over time. © 2012 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 187Issue 4SApril 2012Page: e802-e803 Advertisement Copyright & Permissions© 2012 by American Urological Association Education and Research, Inc.MetricsAuthor Information Ethan Grober Toronto, Canada More articles by this author Majid Fanipour Toronto, Canada More articles by this author Matthew Roberts Ottawa, Canada More articles by this author Uma Victor Toronto, Canada More articles by this author Expand All Advertisement Advertisement PDF DownloadLoading ...

A clinical assessment tool for ultrasound‐guided axillary brachial plexus block

Competency in anesthesia traditionally has been determined subjectively in practice. Optimal training in procedural skills requires valid and reliable forms of assessment. The objective was to examine a procedure-specific clinical assessment tool for ultrasound-guided axillary brachial plexus block for inter-rater reliability and construct validity in a clinical setting. This was a prospective observational study. Using prior work at our institution, the clinical assessment tool was developed consisting of a 63-point task-specific checklist and a global rating scale. The anesthesiologists were assigned to three groups based on prior experience of performing an axillary block: group 1 ('novices') < 10 procedures, group 2 ('intermediates') 50-80 procedures and group 3 ('experts') > 100 procedures. Each participant performed two consecutive blocks that were videotaped. Two blinded independent experts trained in the marking of the tool evaluated the videotapes. There were five participants per group. The inter-rater reliability between assessors was 0.842 and 0.795 for the checklist and global rating scale, respectively. There was a consistent difference between (each pair of) the three groups in terms of both the checklist and global rating scale (P < 0.05). For the checklist, expert vs. intermediate group P = 0.023, expert vs. novice group P < 0.001 and intermediate vs. novice group P = 0.019. For the global rating scale, expert vs. intermediate group P < 0.001, expert vs. novice group gave P < 0.001 and intermediate vs. novice group P = 0.023. The objective of task-specific checklist and global rating scale are reliable and valid measures of axillary block performance between different levels of expertise.

High Fidelity Simulation Based Team Training in Urology: A Preliminary Interdisciplinary Study of Technical and Nontechnical Skills in Laparoscopic Complications Management

Simulation based team training provides an opportunity to develop interdisciplinary communication skills and address potential medical errors in a high fidelity, low stakes environment. We evaluated the implementation of a novel simulation based team training scenario and assessed the technical and nontechnical performance of urology and anesthesiology residents. Urology residents were randomly paired with anesthesiology residents to participate in a simulation based team training scenario involving the management of 2 scripted critical events during laparoscopic radical nephrectomy, including the vasovagal response to pneumoperitoneum and renal vein injury during hilar dissection. A novel kidney surgical model and a high fidelity mannequin simulator were used for the simulation. A debriefing session followed each simulation based team training scenario. Assessments of technical and nontechnical performance were made using task specific checklists and global rating scales. A total of 16 residents participated, of whom 94% rated the simulation based team training scenario as useful for communication skill training. Also, 88% of urology residents believed that the kidney surgical model was useful for technical skill training. Urology resident training level correlated with technical performance (p=0.004) and blood loss during renal vein injury management (p=0.022) but not with nontechnical performance. Anesthesia resident training level correlated with nontechnical performance (p=0.036). Urology residents consistently rated themselves higher on nontechnical performance than did faculty (p=0.033). Anesthesia residents did not differ in the self-assessment of nontechnical performance compared to faculty assessments. Residents rated the simulation based team training scenario as useful for interdisciplinary communication skill training. Urology resident training level correlated with technical performance but not with nontechnical performance. Urology residents consistently overestimated their nontechnical performance.

Development of the implant surgical technique and assessment rating system

PurposeThere has been no attempt to establish an objective implant surgical evaluation protocol to assess residents' surgical competence and improve their surgical outcomes. The present study presents a newly developed assessment and rating system and simulation model that can assist the teaching staffs to evaluate the surgical events and surgical skills of residents objectively.MethodsArticles published in peer-reviewed English journals were selected using several scientific databases and subsequently reviewed regarding surgical competence and assessment tools. Particularly, medical journals reporting rating and evaluation protocols for various types of medical surgeries were thoroughly analyzed. Based on these studies, an implant surgical technique assessment and rating system (iSTAR) has been developed. Also, a specialized dental typodont was developed for the valid and reliable assessment of surgery.ResultsThe iSTAR consists of two parts including surgical information and task-specific checklists. Specialized simulation model was subsequently produced and can be used in combination with iSTAR.ConclusionsThe assessment and rating system provided may serve as a reference guide for teaching staffs to evaluate the residents' implant surgical techniques.

The effect of low‐fidelity endoscopic sinus surgery simulators on surgical skill

Surgical training models are being increasingly used to provide an environment for surgical trainees to improve their skills without risk to patients. This study uses previously validated, inexpensive, low-fidelity training models to determine how pretraining affects endoscopic sinus surgery (ESS) skills. Fourteen Otolaryngology residents were randomized to 1 of 2 groups that were stratified for training level. The first group took part in a pretraining session where they practiced on all 5 different modules whereas the second group did not receive any pretraining. The following day, all participants took part in a cadaveric ESS course. Participants were instructed to complete a set of tasks and their performances were videotaped. The videos were then evaluated using a Global Rating Scale (GRS) and a Task-Specific Checklist (TSC). The performances of those who trained using the models were compared to the performances of those who did not. The intervention (pretraining) group performed better than the nonintervention (no pretraining) group on the cadaveric ESS tasks (p < 0.05). As well, there was a statistical difference between the senior residents who had the pretraining with the simulator models performing better than those who did not. The modules appear to have made a positive impact on ESS skills. These low-cost, easily-constructed training modules have the potential to be integrated into Otolaryngology-Head and Neck Surgery resident training. Assessment of long-term training effects with a larger number of participants is planned for future studies.

A model for training and evaluation of myringotomy and tube placement skills

Simulation is emerging as a mandatory component of surgical training and a means of demonstrating surgical competency. We designed a cost-effective, low-fidelity model to further acquisition of technical skills related to myringotomy and ventilation tube insertion (M&T). The purpose of the study was to examine the skills trainer as a method of assessment to evaluate competency, timeliness, and procedure confidence in junior residents. Prospective, randomized. A simplistic M&T skills box was developed. General surgery interns (n = 20) with no prior procedure training were randomized to receive either didactic instruction or skills training using the model. One hour of lecture or technical skills training was provided to each group. A blinded examiner evaluated the subjects in both groups before and after training. Performance was measured using a global rating scale, task-specific checklist, and time-to-completion. Pre- and postsession questionnaires assessed procedure confidence. Analysis revealed a trend toward improvement in global rating scores between groups. There was a statistically significant difference in time improvement between groups (P = .0211). The skills lab group felt they could perform the procedure faster and with improved abilities, as compared to the didactic group (P = .0069 and 0.0007, respectively). Junior surgical residents performed an M&T procedure using a novel, low-cost model. This study demonstrated the skills lab's positive effect on training as measured by global rating scale, time-to-completion, and overall resident confidence. We anticipate its application to be valuable not only in training residents but also in assessing competency.

Hand Motion Analysis Using the Imperial College Surgical Assessment Device

The Imperial College Surgical Assessment Device (ICSAD) has been validated in various settings as an objective tool to measure technical performance. We sought to establish (1) the construct validity of the ICSAD as an assessment tool in ultrasound-guided supraclavicular block by determining its ability to discriminate between operators of different experience level and (2) the concurrent validity of the ICSAD by correlating it with a task-specific checklist and a global rating scale. We compared 30 performances of ultrasound-guided supraclavicular block by junior residents with 30 performances by highly experienced consultant anesthesiologists. We also studied 10 anesthesiologists undertaking a 1-year regional anesthesia fellowship and compared a performance in their first month to one in their last 3 months. We used the ICSAD to measure 3 dexterity parameters during the scanning and needling phases of each block: time taken, number of movements, and path length traveled by each hand. Two blinded expert observers evaluated video recordings of each block using a 30-item task-specific checklist and a 7-item global rating scale. Consultants (experts) performed significantly better than residents (novices) on all ICSAD parameters in both scanning and needling phases. Fellows demonstrated improvement in all ICSAD parameters between their early and late performance, reflecting their transition from novice to expert. The task-specific checklist and global rating scale were also highly discriminating between novice and expert performances. There was excellent correlation between all 3 measurement tools, thereby establishing their concurrent validity. The ICSAD is both valid and useful in assessing performance of ultrasound-guided supraclavicular block.

High-Fidelity Simulation Demonstrates the Influence of Anesthesiologists' Age and Years from Residency on Emergency Cricothyroidotomy Skills

Age-related deterioration in both cognitive function and the capacity to control fine motor movements has been demonstrated in numerous studies. However, this decline has not been described with respect to complex clinical anesthesia skills. Cricothyroidotomy is an example of a complex, lifesaving procedure that requires competency in the domains of both cognitive processing and fine motor control. Proficiency in this skill is vital to minimize time to reestablish oxygenation during a "cannot intubate, cannot ventilate" scenario. In this prospective, controlled, single-blinded study, we tested the hypothesis that age affects the learning and performance of emergency percutaneous cricothyroidotomy in a high-fidelity simulated cannot intubate/cannot ventilate scenario. Thirty-six staff anesthesiologists (19 aged younger than 45 years and 17 older than 45 years) managed a high-fidelity cannot intubate/cannot ventilate scenario in a high-fidelity simulator before and after a 1-hour standardized training session. The group division cutoff age of 45 years was based on the median age of our sample subject population before enrollment. The scenarios required the insertion of an emergency percutaneous cricothyroidotomy. We compared cricothyroidotomy skills in the older group with those in the younger group using procedural time, 5-point task-specific checklist score, and global rating scale score. Correlation based on age, years from residency, weekly clinical hours worked, previous continuing medical education in airway management, and previous simulation experience was also performed. In both prestandardization and poststandardization, age and years from residency correlated with procedural time, checklist scores, and global rating scores. Baseline, prestandardization variables were all better for the younger group, with a mean age of 37 years, compared with the older group, with a mean age of 58 years. Procedural time was 100 (72-128) seconds versus 152 (120-261) seconds. Checklist scores were 7.0 (6.1-8.0) versus 6.0 (4.8-8.0). Global rating scale scores were 22.0 (17.8-29.8) versus 17.5 (10.4-20.6). After the 1-hour standardized training session, the younger group continued to perform better than the older group with procedural time of 75 (66-91) seconds versus 87 (78-123) seconds, checklist scores of 10.0 (9.1-10.0) versus 9.0 (8.0-10.0), and global rating scale scores of 35.0 (32.1-35.0) versus 32.0 (29.0-33.8). Regression analysis was performed on the poststandardization data. Both age and years from residency independently affected procedural time, checklist scores, and global rating scale scores (all P < 0.05). Baseline proficiency with simulated emergency cricothyroidotomy is associated with age and years from residency. Despite standardized training, operator age and years from residency were associated with decreased proficiency. Further research should explore the potential of using age and years from residency as factors for implementing periodic continuing medical education.