Evaluation Of Surgical Competence Research Articles

Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment

Construct: The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) is a 9-item surgical evaluation tool designed to assess technical competence in surgical trainees using behavioral anchors. Background: The initial development of the O-SCORE produced evidence for valid results. Further work is required to determine if the use of a single surgeon or an unblinded rater introduces bias. In addition, the relationship of the O-SCORE to other currently used technical assessment tools should be explored to provide validity evidence related to the relationship to other measures. We have designed this project to provide continued validity evidence for the O-SCORE related to these two issues. Approach: Nineteen residents and 2 staff Orthopedic Surgeons from the University of Ottawa volunteered to participate in a 2-part OSCE style station. Participants completed a written questionnaire followed by a videotaped 10-minute simulated open reduction and internal fixation of a midshaft radius fracture. Videos were rated individually by 2 blinded staff orthopedic surgeons using an Objective Structured Assessment of Technical Skills (OSATS) global rating scale, an OSATS checklist, and the O-SCORE in random order. Results: O-SCORE results appeared sensitive to surgical training level even when raters were blinded. In addition, strong agreement between two independent observers using the O-SCORE suggests that the measure captures a performance easily recognized by surgical observers. Ratings on the O-SCORE also were strongly associated with global ratings on the currently most validated technical evaluation tool (OSATS). Collectively, these results suggest that the O-SCORE generates accurate, reproducible, and meaningful results when used in a randomized and blinded fashion, providing continued validity evidence for using this tool to evaluate surgical trainee competence. Conclusions: The O-SCORE was able to differentiate surgical trainee level using blinded raters providing further evidence of validity for the O-SCORE. There was strong agreement between two independent observers using the O-SCORE. Ratings on the O-SCORE also demonstrated equivalence to scores on the most validated technical evaluation tool (OSATS). These results suggest that the O-SCORE demonstrates accurate and reproducible results when used in a randomized and blinded fashion providing continued validity evidence for this tool in the evaluation of surgical competence in the trainees.

How to educate and evaluate cardiothoracic surgeon

The history of cardiovascular surgery began with the first clinical usage of a cardiopulmonary bypass in 1953. Subsequently, new operations for congenital heart disease, valvular heart disease, ischemic heart disease, aortic disease, and transplantation were developed in a short-time period. Thoracic surgery, as well, was also expanded with the introduction of endoscopic surgery, lung volume reduction surgery, and lung transplantation. In addition, operative results have improved considerably due to findings in new operative procedures and instruments, assist devices, new drugs, and perioperative managements. During a 50to 60-year span, we have experienced the best and brightest time in cardiothoracic surgical advancement. With further progress in cardiothoracic surgery being developed, there will be an increase in need for greater technical skills and the education of said skills. The evaluation of surgical competency and the assessment of the education and continued competency of practicing surgeons will be more crucial than before. We have learned technical skills through several methods until now. However, procedural training has been unsystematic and unstructured, and proper teaching methods have not been established. Surgical skill training, for instance, has been performed using live patients and animals, human cadavers, and ex vivo animal tissue until now. With regard to these materials, there were several limitations due to ethical problems, the inexact replication of human anatomy, and the reality and expense for animals. In the past, performing on live patients and animals had been considered the best method for learning basic skills. Animal hearts from porcines and bovines, for instance, were useful tools given their similarities in size and anatomy to the human heart, as well as their beating operations. However, recently the methods used on live patients and animals are no longer viable due to the increasing complexity of cases, costs, and ethics. Human cadavers have proven the best alternative for studying the human anatomy, yet it is not prevalent here in Japan. Instead, ex vivo animal tissues have been used to teach basic technical skills for residents. During the last decade, operative methods have undergone significant changes compared to the classic styles, which are relatively invasive open surgery. Minimal invasive surgeries, however, have evolved in both thoracic and cardiovascular disease such as therapeutic bronchoscopy, endoscopic lung resection, robotic approaches to esophageal resection, off pump coronary artery bypass grafting, endovascular stent grafting to aortic disease, hybrid approaches to congenital heart disease, and endovascular repair to valvular heart disease. In such situations, it is important to evaluate the technical skills and surgical competency for the patient’s safety [1]. The simulation system is now spotlighted for learning both technical skills, situational simulation, and for evaluating competency performance and surgical skills objectively. Simulation training was found to improve subsequent performances compared with subjects cake received didactic training alone. This benefit was found to be durable and persistent in the long term. This presidential address was written at the invitation of the editorial committee.

An Anatomic Model for Use in Surgical Skills Training of Tracheotomy

Objective: The objective would be to design a low cost, reproducible anatomic model to simulate tracheotomy in the surgical skills training laboratory. The model was adapted from a base model used previously for verification of proficiency (VOP) assessment of cricothyrotomy by general surgery residents. Method: The following materials were used: ventilator tubing, vessel loops, a 2x4x9-inch block of wood, red and white felt fabric, automotive ceiling headliner, cardboard, foam tape, adhesive spray, and foam. All materials were found easily at a local craft store or in medical discard. Results: The model was assembled by securing ventilator tubing to a block of wood to represent the trachea. Thyroid and cricoid cartilages were fashioned and secured to ensure palpable landmarks. An anterior section of ventilator tubing was removed and replaced with foam tape to allow for a recyclable base. Finally, the thyroid and superficial tissue (including vessels and strap muscles) were assembled using headliner, felt, and vessel loops. After assembly was completed, four junior residents were able to successfully palpate each landmark, “ligate” vessels, identify and retract the thyroid isthmus, and perform a tracheostomy. Conclusion: The model developed is a practical and reproducible anatomic model for simulation training of tracheotomy for otolaryngology residents and can be created utilizing inexpensive, readily available materials. In conjunction with a skills lab training session, this useful tool could play an important role in the overall evaluation of surgical competence.

Digital Video Documentation as Evidence of Clinical Skill Acquisition

Video recordings have become requirements for the assessment and recording of the consulting skills module of the MRCGP examination and surgical skill modules for the post FRCS hand diploma examination. Previous research has shown that the knowledge and judgement skills needed by trainees to achieve operative competence can be reliably assessed using structured checklists. However evaluation of surgical competency should also include technical skills, which have been difficult to document adequately and transparently. There is now evidence demonstrating that surgeons can discriminate between the video recordings of a competent and non‐competent trainee. Like those of previous researchers our findings indicate that intra‐operative video monitoring enables an objective and permanent recordable assessment of the a trainees skill level whilst completing the checklist after a particular procedure. Video evidence can also be distributed widely to assessors outside the trainee's locale.

Von der kontinuierlichen medizinischen Fortbildung zur kontinuierlichen professionellen Entwicklung am Beispiel der chirurgischen Fortbildung

In addition to medical knowledge and skills, medical competency includes critical self-refection and professional mistake management, the ability to manage the administrative workload and techniques of professional communication and leadership. In addition to that physicians have to develop social competency and empathy for staff members and patients. Continuing medical education (CME) focuses on continuing development of medical knowledge and skills. To develop the additional aspects of medical competency the concept of medical education has to be broadened to continuing professional development (CPD). In different European countries and North America CPD includes a regular mandatory system to evaluate medical competency. Assessment systems like PAR in Alberta/Canada provide doctors with a 360 degree analysis of their daily work, showing the potential for future improvement. ECKO (Evaluation of Surgical Competency) is an adapted assessment system for German surgeons available on a voluntary basis. For the first time in Germany physicians are able to take part in a strength-weakness analysis of their daily work. They receive recommendations on practice improvement and an educational prescription for individual further education.

Evaluating surgical competency with the American Board of Surgery In-Training Examination, skill testing, and intraoperative assessment

Background. Evaluation of surgical competency should include assessment of knowledge, technical skill, and judgment. The purpose of this study was to determine the relationship between the American Board of Surgery In-Training Examination (ABSITE), skill testing, and intraoperative assessment. Methods. Postgraduate year 2 (PGY-2) and postgraduate year 3 (PGY-3) surgery residents (n = 33) were tested by means of (1) the ABSITE, (2) skill testing on a laparoscopic video-trainer, and (3) intra-operative global assessments during laparoscopic cholecystectomy. The Pearson correlation was used to determine the correlation between the ABSITE, skill testing, and intraoperative assessments. For the comparison of PGY-2 and PGY-3 resident performance, Wilcoxon rank sum tests were used. Results. The ABSITE scores did not correlate with skill testing or intraoperative assessments (not significant). Skill testing correlated with the intraoperative composite score and with 4 of 8 operative performance criteria (P <.05). The ABSITE scores and skill testing were not different for PGY-2 and PGY-3 residents (not significant). Intraoperative assessments were better in 5 of 8 criteria and the composite score for PGY-3 versus PGY-2 residents (P <.05), which demonstrated construct validity. Conclusions. The ABSITE measures knowledge but does not correlate with technical skill or operative performance. Residency programs should use multiple assessment instruments to evaluate competency. There may be a role for both skill testing and intraoperative assessment in the evaluation of surgical competency. (Surgery 2000;128:613-22)

New Simulation Technologies for Surgical Training and Certification: Current Status and Future Projections

Computer-based virtual reality technology has evolved to the point at which medical simulation can be incorporated into medical education and into the eventual objective evaluation of surgical competence. Flight simulation provides a model for examining the role of computer-based simulation in medical training and certification. The plan by which medical simulators are being designed and validated for surgical training is presented, as is a description of the current state of medical simulation and the limitations of the technology. A realistic argument for adoption is suggested that takes into account lower price constraints, technological limitations, and professional barriers to the implementation of simulator-based training and accreditation.1