Procedural Skills Performance Research Articles

Changing trends in assessment: Effectiveness of Direct observation of procedural skills (DOPS) as an assessment tool in anesthesiology postgraduate students.

Background and Aims:National Medical commission of India (NMC) has introduced Competency based Medical Education (CBME) following the international trend. Competency based assessment differs from traditional assessment and we need to adopt to newer work place-based assessments (WPBA). Direct Observation of Procedural Skills (DOPS) is one such assessment tool, which assesses procedural skills of a student. Hence this study was conducted to evaluate the effectiveness of DOPS among Anesthesiology postgraduate students.Material and Methods:A prospective interventional study was conducted, where in 55 postgraduate students underwent DOPS assessment while performing three routine core skills of Anesthesiology. Observations were documented (Pre-DOPS-Score) and immediate verbal feedback was given. Students were educated through demonstration of all three procedures on actual patients by three faculty members using the DOPS checklist. The students were finally assessed again with proforma and DOPS scores were documented (Post-DOPS Score). Mean Pre-DOPS and Post-DOPS assessment Scores were compared. Student and faculty Perceptions were taken regarding the DOPS assessment.Results:There was significant improvement in the procedural skill performance after DOPS with a statistically significant difference. Average time taken for DOPS test was 11 min. Average time taken for giving feedback was 13 min. Students and faculty members gave positive feedback about DOPS.Conclusion:DOPS is very feasible and effective assessment tool, improves procedural skills of Anaesthesiology postgraduate students, helps to develop good clinical skills which finally brings good clinical care.

A pre-clerkship simulation-based procedural skills curriculum: decreasing anxiety and improving confidence of procedural skill performance.

IntroductionPre-clerkship procedural skills training is not yet a standard across Canadian medical school curricula, resulting in limited exposure to procedures upon entering clerkship. While simulated skills training has been documented in the literature to improve performance in technical ability, anxiety and confidence have yet to be investigated despite their documented impact on performance and learning. This study therefore aims to evaluate the effect of pre-clerkship procedural skills training on medical student anxiety and confidence.Methods:A procedural skills training program was designed based on an evidence-based near-peer, flipped classroom model of education. Ninety-two second-year medical students volunteered for the study. Fifty-six were randomized to the training group, and 36 were randomized to the control group. Students in the training group attended seven procedural skills tutorials over seven months. The control group represented the average medical school student without standardized procedural training. Student anxiety and confidence were assessed at the beginning and end of the program using the State Trait Anxiety Inventory and Confidence Questionnaires.Results:Students who participated in the procedural skills program demonstrated greater reductions in their state anxiety and greater improvements in confidence compared to the control group.Conclusion:Longitudinal procedural skills training in the simulation setting has demonstrated improvements in anxiety and confidence among pre-clerkship medical students. These added benefits to training have the potential to ease medical students’ transition into clerkship, while also contributing to a safer and more effective clinical experience. Therefore, future integration of standardized pre-clerkship procedural skills training within medical school curricula should be considered.

The Effect of Repeated Direct Observation of Procedural Skills (R-DOPS) Assessment Method on the Clinical Skills of Anesthesiology Residents.

BackgroundThe ultimate result of patient care is one of the most important outcomes in medical education. Several methods, including the direct observation of procedural skills (DOPS), have been proposed to assess professional competencies in clinical practice.ObjectivesThis study aimed to assess the effects of the Repeated DOPS (R-DOPS) method on the performance of procedural skills in anesthesiology residents.MethodsThe procedural skill performance of anesthesiology residents was assessed using a standard DOPS protocol from May to October 2019. Their scores were then objectively recorded, and the satisfaction rates regarding the 2 DOPS exams were assessed.ResultsWe found a considerable improvement in anesthesiology residents’ procedural skill performance, especially in the anesthesiology residency curriculum’s basic items. Besides, anesthesiology residents’ satisfaction was significantly improved after the 2nd DOPS.ConclusionsR-DOPS leads to improved training outcomes, including assessing the procedural skills, time to feedback to trainees, and trainee satisfaction.

Medical Student Comfort With Procedural Skills Performance Based on Elective Experience and Career Interest.

IntroductionDespite increased efforts, studies suggest that exposure to procedural skills in undergraduate medical training is insufficient. As medical students have low self-reported competence in many skills, a significant concern is that medical students are underprepared for a clerkship. Furthermore, pre-clerkship electives selected based on student career interests can provide students with additional skills learning opportunities. The impact of career interest and elective choice on student comfort with procedural skills is unclear. This study examines the relationship between student procedural skills comfort, career interest, and elective choices.Materials and methodsAn evidence-based questionnaire was synthesized following a literature search using PubMed, Embase, and Google Scholar. Surveys were completed by second-year medical students. A Likert scale was used to evaluate students’ exposure, comfort, and motivation to learn common procedural skills. Descriptive, Pearson’s chi-square and Spearman’s rho correlation coefficient analyses were performed to evaluate the relationship between career interests, elective exposure, and procedural skills.ResultsMedical students (>60%) reported poor comfort levels for most skills, despite >80% of students displaying high motivation to learn. Elective choice impacted student comfort levels as students who completed electives in anesthesiology were more comfortable with performing intubation (23% vs 10%, p = 0.026) and IV insertion (38% vs 13%, p = 0.002). Those with surgical career interests were less comfortable performing Foley catheter insertion in males (7% vs 5%, p = 0.033) and in females (7% vs 5%, p = 0.008).ConclusionsThis study supports that medical students feel low levels of comfort with performing procedural skills despite high motivation for learning. Comfort was influenced by both career interest and elective experience. Programs aiming to increase students’ comfort levels in performing procedural skills should adapt curricula toward increasing early exposure to these skills.

Retention of Critical Procedural Skills After Simulation Training: A Systematic Review.

While short-term gains in performance of critical emergency procedures are demonstrated after simulation, long-term retention is relatively uncertain. Our objective was to determine whether simulation of critical emergency procedures promotes long-term retention of skills in nonsurgical physicians. We searched multiple electronic databases using a peer-reviewed strategy. Eligible studies 1) were observational cohorts, quasi-experimental or randomized controlled trials; 2) assessed intubation, cricothyrotomy, pericardiocentesis, tube thoracostomy, or central line placement performance by nonsurgical physicians; 3) utilized any form of simulation; and 4) assessed skill performance immediately after and at≥3months after simulation. The primary outcome was skill performance at or above a preset performance benchmark at≥3months after simulation. Secondary outcomes included procedural skill performance at 3, 6, and≥12months after simulation. We identified 1,712 citations, with 10 being eligible for inclusion. Methodologic quality was moderate with undefined primary outcomes; inadequate sample sizes; and use of nonstandardized, unvalidated tools. Three studies assessed performance to a specific performance benchmark. Two demonstrated maintenance of the minimum performance benchmark while two demonstrated significant skill decay. A significant decline in the mean performance scores from immediately after simulation to 3, 6, and≥12months after simulation was observed in four of four, three of four, and two of five studies, respectively. Scores remained significantly above baseline at 3, 6, and≥12months after simulation in three of four, three of four, and four of four studies, respectively. There were a limited number of studies examining the retention of critical skills after simulation training. While there was some evidence of skill retention after simulation, overall most studies demonstrated skill decline over time.

Procedural Skills Training in Emergency Medicine Physicians Within the Edmonton Zone: A Needs Assessment.

The objectives were to describe the current procedural skill practices, attitudes toward procedural skill competency, and the role for educational skills training sessions among emergency medicine (EM) physicians within a geographic health zone. This is a multicenter descriptive cross-sectional survey of all EM physicians working at 12 emergency departments (EDs) within the Edmonton Zone in 2019. Survey items addressed current procedural skill performance frequency; perceived importance and confidence; current methods to maintain competence; barriers and facilitating factors to participation in a curriculum; preferred teaching methods; and desired frequency of practice for each procedural skill. Survey response rate was 53.6%. Variability in frequency of performed procedures was seen across the type of hospital sites. For the majority of skills, there was a significantly positive correlation between the frequency at which a skill was performed and the perceived confidence performing said skill. There was inconsistency and no significant correlation with perceived importance, perceived confidence or frequency performing a given skill, and the desired frequency of training for that skill. Course availability (76.2%) and time (72.8%) are the most common identified barriers to participation in procedural skills training. This study summarized the current ED procedural skill practices among EM physicians in the Edmonton Zone and attitudes toward an educational curriculum for procedural skill competency. This represents a step toward targeted continuing professional development in staff physicians.

P033: Procedural skills training in emergency medicine physicians within the Edmonton zone: a needs assessment

Introduction: Procedural skills are a key component of an emergency physician's practice. The Edmonton Zone is a health region that comprises twelve tertiary, urban community and rural community emergency departments (EDs) and represents over three hundred emergency physicians. This study describes the current attitudes toward procedural skill competency, current procedural skill practices, and the role for educational skills training sessions among emergency medicine physicians within a geographical health region. Methods: Multicenter descriptive cross-sectional survey of all emergency medicine physicians working at 12 emergency departments within the Edmonton Zone in 2019 (n = 274). The survey underwent several phases of systematic review; including item generation and reduction, pilot testing, and clinical sensibility testing. Survey items addressed current procedural skill performance frequency, perceived importance and confidence, current methods to maintain competence, barriers and facilitating factors to participation in a curriculum, preferred teaching methods, and desired frequency of practice for each procedural skill. Results: Survey response rate was 53.6%. Variability in frequency of performed procedures was apparent across the type of hospital sites. For majority of skills, there was a significantly positive correlation between the frequency at which a skill was performed and the perceived confidence performing said skill. There was inconsistency and no significant correlation with perceived importance, perceived confidence, or frequency performing a given skill and the desired frequency of training for that skill. Course availability (76.2%) and time (72.8%) are the most common identified barriers to participation in procedural skills training. Conclusion: This study summarized the current emergency department procedural skill practices and attitudes toward procedural skill competency and an educational curriculum among emergency medicine physicians in Edmonton. This represents a step towards targeted continuing professional development in the growing realm of competency-based medical education.

Preclinical Simulation Training of Medical Students Results in Better Procedural Skills Performance in End of the Year Three Objective Structured Clinical Evaluation Assessments

Objectives The Association of American Medical Colleges published guidelines for procedural skills to be included in medical school curricula; however, standardized training and objective assessments of procedural skills are limited. We implemented and evaluated a simulation-based mandatory procedural skills training program for preclinical medical students for nine procedures students were expected to gain proficiency before graduation.

Interns reflect: the effect of formative assessment with feedback during pre-internship.

BackgroundIt is widely known that the opportunity for medical students to be observed and to receive feedback on their procedural skills performance is variable in the senior years. To address this problem, we provided our Pre-Intern (PrInt) students with “one-to-one” formative feedback on their ability to perform urethral catheterization (U/C) and hypothesized that their future practice of U/C as interns would benefit. This study sought to evaluate the performance and practice of interns in U/C 4–5 months after having received feedback on their performance of U/C as PrInt students.MethodsBetween 2013 and 2014, two cohorts of interns, (total n=66) who had received recent formative feedback on their U/C performance as PrInt students at Central Clinical School, were invited to complete an anonymous survey. The survey contained nine closed unvalidated questions and one open-ended question, designed to allow interns to report on their current practice of U/C.ResultsForty-one out of 66 interns (62%) completed the survey. Thirty-five out of 41 respondents (85%) reported that the assessment with feedback during their PrInt term was beneficial to their practice. Thirty of 41 (73%) reported being confident to perform U/C independently. Eleven out of 41 respondents (27%) reported that they had received additional training at intern orientation. Nine of the 11 interns (82%) reported that they had a small, but a significant, increase in confidence to perform U/C when compared with the 30 of the 41 respondents (73%) who had not (p=0.03).ConclusionOur results substantiate our hypothesis that further education by assessment with feedback in U/C during PrInt was of benefit to interns’ performance. Additional educational reinforcement in U/C during intern orientation further improved intern confidence. Our results indicate that extra pre- and post-graduation procedural skills training, with feedback, should be universal.

Education for practitioners and patients

Education for practitioners and patients

Pediatrics Residents' Preparedness for Neonatal Resuscitation Assessed Using High-Fidelity Simulation

Pediatrics residents are expected to demonstrate preparedness for neonatal resuscitation, yet research has shown gaps in residents' readiness to perform this skill. To evaluate procedural skills and team performance of pediatrics residents during neonatal resuscitation (NR) using a high-fidelity mannequin, and to assess residents' confidence in their NR skills before and after training. Two teams of residents (all had completed NR program training) participated in 2 separate, 90-minute sessions (2 to 3weeks apart) in an off-site delivery room during their neonatal intensive care rotation. Residents' confidence in assisting and leading NR was surveyed before each session. Teams participated in a scenario (adapted from the NR program), which required 5 skills (positive pressure ventilation, chest compressions, endotracheal intubation, umbilical vein catheterization, and epinephrine administration). Video recording was used for debriefing and scoring. Skills were scored for technique and timeliness, and team behaviors were scored for communication, management, and leadership. Twenty-six residents (11 teams) completed 2 paired sessions. Self-confidence scores increased between the 2 sessions but were not correlated with performance. Gaps in procedural skill performance were observed, and timeliness for most skills did not meet expectations. Significant improvement in team communication was noted. Important gaps in procedural skill performance, particularly timeliness, were detected by NR simulation training; residents' improvements in self-confidence did not reflect gains in actual performance. Their relative unpreparedness for NR (despite prior certification) highlights the need for deliberate practice and specific team training before and during neonatal intensive care delivery room rotations.

Smartphone technology enhances newborn intubation knowledge and performance amongst paediatric trainees

ObjectiveSmartphones are widely used by physicians, but their effectiveness in improving teaching of clinical skills is not known. The aim of this study was to determine if pre procedural use of a smartphone neonatal intubation instructional application (NeoTube) improves trainee knowledge and enhances procedural skills performance in newborn intubation. DesignNeonatal Resuscitation Program certified trainees in paediatrics and neonatology completed a knowledge based questionnaire on neonatal intubation, and were recorded intubating a term newborn manikin model. They then used the NeoTube iPhone application for 15min, before completing the questionnaire and intubation again. Video recordings were later reviewed by two independent assessors, blinded to whether it was pre or post NeoTube use. Results20 paediatric trainees (12 fellows and 8 residents) participated in this study. Comparing pre and post-viewing of the application, Questionnaire Scores (median (range)) increased from 18.5 (8–28) to 31 (24–35) (P<0.001), with calculation scores increasing from 6 (0–11) to 11 (6–12) (P<0.001), Skill Scores increased from 11 (9–15) to 12.5 (9–16) (P=0.016), and the duration of intubation attempt decreased from 39 to 31s (P=0.044) following utilisation of the application. There was a significant positive correlation with duration of specialist training for procedure performance post viewing, but not pre viewing of the application. ConclusionsBedside use of smartphones can enhance both knowledge of newborn intubation and improves procedural performance, including reducing the time to successfully intubate. Smartphones may have a useful role in bringing procedural skills training closer to the bedside.

An N-methyl-d-aspartate receptor agonist facilitates sleep-independent synaptic plasticity associated with working memory capacity enhancement

Working memory (WM) capacity improvement is impacted by sleep, and possibly by N-methyl-D-aspartate (NMDA) agonists such as D-cycloserine (DCS), which also affects procedural skill performance. However, the mechanisms behind these relationships are not well understood. In order to investigate the neural basis underlying relationships between WM skill learning and sleep, DCS, and both sleep and DCS together, we evaluated training-retest performances in the n-back task among healthy subjects who were given either a placebo or DCS before the task training, and then followed task training sessions either with wakefulness or sleep. DCS facilitated WM capacity enhancement only occurring after a period of wakefulness, rather than sleep, indicating that WM capacity enhancement is affected by a cellular heterogeneity in synaptic plasticity between time spent awake and time spent asleep. These findings may contribute to development, anti-aging processes, and rehabilitation of higher cognition.

Ambulance Personnel Perceptions of Near Misses and Adverse Events in Pediatric Patients

Objective. To identify emergency medical services (EMS) provider perceptions of factors that may affect the occurrence, identification, reporting, and reduction of near misses and adverse events in the pediatric EMS patient. Methods. This was a subgroup analysis of a qualitative study examining the nature of near misses and adverse events in EMS as it relates to pediatric prehospital care. Complementary qualitative methods of focus groups, interviews, and anonymous event reporting were used to collect results and emerging themes were identified and assigned to specific analytic domains. Results. Eleven anonymous event reports, 17 semistructured interviews, and two focus groups identified 61 total events, of which 12 were child-related. Eight of those were characterized by participants as having resulted in no injury, two resulted in potential injury, and two involved an ultimate fatality. Three analytic domains were identified, which included the following five themes: reporting is uncommon, blaming errors on others, provider stress/discomfort, errors of omission, and limited training. Among perceived causes of events, participants noted factors relating to management problems specific to pediatrics, problems with procedural skill performance, medication problems/calculation errors, improper equipment size, parental interference, and omission of treatment related to providers’ discomfort with the patient's age. Few participants spoke about errors they had committed themselves; most discussions centered on errors participants had observed being made by others. Conclusions. It appears that adverse events and near misses in the pediatric EMS environment may go unreported in a large proportion of cases. Participants attributed the occurrence of errors to the stress and anxiety produced by a lack of familiarity with pediatric patients and to a reluctance to cause pain or potential harm, as well as to inadequate practical training and experience in caring for the pediatric population. Errors of omission, rather than those of commission, were perceived to predominate. This study provides a foundation on which to base additional studies of both a qualitative and quantitative nature that will shed further light on the factors contributing to the occurrence, reporting, and mitigation of adverse events and near misses in the pediatric EMS setting.

A systematic review of medical skills laboratory training: where to from here?

The aim of this review was to evaluate the effectiveness of medical skills laboratories or simulators. In particular, it aimed to determine if performance in medical skills laboratories is transferable to actual clinical performance and maintained over time. A range of databases was utilised to search for relevant papers published from 1998 to June 2006. Articles were included in the review if they met a number of criteria that included the evaluation of a skills laboratory or simulator for the purpose of procedural skills training, that participants were either undergraduate medical students or postgraduate medical trainees, and that the study used a randomised, controlled trial (RCT) research design in evaluation. A total of 44 RCTs were identified for inclusion in the review. Overall, 32 (70%) studies reported that simulator training significantly improved procedural skills performance in comparison with standard or no training. Twenty (45%) RCTs assessed the transfer of simulator performance to clinical skills performance; however, 8 of these used animal models, not real patients. Only 2 studies assessed the maintenance of skills post-intervention, both at 4-month follow-up periods. Medical skills laboratories do lead to improvement in procedural skills compared with standard or no training at all when assessed by simulator performance and immediately post-training. However, there is a lack of well designed trials addressing the crucial issues of transferability to clinical practice and retention of skills over time. Further research must be carried out to address these matters if medical skills laboratories are to remain an integral component of medical education.