Nontechnical Scores Research Articles

A Customizable Digital Cognitive Aid for Neonatal Resuscitation: A Simulation-Based Randomized Controlled Trial.

Adherence to the International Liaison Committee on Resuscitation (ILCOR) algorithm optimizes the initial management of critically ill neonates. In this randomized controlled trial, we assessed the impact of a customizable sequential digital cognitive aid (DCA), adapted from the 2020 ILCOR recommendations, compared with a poster cognitive aid (standard of care [SOC]), on technical and nontechnical performance of junior trainees during a simulated critical neonatal event at birth. For this prospective, bicentric video-recorded study, students were recruited on a voluntary basis, and randomized into groups of 3 composed of a pediatric resident and two midwife students. They encountered a simulated cardiac arrest at birth either (1) with DCA use and ILCOR algorithm poster displayed on the wall (intervention group) or (2) with sole ILCOR algorithm poster (poster cognitive aid [SOC]). Technical and nontechnical skills (NTS) between the two groups were assessed using a standardized scoring of videotaped performances. A neonate specific NTS score was created from the adult Team score. 108 students (36 groups of three) attended the study, 20 groups of 3 in the intervention group and 16 groups of 3 in the poster cognitive aid (SOC) group. The intervention group showed a significant improvement in the technical score ( P < 0.001) with an average of 24/27 points (24.0 [23.5-25.0]) versus 20.8/27 (20.8 [19.9-22.5]) in poster cognitive aid (SOC) group. No nontechnical score difference was observed. Feedback on the application was positive. During a simulated critical neonatal event, use of a DCA was associated with higher technical scores in junior trainees, compared with the sole use of ILCOR poster algorithm.

Team performance during vacuum-assisted vaginal delivery: video review of obstetric multidisciplinary teams.

Vacuum extraction is generally considered an operator-dependent task, with most attention directed toward the obstetrician's technical abilities (1-3). Little is known about the effect of the team and non-technical skills on clinical outcomes in vacuum-assisted delivery. This study aimed to investigate whether the non-technical skills of obstetricians were correlated with their level of clinical performance via the analysis of video recordings of teams conducting actual vacuum extractions. We installed between two or three video cameras in each delivery room at Aarhus University Hospital and Horsens Regional Hospital and obtained 60 videos of teams managing vacuum extraction. Appropriate consent was obtained. Two raters carefully reviewed the videos and assessed the teams' non-technical skills using the Assessment of Obstetric Team Performance (AOTP) checklist, rating all items on a Likert scale score from 1 to 5 (1 = poor; 3 = average; and 5 = excellent). This resulted in a total score ranging from 18 to 90. Two different raters independently assessed the teams' clinical performance (adherence to clinical guidelines) using the TeamOBS-Vacuum-Assisted Delivery (VAD) checklist, rating each item (0 = not done, 1 = done incorrectly; and 2 = done correctly). This resulted in a total score with the following ranges (low clinical performance: 0-59; average: 60-84; and high: 85-100). Interrater agreement was analyzed using intraclass correlation (ICC), and the risk of high or low clinical performance was analyzed on a logit scale to meet the assumption of normality. Teams that received excellent non-technical scores had an 81% probability of achieving high clinical performance, whereas this probability was only 12% among teams with average non-technical scores (p < 0.001). Teams with a high clinical performance often had excellent behavior in the non-technical items of "team interaction," "anticipation," "avoidance fixation," and "focused communication." Teams with a low or average clinical performance often neglected to consider analgesia, had delayed abandonment of the attempted vaginal delivery and insufficient use of appropriate fetal monitoring. Interrater reliability was high for both rater-teams, with an ICC for the non-technical skills of 0.83 (95% confidence interval [CI]: 0.71-0.88) and 0.84 for the clinical performance (95% CI: 0.74-0.90). Although assisted vaginal delivery by vacuum extraction is generally considered to be an operator-dependent task, our findings suggest that teamwork and effective team interaction play crucial roles in achieving high clinical performance. Teamwork helped the consultant anticipate the next step, avoid fixation, ensure adequate analgesia, and maintain thorough fetal monitoring during delivery.

COLLABORATIVE TEAM TRAINING IN IMMERSIVE VIRTUAL REALITY IS SUPERIOR TO INDIVIDUAL LEARNING FOR PERFORMING TOTAL HIP ARTHROPLASTY: A RANDOMIZED CONTROLLED TRIAL

Superior teamwork in the operating theatre is associated with improved technical performance and clinical outcomes. Yet modern rota patterns, workforce shortages, and increasing complexity of surgery, means that there is less familiarity between staff and the required choreography. Immersive Virtual Reality (iVR) can successfully train surgical staff individually, however iVR team training has yet to be investigated. We aimed to design a multiplayer iVR platform for anterior approach total hip arthroplasty (AA-THA) and assess if multiplayer iVR training was superior to single player training for acquisition of both technical and non-technical skills.An iVR platform with choreographed roles for the surgeon and scrub nurse was developed using Cognitive Task Analysis. Forty participants were randomised to individual or team iVR training. Individually- trained participants practiced alongside virtual avatar counterparts, whilst teams trained live in pairs. Both groups underwent five iVR training sessions over 6-weeks. Subsequently, they underwent a real-life assessment in which they performed AA-THA on a high-fidelity model with real equipment in a simulated theatre. Teams performed together and individually trained participants were randomly paired up. Videos were marked by two blinded assessors recording the NOTSS, NOTECHS II and SPLINTS scores - validated technical and non-technical scores assessing surgeon and scrub nurse skills. Secondary outcomes were procedure time and number of technical errors.Teams outperformed individually trained participants for non-technical skills in the real-world assessment (NOTSS 13.1 ± 1.5 vs 10.6 ± 1.6, p =0.002, NOTECHS-II score 51.7 ± 5.5 vs 42.3 ± 5.6, p=0.001 and SPLINTS 10 ± 1.2 vs 7.9 ± 1.6, p = 0.004). They completed the assessment 28.1% faster (27.2 minutes ± 5.5 vs 41.8 ±8.9, p&lt;0.001), and made fewer than half the number of technical errors (10.4 ± 6.1 vs 22.6 ± 5.4, p&lt;0.001).Multiplayer training leads to faster surgery with fewer technical errors and the development of superior non-technical skills for anterior approach total hip arthroplasty. The convention of surgeons and nurses training separately, but undertaking real complex surgery together, can be supplanted by team training, delivered through immersive virtual reality.

Teamwork and Adherence to Guideline on Newborn Resuscitation-Video Review of Neonatal Interdisciplinary Teams.

BackgroundLittle is known about the importance of non-technical skills for the adherence to guidelines, when teams of midwives, obstetricians, anesthesiologists, and pediatricians resuscitate and support the transition of newborns. Non-technical skills are competences underpinning successful teamwork in healthcare. These are usually referred to as leadership, situational awareness, communication, teamwork, decision making, and coping with stress and fatigue.ObjectiveBy review of videos of teams managing newborns with difficult transition, we aimed to investigate whether the level of the teams' non-technical skills was associated with the degree of adherence to guidelines for newborn resuscitation and transitional support at birth.MethodsFour expert raters independently assessed 43 real-life videos of teams managing newborns with transitional difficulties, two assessed the non-technical score and two assessed the clinical performance. Exposure was the non-technical score, obtained by the Global Assessment Of Team Performance checklist (GAOTP). GAOTP was rated on a Likert Scale 1–5 (1 = poor, 3 = average and 5 = excellent). The outcome was the clinical performance score of the team assessed according to adherence of the European Resuscitation Counsel (ERC) guideline for neonatal resuscitation and transitional support. The ERC guideline was adapted into the checklist TeamOBS-Newborn to facilitate a structured and simple performance assessment (low score 0–60, average 60–84, high 85–100). Interrater agreement was analyzed by intraclass correlation (ICC), Bland-Altman analysis, and Cohen's kappa weighted. The risk of high and low clinical performance was analyzed on the logit scale to meet the assumptions of normality and constant standard deviation.ResultsTeams with an excellent non-technical score had a relative risk 5.5 [95% confidence interval (CI) 2.4–22.5] of high clinical performance score compared to teams with average non-technical score. In addition, we found a dose response like association. The specific non-technical skills associated with the highest degree of adherence to guidelines were leadership and teamwork, coping with stress and fatigue, and communication with parents. Inter-rater agreement was high; raters assessing non-technical skills had an interclass coefficient (ICC) 0.88 (95% CI 0.79–0.94); the neonatologists assessing clinical performance had an ICC of 0.81 (95% CI 0.66–0.89).ConclusionTeams with an excellent non-technical score had five times the chance of high clinical performance compared to teams with average non-technical skills. High performance teams were characterized by good leadership and teamwork, coping with stress, and fatigue and communication with parents.

Utilizing Black Box Technology to Identify and Describe Intraoperative Delays, Distractions and Threats in the Gynecology OR: A Pilot Study

Study Objective Operating room delays can result from a chain of intra-operative events including technical factors, equipment failures or unavailability, and non-technical human factors. Operating Room Black Box (ORBB) technology can be used to capture information during surgery for analysis and potential identification of root causes of inefficiencies. Our objective was to measure and characterize procedural steps, intra-operative distractions and delays during a common minimally invasive gynecologic procedure and identify if these delays correlated with team technical and non-technical scores. Design Cross-sectional study. Setting Canadian tertiary care academic hospital. Patients or Participants Eleven patients undergoing total laparoscopic hysterectomy (TLH) between May and October 2019. Interventions Video, audio, and patient physiologic data from all procedures were obtained through a multichannel synchronized recording device. Trained analysts reviewed the recordings and coded procedural timing, distractions, delays, threats and team technical and non-technical scores. Measurements and Main Results Mean total case time was 172 minutes (SD +/- 16 minutes) and mean procedure time was 117 minutes (SD +/- 17 minutes). Mean surgical times for the pre-operative, intra-operative and post-operative phases were 85 mins (SD ± 40 mins), 116 mins (SD ± 17 mins) and 28 mins (SD ± 16 mins) respectively. Correlations between total case time and team non-technical scores and procedure time and team non-technical scores were positive and moderate in strength. Medians of 83 door openings [interquartile range (IQR) 76-104], 173 distractions (IQR 19-190) and 3 threats (IQR 2-5) were identified per case. Conclusion The ORBB allowed us to identify variations in procedural case times for elective TLH. There were frequent intraoperative distractions as well as latent threats noted and there was a moderately positive correlation between case times and team non-technical skills.

Utilization of a Voice-Based Virtual Reality Advanced Cardiac Life Support Team Leader Refresher: Prospective Observational Study.

BackgroundThe incidence of cardiac arrests per year in the United States continues to increase, yet in-hospital cardiac arrest survival rates significantly vary between hospitals. Current methods of training are expensive, time consuming, and difficult to scale, which necessitates improvements in advanced cardiac life support (ACLS) training. Virtual reality (VR) has been proposed as an alternative or adjunct to high-fidelity simulation (HFS) in several environments. No evaluations to date have explored the ability of a VR program to examine both technical and behavioral skills and demonstrate a cost comparison.ObjectiveThis study aimed to explore the utility of a voice-based VR ACLS team leader refresher as compared with HFS.MethodsThis prospective observational study performed at an academic institution consisted of 25 postgraduate year 2 residents. Participants were randomized to HFS or VR training and then crossed groups after a 2-week washout. Participants were graded on technical and nontechnical skills. Participants also completed self-assessments about the modules. Proctors were assessed for fatigue and task saturation, and cost analysis based on local economic data was performed.ResultsA total of 23 of 25 participants were included in the scoring analysis. Fewer participants were familiar with VR compared with HFS (9/25, 36% vs 25/25, 100%; P<.001). Self-reported satisfaction and utilization scores were similar; however, significantly more participants felt HFS provided better feedback: 99 (IQR 89-100) vs 79 (IQR 71-88); P<.001. Technical scores were higher in the HFS group; however, nontechnical scores for decision making and communication were not significantly different between modalities. VR sessions were 21 (IQR 19-24) min shorter than HFS sessions, the National Aeronautics and Space Administration task load index scores for proctors were lower in each category, and VR sessions were estimated to be US $103.68 less expensive in a single-learner, single-session model.ConclusionsUtilization of a VR-based team leader refresher for ACLS skills is comparable with HFS in several areas, including learner satisfaction. The VR module was more cost-effective and was easier to proctor; however, HFS was better at delivering feedback to participants. Optimal education strategies likely contain elements of both modalities. Further studies are needed to examine the utility of VR-based environments at scale.

Pilot One-Hour Multidisciplinary Team Training Simulation Intervention in the Operating Room Improves Team Nontechnical Skills

The primary objective of this study was to determine if a 1-hour simulation-based training with interdisciplinary operating room (OR) teams could improve nontechnical skills of the providers as assessed by the nontechnical skills tool (NOTECHS II). Interprofessional otolaryngology OR teams consisting of surgery faculty and resident, anesthesiology faculty and resident, and OR nurses, scrub technician, and perioperative technician underwent a 1-hour simulation-based intervention in the OR. The teams were rated on their nontechnical skills during the intervention and throughout the clinical day following. They also completed self-reflection surveys (SRS) before the intervention and in 3 intervals after the intervention (immediately following the intervention, at the end of their shift on the day of the intervention, and again 2 weeks later). Four interprofessional teams with a total of 26 unique participants participated in this pilot program. Team nontechnical skills, assessed using NOTECHS II, improved from the first simulation to the second simulation during the intervention. Team NOTECHS II scores remained higher throughout the clinical day. Individual self-reflection scores (SRS) followed the same trend. On-site interprofessional OR team training simulation can take place in a brief time period that is dedicated for education. A brief intervention resulted in improved team nontechnical scores when assessed following intervention. In addition, participants found the intervention to be effective and beneficial to their learning.

The importance of non-technical performance for teams managing postpartum haemorrhage: video review of 99 obstetric teams.

Little is known about how teams' non-technical performance influences clinical performance in obstetric emergencies such as postpartum haemorrhage. Video review - observational study. A university hospital (5000 deliveries) and a regional hospital (2000 deliveries) in Denmark. Obstetric teams managing real-life postpartum haemorrhage. We systematically assessed 99 video recordings of obstetric teams managing real-life major postpartum haemorrhage. Exposure was the non-technical score (AOTP); outcomes were the clinical performance score (TeamOBS) and the delayed transfer to the operating theatre (defined as blood loss>1500ml in the delivery room). Teams with an excellent non-technical score performed significantly better than teams with a poor non-technical score: 83.7 versus 0.3% chance of a high clinical performance score (P<0.001), 0.2 versus 80% risk of a low clinical performance score (P<0.001), and 3.5 versus 31.7% risk of delayed transfer to the operating theatre (P=0.008). The results remained robust when adjusting for potential confounders such as bleeding velocity, aetiology, time of day, team size, and hospital. The specific non-technical skills associated with high clinical performance were vigilance, role assignment, problem-solving, management of disruptive behavior, and leadership. Communication with the patient and closing the loop were of minor importance. All performance assessments showed good reliability: the intraclass correlation was 0.97 (95% CI 0.96-0.98) for the non-technical score and 0.84 (95% CI 0.76-0.89) for the clinical performance score. Video review offers a new method and new perspectives for research in obstetric teams to identify how teams become effective and safe; the skills identified in this study can be included in future obstetric training programmes. Non-technical performance is important for teams managing postpartum haemorrhage; video review of 99 obstetric teams.

Perioperative laryngospasm management in paediatrics: a high-fidelity simulation study

AimPaediatric anaesthesia is a very specialised domain lacking training during the traditional curriculum. The laryngospasm is a stressful and life-threatening event that requires immediate action. The main objective of this study was to assess the effect of knowledge of a simple algorithm on the management of laryngospasm by trainee anaesthetists and nurse anaesthetists during a high-fidelity simulation session.MethodResidents in paediatric anaesthesia and training nurse anaesthetists with similar curriculum were randomly allocated to undergo a simulation session of laryngospasm with the help of a simple algorithm administered to them 5 min before the simulation session (group A) or as taught in their curriculum (group C). The primary endpoint was the assessment score of laryngospasm management using 10 technical items (validated in experienced paediatric anaesthetists). Secondary endpoints were: the non-technical skills using the Anaesthetists' Non-Technical Skills (ANTS) score and timing of critical management steps. The correlation between the technical and non-technical scores was also studied. Data are expressed as median (range).Results72 participants (18 teams in each group) were included in this study. There was a statistically significant difference between group A and C on the primary endpoint: 8.5 (2–10) vs 5 (2–8), respectively (p&lt;0.0001). There was also a significant difference between the two groups for the ANTS score 12 (7–16) vs 8 (6–12), respectively (p&lt;0.0001). No difference in timing of management was observed. Finally, there was a strong correlation between the technical skills and all the non-technical skills categories in the A group.ConclusionA simple algorithm improved the technical and non-technical skills of students during the management of a simulated laryngospasm.

Performances of low level hospital health caregivers after a neonatal resuscitation course.

BackgroundHigh fidelity simulation has been executed to allow the evaluation of technical and non-technical skills of health caregivers. Our objective was to assess technical and non-technical performances of low level hospitals health caregivers who attended a Neonatal Resuscitation course using high fidelity simulation in a standard-setting scenario.MethodsTwenty-three volunteers were asked to manage a simple scenario (infant with secondary apnea) after the course. Technical and non-technical skills were assessed by using previously published scores. Performances were assessed during the scenario and after 2 months by filmed video recordings.ResultsSixteen (69.5%) participants failed to pass the minimum required technical score. Staff experience and participation in previous courses were associated to higher score in technical and non-technical skills, while working in level I or II hospitals did not affect the scores. Previous experience in neonatal resuscitation requiring positive pressure ventilation was associated to better non-technical performance. Technical and non-technical scores were significantly correlated (r = 0.67, p = 0.0005). Delayed and direct evaluation of technical skills provided the same scores.ConclusionsA neonatal resuscitation course, performed by using a high fidelity simulation manikin, had a limited impact on technical and non-technical skills of participants working in low level hospitals. Training programs should be tailored to the participants’ professional background and to the more relevant sessions.

Leadership and teamwork in medical emergencies: performance of nursing students and registered nurses in simulated patient scenarios.

To examine nursing students' and registered nurses' teamwork skills whilst managing simulated deteriorating patients. Studies continue to show the lack of timely recognition of patient deterioration. Management of deteriorating patients can be influenced by education and experience. Mixed methods study conducted in two universities and a rural hospital in Victoria, and one university in Queensland, Australia. Three simulation scenarios (chest pain, hypovolaemic shock and respiratory distress) were completed in teams of three by 97 nursing students and 44 registered nurses, equating to a total of 32 student and 15 registered nurse teams. Data were obtained from (1) Objective Structured Clinical Examination rating to assess performance; (2) Team Emergency Assessment Measure scores to assess teamwork; (3) simulation video footage; (4) reflective interview during participants' review of video footage. Qualitative thematic analysis of video and interview data was undertaken. Objective structured clinical examination performance was similar across registered nurses and students (mean 54% and 49%); however, Team Emergency Assessment Measure scores differed significantly between the two groups (57% vs 38%, t = 6·841, p < 0·01). In both groups, there was a correlation between technical (Objective Structured Clinical Examination) and nontechnical (Team Emergency Assessment Measure) scores for the respiratory distress scenario (student teams: r = 0·530, p = 0·004, registered nurse teams r = 0·903, p < 0·01) and hypovolaemia scenario (student teams: r = 0·534, p = 0·02, registered nurse teams: r = 0·535, p = 0·049). Themes generated from the analysis of the combined quantitative and qualitative data were as follows: (1) leadership and followership behaviours; (2) help-seeking behaviours; (3) reliance on previous experience; (4) fixation on a single detail; and (5) team support. There is scope to improve leadership, team work and task management skills for registered nurses and nursing students. Simulation appears to be beneficial in enabling less experienced staff to assess their teamwork skills. There is a need to encourage less experienced staff to become leaders and for all staff to develop improved teamwork skills for medical emergencies.

A procedural skills OSCE: assessing technical and non-technical skills of internal medicine residents.

Internists are required to perform a number of procedures that require mastery of technical and non-technical skills, however, formal assessment of these skills is often lacking. The purpose of this study was to develop, implement, and gather validity evidence for a procedural skills objective structured clinical examination (PS-OSCE) for internal medicine (IM) residents to assess their technical and non-technical skills when performing procedures. Thirty-five first to third-year IM residents participated in a 5-station PS-OSCE, which combined partial task models, standardized patients, and allied health professionals. Formal blueprinting was performed and content experts were used to develop the cases and rating instruments. Examiners underwent a frame-of-reference training session to prepare them for their rater role. Scores were compared by levels of training, experience, and to evaluation data from a non-procedural OSCE (IM-OSCE). Reliability was calculated using Generalizability analyses. Reliabilities for the technical and non-technical scores were 0.68 and 0.76, respectively. Third-year residents scored significantly higher than first-year residents on the technical (73.5 vs. 62.2%) and non-technical (83.2 vs. 75.1%) components of the PS-OSCE (p < 0.05). Residents who had performed the procedures more frequently scored higher on three of the five stations (p < 0.05). There was a moderate disattenuated correlation (r = 0.77) between the IM-OSCE and the technical component of the PS-OSCE scores. The PS-OSCE is a feasible method for assessing multiple competencies related to performing procedures and this study provides validity evidence to support its use as an in-training examination.

The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: Development and evaluation of feasibility and validity

AimTo develop and test the feasibility, reliability, and validity of a practical toolkit for the assessment and feedback of skills required to manage paediatric emergencies in critical care settings. MethodsThe Imperial Paediatric Emergency Training Toolkit (IPETT) was developed based on current evidence-base and expert input. IPETT assesses both technical and non-technical skills. The technical component covers skills in the areas of clinical assessment, airway and breathing, cardiovascular, and drugs. The non-technical component is based on the validated NOTECHS tool and covers communication and interaction, cooperation and team skills, leadership and managerial skills, and decision-making. The reliability (internal consistency), content validity (inter-correlations between different skills) and concurrent validity (correlations between global technical and non-technical scores) of IPETT were prospectively evaluated in 45 simulated paediatric crises carried out in a PICU with anaesthetic and paediatric trainees (N=52). Non-parametric analyses were carried out. Significance was set at P<0.05. ResultsCronbach alpha reliability coefficients were overall acceptable for the technical (alpha range=0.638–0.810) and good for the non-technical (alpha range=0.701–0.899) component of IPETT. The median inter-skill correlation was rho=0.564 and rho=0.549 for the technical and non-technical components, respectively. These indicate good content validity, as the skills were inter-related but not redundant. We also demonstrate a correlation between the global technical and non-technical scores (rho=0.471) – all Ps<0.05 during the assessments. ConclusionIPETT offers a psychometrically viable and feasible to use tool in the context of paediatric emergencies training. This study shows that assessment of technical and non-technical skills in combination may offer a more clinically relevant model for training in paediatric emergencies. Further validation should aim to demonstrate skill retention over time and skill transfer from simulation-based training to real emergencies.

Evaluating the management of septic shock using patient simulation

Develop a scoring system that can assess the management of septic shock by individuals and teams. Retrospective review of videotapes of critical care house staff managing a standardized simulation of septic shock. Academic medical center; videotapes were made in a recreated intensive care unit environment using a high-fidelity patient simulator. Residents in medicine, surgery, and anesthesiology who had participated in the intensive care unit rotation. The septic patient was managed by the intensive care unit team in a graded manner with interns present for the first 10 mins and more senior-level help arriving after 10 mins. The intern was graded separately for the first 10 mins, and the team was graded for the entire 35-min performance. Both technical and nontechnical scoring systems were developed to rate the management of septic shock. Technical scores are based on guidelines and principles of managing septic shock. Team leadership, communication, contingency planning, and resource utilization were addressed by the nontechnical rating. Technical scores were calculated for both interns and teams; nontechnical scores applied only to the team. Of 16 technical checklist items, interns completed a mean of 7 with a range of 1.5-11. Team technical ratings had a mean of 9.3 with a range of 3.3-13. Nontechnical scores showed similar intergroup variability with a mean of 26 and a range of 10-35. Technical and nontechnical scores showed a modest correlation (r = .40, p = .05). Interrater reliabilities for intern and team technical scores were both r = .96 and for nontechnical scores r = .88. Objective measures of both knowledge-based and behavioral skills pertinent to the management of septic shock were made. Scores identified both adequate and poor levels of performance. Such assessments can be used to benchmark clinical skills of individuals and groups over time and may allow the identification of interventions that improve clinical effectiveness in sepsis management.