Team Performance Assessment Research Articles

Peer-Project-Based Learning in CNC Simulation Programming Courses

The challenges of working with various dynamics and diverse job opportunities require students to have self-efficacy and creativity. Selfefficacy emphasizes courage in solving work challenges and creativity related to the supporting capacity of student innovation. This study aims to develop a peer-project-based learning (Peer PbL) model through model feasibility testing and model effectiveness testing. The research method uses a research and development approach by testing the model using an experimental method with a one-shot case study design. A total of 15 students were involved in this research, with the main topic discussion being CNC turning simulation. The results showed that the steps in the Peer-PbL model include team creation, providing challenges, preparing project plans, validating project suitability, doing projects, monitoring team performance, project presentations, and peer assessment. Feasibility assessments from machine learning experts, learning technologists, evaluation experts, CNC professionals, and CAD/CAM professionals indicate strong rater results and correlations. The results of testing the effectiveness of the Peer- PbLmodel in implementing classroom conditions turned out to be able to increase self-efficacy and creativity before and after the Peer-PbL model. This research is a strategy for updating PbL as a learning model by CNC simulation and programming courses.

Video-based peer assessment of collaborative teamwork in a large-scale interprofessional learning activity

BackgroundThe assessment of team performance within large-scale Interprofessional Learning (IPL) initiatives is an important but underexplored area. It is essential for demonstrating the effectiveness of collaborative learning outcomes in preparing students for professional practice. Using Kane’s validity framework, we investigated whether peer assessment of student-produced videos depicting collaborative teamwork in an IPL activity was sufficiently valid for decision-making about team performance, and where the sources of error might lie to optimise future iterations of the assessment.MethodsA large cohort of health professional students (n = 1218) of 8 differing professions was divided into teams containing 5–6 students. Each team collaborated on producing a short video to evidence their management of one of 12 complex patient cases. Students from two other teams, who had worked on the same case, individually rated each video using a previously developed assessment scale. A generalisability study quantified sources of error that impacted the reliability of peer assessment of collaborative teamwork. A decision study modeled the impact of differing numbers of raters. A modified Angoff determined the pass/fail mark.ResultsWithin a large-scale learning activity, peer assessment of collaborative teamwork was reliable (G = 0.71) based on scoring by students from two teams (n = 10–12) for each video. The main sources of variation were the stringency and subjectivity of fellow student assessors. Whilst professions marked with differing stringency, and individual student assessors had different views of the quality of a particular video, none of that individual assessor variance was attributable to the assessors’ profession. Teams performed similarly across the 12 cases overall, and no particular professions marked differently on any particular case.ConclusionA peer assessment of a student-produced video depicting interprofessional collaborative teamwork around the management of complex patient cases can be valid for decision-making about student team performance. Further refining marking rubrics and student assessor training could potentially modify assessor subjectivity. The impact of professions on assessing individual peers and the case-specificity of team performances in IPL settings need further exploration. This innovative approach to assessment offers a promising avenue for enhancing the measurement of collaborative learning outcomes in large-scale Interprofessional learning initiatives.

Pediatric critical care nursing empowerment through distance education and simulation in Accra, Ghana

BackgroundThe study assessed a distance simulation and education intervention designed to improve nursing empowerment in the pediatric intensive care unit (PICU) in Korle Bu Teaching Hospital (KBTH) in Ghana. MethodsThis was a single center, mixed methods, longitudinal simulation-based pilot study. A distance pediatric critical care nursing curriculum that combined interactive learning via Zoom and distance simulation using the Virtual Resus Room was designed and delivered from the United States to KBTH PICU nursing staff. ResultsThe Clinical Teamwork Scale, used to assess team performance after each simulation, demonstrated an upward trend over time in all categories. The modified Simulation Effectiveness Tool was utilized to evaluate the learners' perceptions of the effectiveness of the simulation in meeting their learning objectives. Qualitative content analysis of this free-text feedback highlighted the course's educational content, organizational aspects, learning experience, simulation case scenarios and relevance to pediatric critical care in Ghana. ConclusionsA distance pediatric critical care nursing curriculum, relevant to local practice and resources, was implemented through an international partnership and contributed to nursing empowerment through improved learning and confidence.

TEAMs go VR—validating the TEAM in a virtual reality (VR) medical team training

BackgroundInadequate collaboration in healthcare can lead to medical errors, highlighting the importance of interdisciplinary teamwork training. Virtual reality (VR) simulation-based training presents a promising, cost-effective approach. This study evaluates the effectiveness of the Team Emergency Assessment Measure (TEAM) for assessing healthcare student teams in VR environments to improve training methodologies.MethodsForty-two medical and nursing students participated in a VR-based neurological emergency scenario as part of an interprofessional team training program. Their performances were assessed using a modified TEAM tool by two trained coders. Reliability, internal consistency, and concurrent validity of the tool were evaluated using intraclass correlation coefficients (ICC) and Cronbach’s alpha.ResultsRater agreement on TEAM’s leadership, teamwork, and task management domains was high, with ICC values between 0.75 and 0.90. Leadership demonstrated strong internal consistency (Cronbach’s alpha = 0.90), while teamwork and task management showed moderate to acceptable consistency (alpha = 0.78 and 0.72, respectively). Overall, the TEAM tool exhibited high internal consistency (alpha = 0.89) and strong concurrent validity with significant correlations to global performance ratings.ConclusionThe TEAM tool proved to be a reliable and valid instrument for evaluating team dynamics in VR-based training scenarios. This study highlights VR’s potential in enhancing medical education, especially in remote or distanced learning contexts. It demonstrates a dependable approach for team performance assessment, adding value to VR-based medical training. These findings pave the way for more effective, accessible interdisciplinary team assessments, contributing significantly to the advancement of medical education.

Assessing Team Performance in a Longitudinal Neonatal Resuscitation Simulation Training Program: Comparing Validity Evidence to Select the Best Tool.

Introduction Neonatal resuscitation is a high-acuity, low-occurrence event that requires ongoing practice by interprofessional teams to maintain proficiency.Simulation provides an ideal platform for team training and evaluation of team performance. Our simulation center supports a longitudinal in situ simulation training program for delivery room teams. In addition to adherence to the Neonatal Resuscitation Program standards, team performance assessment is an essential component of program evaluation and participant feedback. Multiple published teamwork assessment tools exist. Our objective was to select the tool with the best validity evidence for our program's needs. Methods We used Messick's framework to assess the validity of evidence for potential teamwork assessment tools. Four possible tools were identified from the literature: the Mayo High Performance Teamwork Scale (Mayo), Team Performance Observation Tool (TPOT), Clinical Teamwork Scale (CTS), and Team Emergency Assessment Measure (TEAM). Relevant context included team versus individual focus, external evaluator versus self-evaluation, and ease of use (which included efficiency, clarity of interpretation, and overall assessment). Three simulation experts identified consensus anchors for each tool and independently reviewed and scored 10 pre-recorded neonatal resuscitation simulations. Raters assigned each tool a rating according to efficiency, ease of interpretation, and completeness of teamwork assessment. Interrater reliability (IRR) was calculated using intraclass correlation for each tool across the three raters. Average team performance scores for each tool were correlated with neonatal resuscitation adherence scores for each video using Spearman's rank coefficient. Results There was a range of IRR between the tools, with Mayo having the best (single 0.55 and multi 0.78). Each of the three raters ranked Mayo optimally in terms of efficiency (mean 4.66 + 0.577) and ease of use (4+1). However, TPOT and CTS scored highest (mean 4.66 ± 0.577) for overall completeness of teamwork assessment. There was no significant correlation to NRP adherence scores for any teamwork tool. Conclusion Of the four tools assessed, Mayo demonstrated moderate IRR and scored highest for its ease of use and efficiency, though not completeness of assessment. The remaining three tools had poor IRR, which is not an uncommon problem with teamwork assessment tools. Our process emphasizes the fact that assessment tool validity is contextual. Factors such as a relatively narrow (and high) performance distribution and clinical context may have contributed to reliability challenges for tools that offered a more complete teamwork assessment.

Assessing Team Performance in Complex Team-Based Command and Control Missions Through Information Sharing and Team Cohesion

Assessing Team Performance in Complex Team-Based Command and Control Missions Through Information Sharing and Team Cohesion

Enhancing Science Education Through the Student Teams Achievement Division (STAD) Learning Model: An Experimental Study on Process Skills and Learning Outcomes at Middle School

This study investigates the implementation of the Student Teams Achievement Division (STAD) Learning Model, which emphasizes teamwork and individual accountability. Traditional teaching methods often fail to actively engage students in developing analytical and practical skills. The STAD model aims to enhance students' process skills and learning outcomes in science education. The research was conducted at SMP Muhammadiyah Ambon with seventh-grade students from four classes. Two classes were randomly selected, with class VIIa as the experimental group (30 students) and another class as the control group (30 students). Data were analyzed using descriptive analysis and multivariate analysis (MANOVA) with SPSS version 26.0. A quantitative experimental approach was employed, using a Pre-test, Post-test, and Non-equivalent Control Group design. Data collection focused on science process skills and learning outcomes. The results demonstrated a significant impact of the STAD model on improving learning outcomes and science process skills. There was a notable increase in average scores from pre-test to post-test in both experimental and control groups. MANOVA results (Pillai's Trace, Wilks' Lambda, Hotelling's Trace, and Roy's Largest Root) showed very high F-values, indicating significant differences between the groups in enhancing students' science process skills and learning achievements. The STAD model proved more effective than traditional methods, suggesting its integration into a collaboration-based curriculum. Effective implementation requires STAD-based teacher training, including team formation techniques, motivational strategies, and methods for assessing team performance. Adopting these recommendations can significantly boost student participation, teamwork skills, and overall learning outcomes.

Assessing Team Performance: A Mixed-Methods Analysis Using Interprofessional in situ Simulation.

Optimizing the performance of emergency department (ED) teams impacts patient care, but the utility of current, team-based performance assessment tools to comprehensively measure this impact is underexplored. In this study we aimed to 1) evaluate ED team performance using current team-based assessment tools during an interprofessional in situ simulation and 2) identify characteristics of effective ED teams. This mixed-methods study employed case study methodology based on a constructivist paradigm. Sixty-three eligible nurses, technicians, pharmacists, and postgraduate year 2-4 emergency medicine residents at a tertiary academic ED participated in a 10-minute in situ simulation of a critically ill patient. Participants self-rated performance using the Team Performance Observation Tool (TPOT) 2.0 and completed a brief demographic form. Two raters independently reviewed simulation videos and rated performance using the TPOT 2.0, Team Emergency Assessment Measure (TEAM), and Ottawa Crisis Resource Management Global Rating Scale (Ottawa GRS). Following simulations, we conducted semi-structured interviews and focus groups with in situ participants. Transcripts were analyzed using thematic analysis. Eighteen team-based simulations took place between January-April 2021. Raters' scores were on the upper end of the tools for the TPOT 2.0 (R1 4.90, SD 0.17; R2 4.53, SD 0.27, IRR [inter-rater reliability] 0.47), TEAM (R1 3.89, SD 0.19; R2 3.58, SD 0.39, IRR 0.73), and Ottawa GRS (R1 6.6, SD 0.56; R2 6.2, SD 0.54, IRR 0.68). We identified six themes from our interview data: team member entrustment; interdependent energy; leadership tone; optimal communication; strategic staffing; and simulation empowering team performance. Current team performance assessment tools insufficiently discriminate among high performing teams in the ED. Emergency department-specific assessments that capture features of entrustability, interdependent energy, and leadership tone may offer a more comprehensive way to assess an individual's contribution to a team's performance.

Analyzing Players' Tactical Positions and Movement Trajectories in Soccer Matches Using Machine Vision Algorithms

Tactical positions and movement trajectories in soccer matches using machine vision algorithms involves leveraging computer vision techniques to track players and the ball throughout the game. By processing video feeds from multiple cameras, machine vision algorithms can identify players, recognize their positions on the field, and map out their movement trajectories over time. This analysis provides valuable insights into team formations, player positioning, and tactical strategies employed during matches. Coaches and analysts can use this information to assess team performance, identify patterns in gameplay, and make data-driven decisions to optimize tactics and training regimes. Machine vision algorithms enhance the understanding of soccer dynamics, facilitating more strategic and effective approaches to coaching and gameplay.. This paper presents a comprehensive analysis of player movement, tactical positions, and detection in soccer matches using machine learning algorithms. Leveraging a simulation environment built upon a Recurrent Neural Network (RNN) model trained on historical match data, we investigate the efficacy of the model in accurately predicting future player positions based on contextual features such as player trajectories and ball movement. Furthermore, we explore the model's ability to estimate tactical movements during specific time intervals, providing valuable insights for coaches and analysts in understanding team strategies and adapting to match dynamics. Additionally, we evaluate player detection systems to assess their capabilities and limitations, laying the groundwork for future improvements in player tracking technologies. Our findings underscore the potential of machine learning in soccer analytics, offering actionable insights for enhancing performance analysis, strategic decision-making, and overall understanding of the game. Our results show a high prediction accuracy, with an average Mean Squared Error (MSE) of 0.05. Furthermore, we explore the model's ability to estimate tactical movements during specific time intervals, achieving an overall prediction accuracy of 85%. Additionally, we evaluate player detection systems to assess their capabilities and limitations, achieving a detection accuracy of 90%.

Flipped Classroom: Improved team performance during resuscitation training through interactive pre-course content – a cluster-randomised controlled study

BackgroundResuscitation is a team effort, and it is increasingly acknowledged that team cooperation requires training. Staff shortages in many healthcare systems worldwide, as well as recent pandemic restrictions, limit opportunities for collaborative team training. To address this challenge, a learner-centred approach known as flipped learning has been successfully implemented. This model comprises self-directed, asynchronous pre-course learning, followed by knowledge application and skill training during in-class sessions. The existing evidence supports the effectiveness of this approach for the acquisition of cognitive skills, but it is uncertain whether the flipped classroom model is suitable for the acquisition of team skills. The objective of this study was to determine if a flipped classroom approach, with an online workshop prior to an instructor-led course could improve team performance and key resuscitation variables during classroom training.MethodsA single-centre, cluster-randomised, rater-blinded study was conducted on 114 final year medical students at a University Hospital in Germany. The study randomly assigned students to either the intervention or control group using a computer script. Each team, regardless of group, performed two advanced life support (ALS) scenarios on a simulator. The two groups differed in the order in which they completed the flipped e-learning curriculum. The intervention group started with the e-learning component, and the control group started with an ALS scenario.Simulators were used for recording and analysing resuscitation performance indicators, while professionals assessed team performance as a primary outcome.ResultsThe analysis was conducted on the data of 96 participants in 21 teams, comprising of 11 intervention groups and 10 control groups. The intervention teams achieved higher team performance ratings during the first scenario compared to the control teams (Estimated marginal mean of global rating: 7.5 vs 5.6, p < 0.01; performance score: 4.4 vs 3.8, p < 0.05; global score: 4.4 vs 3.7, p < 0.001). However, these differences were not observed in the second scenario, where both study groups had used the e-learning tool.ConclusionFlipped classroom approaches using learner-paced e-learning prior to hands-on training can improve team performance.Trial registrationGerman Clinical Trials Register (https://drks.de/search/de/trial/DRKS00013096).

Team performance during vacuum-assisted vaginal delivery: video review of obstetric multidisciplinary teams.

Vacuum extraction is generally considered an operator-dependent task, with most attention directed toward the obstetrician's technical abilities (1-3). Little is known about the effect of the team and non-technical skills on clinical outcomes in vacuum-assisted delivery. This study aimed to investigate whether the non-technical skills of obstetricians were correlated with their level of clinical performance via the analysis of video recordings of teams conducting actual vacuum extractions. We installed between two or three video cameras in each delivery room at Aarhus University Hospital and Horsens Regional Hospital and obtained 60 videos of teams managing vacuum extraction. Appropriate consent was obtained. Two raters carefully reviewed the videos and assessed the teams' non-technical skills using the Assessment of Obstetric Team Performance (AOTP) checklist, rating all items on a Likert scale score from 1 to 5 (1 = poor; 3 = average; and 5 = excellent). This resulted in a total score ranging from 18 to 90. Two different raters independently assessed the teams' clinical performance (adherence to clinical guidelines) using the TeamOBS-Vacuum-Assisted Delivery (VAD) checklist, rating each item (0 = not done, 1 = done incorrectly; and 2 = done correctly). This resulted in a total score with the following ranges (low clinical performance: 0-59; average: 60-84; and high: 85-100). Interrater agreement was analyzed using intraclass correlation (ICC), and the risk of high or low clinical performance was analyzed on a logit scale to meet the assumption of normality. Teams that received excellent non-technical scores had an 81% probability of achieving high clinical performance, whereas this probability was only 12% among teams with average non-technical scores (p < 0.001). Teams with a high clinical performance often had excellent behavior in the non-technical items of "team interaction," "anticipation," "avoidance fixation," and "focused communication." Teams with a low or average clinical performance often neglected to consider analgesia, had delayed abandonment of the attempted vaginal delivery and insufficient use of appropriate fetal monitoring. Interrater reliability was high for both rater-teams, with an ICC for the non-technical skills of 0.83 (95% confidence interval [CI]: 0.71-0.88) and 0.84 for the clinical performance (95% CI: 0.74-0.90). Although assisted vaginal delivery by vacuum extraction is generally considered to be an operator-dependent task, our findings suggest that teamwork and effective team interaction play crucial roles in achieving high clinical performance. Teamwork helped the consultant anticipate the next step, avoid fixation, ensure adequate analgesia, and maintain thorough fetal monitoring during delivery.

Understanding the role of gaze behaviour and coaching experience in the assessment of youth soccer teams

ABSTRACT This study aimed to explore the perceptual-cognitive characteristics of coaches as they assessed team performance in youth soccer. The primary focus was to investigate the alignment between coaches’ subjective analyses of team behaviour and objective analyses, while also examining the relationship between coaches’ gaze behaviour and their levels of coaching experience, particularly considering the potential differences that may exist among coaches with varying levels of experience. Sixty-five male and female adults with various soccer coaching experience (experienced, novice, other team sport experience, and non-team sport experience) watched five 4-minute videos and assessed team behaviour. These subjective evaluations were compared to objective data obtained from video analysis and GPS, which included measures of completed skills, spatiotemporal characteristics, and passing networks. The participants’ fixation duration and frequency were measured for each video clip, and the area around the ball specifically. The study found no significant differences between groups regarding the number of times participants’ subjective analysis aligned with the objective data (p = 0.059, ɳ2 = 0.07). However, coaches with soccer coaching experience demonstrated a higher fixation frequency and more revisits to the ball area when compared to participants without soccer experience (p = <.001, ɳ2 = 0.09). The current study offers a unique approach to uncovering soccer coaching expertise by combining objective and subjective evaluations of team performance. In summary, the study reveals that coaching experience did not impact how often coaches subjective analyses matched objective data. However, soccer coaches had distinctive gaze behaviour patterns where they revisited the area around the ball more often.

A comparison of rapid cycle deliberate practice and traditional reflective debriefing on interprofessional team performance

BackgroundIn simulation-based education, debriefing is necessary to promote knowledge acquisition and skill application. Rapid Cycle Deliberate Practice (RCDP) and Traditional Reflective Debriefing (TRD) are based in learning theories of deliberate practice and reflective learning, respectively. In this study, we compared the effectiveness of TRD versus RCDP on acquisition of conceptual knowledge and teamwork skills among interdisciplinary learners in the pediatric emergency department.MethodsOne hundred sixty-four learners including emergency department attending physicians, fellows, nurses, medical technicians, paramedics, and respiratory therapists, participated in 28 in-situ simulation workshops over 2 months. Groups were quasi-randomized to receive RCDP or TRD debriefing. Learners completed a multiple-choice test to assess teamwork knowledge. The TEAM Assessment Tool assessed team performance before and after debriefing. Primary outcomes were teamwork knowledge and team performance.ResultsAverage pre-intervention baseline knowledge assessment scores were high in both groups (TRD mean 90.5 (SD 12.7), RCDP mean 88.7 (SD 15.5). Post-test scores showed small improvements in both groups (TRD mean 93.2 (SD 12.2), RCDP mean 89.9 (SD 13.8), as indicated by effect sizes (ES = 0.21 and 0.09, for TRD and RCDP, respectively). Assessment of team performance demonstrated a significant improvement in mean scores from pre-assessment to post-assessment for all TEAM Assessment skills in both TRD and RCDP arms, based on p-values (all p < 0.01) and effect sizes (all ES > 0.8). While pre-post improvements in TEAM scores were generally higher in the RCDP group based on effect sizes, analysis did not indicate either debriefing approach as meaningfully improved over the other.ConclusionsOur study did not demonstrate that either TRD versus RCDP was meaningfully better in teamwork knowledge acquisition or improving skill application and performance. As such, we propose Reflective Deliberate Practice as a framework for future study to allow learners to reflect on learning and practice in action.

A psychometric evaluation of inter-professional education competency tool in nursing and medicine students

IntroductionCollaboration between nurses and doctors is necessary for offering care to patients. Using team performance assessment tools and surveying them can be effective in promoting inter-professional collaboration, and the lack of a credible tool to assess inter-professional collaboration competency between the two groups is a major challenge in the healthcare sector. The present study aimed to translate and conduct a psychometric investigation on the inter-professional education collaboration (IPEC) tool for the students of medicine and nursing.MethodsThe present study was a cross-sectional one conducted as a psychometric investigation of the IPEC tool at the Iran University of Medical Sciences in 2022. The initial tool contained 42 items developed according to a 5-point Likert scale, which was translated into Persian with the consent of the original researcher. The validity index and the content validity ratio were investigated by a panel of 11 specialists in medical and clinical education, and its construct validity was evaluated using confirmatory factor analysis. Also, the second population of the study included medical and nursing students of Iran University of Medical Sciences and simple random sampling method. Moreover, the reliability of the instrument was investigated using internal consistency, Cronbach’s Alpha, and test–retest methods.ResultsBased on the indicators calculated to perform a psychometric investigation over the above tool, it had acceptable reliability and validity according to the specialists. The tool evaluates inter-professional collaboration competency between the students of medicine and nursing across four areas (values and ethics, roles and responsibilities, inter-professional communication, and team-based care and teamwork). Moreover, Cronbach’s Alpha coefficient for the tool was determined at 0.84.ConclusionThe results of the study showed that the above tool could evaluate inter-professional competency as a valid and reliable questionnaire, and its results could be utilized in planning and education.

Impact of Just-in-Time TeamSTEPPS Training on Team Performance in a Pediatric Escape Room Interprofessional Experience

ObjectiveTo compare differences in observed and self-reported team dynamics between those who received versus those who did not receive TeamSTEPPS training prior to an interprofessional escape room scenario. MethodsFinal-year pharmacy (N = 27) and physician assistant (N = 88) students participated in an escape room featuring pediatric dosing, asthma, otitis media, type 1 diabetes, and vaccine fundamentals. In total, 29 teams were randomized to control (14) or intervention arm (15). The control group went to the escape room and then returned to the classroom for a team-building module including TeamSTEPPS training. The intervention group started with the team-building module then went to the escape room. One faculty observer was assigned to each team and utilized the Performance Assessment Communication and Teamwork (PACT) expert-observer form to assess team performance. Post-event, students completed an abbreviated PACT post-assessment and a self-reflection on the prompt “What were the most important takeaways from today's interprofessional education experience?” ResultsOn the PACT expert-observer form, significantly higher global scores were noted for the intervention arm on all items except situation monitoring. On the aggregate observer data from all stations, leadership emerged as statistically different between the 2 groups. On the PACT post-assessment, the intervention arm reported significantly more leadership during the simulation. On the self-reflections, 70.9% versus 64% in the control arm recognized the importance of teamwork in managing patients. ConclusionsJust-in-time TeamSTEPPS training improved observed and to a lesser extent, self-reported teamwork and communication skills. Incorporating TeamSTEPPS training immediately prior to an interprofessional education experience may enhance team performance.

Exploring objective measures for assessing team performance in healthcare: an interview study

IntroductionEffective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach.MethodsA total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question.ResultsThe findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures.DiscussionThe study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research.

Self- and Team-Efficacy in Military Trainees: Development of Two Novel Military Trainee-Appropriate Assessments

Many currently published studies examining elements of self- and team-efficacy in military personnel have not utilized military-specific assessments, thereby limiting the generalizability of their findings to the broader military population. The present study evaluates the factor structure and performance correlates of the Trainee Self-Efficacy Scale (TSES) and the Team-Efficacy and Training Climate Inventory (TETCI) in two samples of military trainees. Exploratory factor analysis was conducted in sample 1 (N = 481). Confirmatory factor analysis, measure validity, and structure validation were conducted in sample 2 (N = 442). Factor analysis results revealed the TSES was best represented by a three-factor correlated traits model including constructs related to performance confidence, performance uncertainty, and performance optimism. The TETCI was best represented by a two-factor correlated traits model including constructs related to team member efficacy and leadership climate. TSES subscales exhibited small or moderate correlations with assessments of soldier performance (e.g., physical fitness, land navigation, and confidence in rifle marksmanship) in the expected directions. The TETCI subscales exhibited small positive correlations with an assessment of team performance. Findings indicate that the TSES and TETCI are parsimonious measures (10 and 7-items, respectively) that have the potential to be useful tools for the continued assessment of military trainees.

A new standardized tool for quantification of closed-loop communication in trauma care: CAST Grid reliability study

BackroundThe CAST Grid has been developed to evaluate the use of closed-loop communication (CLC) in the trauma bay. MethodsThe CAST Grid and two validated non-technical team performance assessment tools (the TEAM and T-NOTECHS grids) were completed by 2 independent reviewers based on trauma care simulation videos from a French Level 1 trauma center. Intra- and inter-rater agreements were evaluated for CLC parameters and non-technical performance, and correlations between these parameters were analyzed. ResultsThe study analyzed 11 videos. The intra- and inter-rater agreement for the number of CLC per minute (CLC/min) was moderate and good, respectively, based on Lin's concordance correlation coefficient [95%CI] (0.57 [−0.40;0.94] and 0.77 [0.33;0.94]). However, the agreement was poor for the percentage of CLC (0.37 [−0.58;0.89] and -0.36 [−0.71;0.14], respectively). The study found that a lower number of CLC/min was correlated with an increased duration of the simulation (r = −0.75 [−0.93; −0.25]). ConclusionThe CAST Grid showed a relatively good inter-rater agreement to quantify the number of CLC/min which was inversely correlated with the duration of care. This tool opens up the possibility of quantifying CLC and allows for new analyses of team functioning and interactions.

Reliability and validity testing of team emergency assessment measure in a distributed team context.

Medical multi-professional teams are increasingly collaborating via telemedicine. In distributed team settings, members are geographically separated and collaborate through technology. Developing improved training strategies for distributed teams and finding appropriate instruments to assess team performance is necessary. The Team Emergency Assessment Measure (TEAM), an instrument validated in traditional collocated acute-care settings, was tested for validity and reliability in this study when used for distributed teams. Three raters assessed video recordings of simulated team training scenarios (n = 18) among teams with varying levels of proficiency working with a remotely located physician via telemedicine. Inter-rater reliability, determined by intraclass correlation, was 0.74-0.92 on the TEAM instrument's three domains of leadership, teamwork, and task management. Internal consistency (Cronbach's alpha) ranged between 0.89-0.97 for the various domains. Predictive validity was established by comparing scores with proficiency levels. Finally, concurrent validity was established by high correlations, >0.92, between scores in the three TEAM domains and the teams' overall performance. Our results indicate that TEAM can be used in distributed acute-care team settings and consequently applied in future-directed learning and research on distributed healthcare teams.

Virtual escape room paired with simulation improves health professions students' readiness to function in interprofessional teams

BackgroundThis project investigated the change in nursing, pharmacy, and physical therapy students' interprofessional socialization after participation in a virtual interprofessional escape room and case conference simulation. Interprofessional education activityInterprofessional teams of nursing (n = 93), pharmacy (n = 75) and physical therapy (n = 33) students completed asynchronous, online learning (sepsis recognition and total hip replacement post-operative precautions) followed by a virtual escape room and a virtual simulated patient case conference. During the case conference, interprofessional student teams developed a discharge plan for an individual after a hip replacement complicated by post-operative sepsis. Before and after the experience, students completed a knowledge test and a validated survey instrument that assessed their interprofessional socialization (Interprofessional Socialization and Valuing Scale-21). During the simulated patient case conference, faculty assessed student performance using a standardized rubric. After the experience students completed a program evaluation. DiscussionInterprofessional socialization significantly increased (5.5 ± 0.9 vs. 6.0 ± 0.9) among all students with a medium effect size (Cohen's d = 0.56). Faculty assessment of individual student's team performance during the virtual simulation revealed a moderate rate meeting competency, with good interrater reliability. Students highly valued this learning experience as being both effective and important to their professional development, as indicated on the program evaluation. ImplicationsA virtual interprofessional experience consisting of asynchronous online learning, a virtual escape room, and a virtual case conference positively influenced students' interprofessional socialization. Students valued the experience and recognized its importance in their development as student health professionals.