Situation Awareness Global Assessment Technique Research Articles

Measuring Performance in an Electronic Health Record (EHR) based Simulation Using the Situation Awareness Global Assessment Technique

Simulation is an educational approach well suited to development of knowledge and decision-making skills for emergent or infrequent scenarios. Electronic Health Record (EHR) based simulation, in which participants retrieve information from a simulated EHR, provides an authentic training environment with fidelity to the typical clinical decision-making process and has been associated with enduring changes in EHR use patterns. However, we do not know whether these behavior changes reflect better decision-making. We aimed to develop a measure of pediatric resident performance in an EHR based simulation using the Situation Awareness Global Assessment Technique (SAGAT).

An observational study using eye tracking to assess resident and senior anesthetists' situation awareness and visual perception in postpartum hemorrhage high fidelity simulation.

BackgroundThe postpartum hemorrhage (PPH) is the leading cause of maternal mortality in the world. Human factors and especially situation awareness has primarily responsibility to explain suboptimal cares. Based on eye tracking and behavior analysis in high fidelity simulation of PPH management, the goal of this study is to identify perceptual and cognitive key parameters of the expertise.MethodsTwo groups of fifteen anesthetists (residents and experienced anesthetists) watched the beginning of a severe simulated PPH management. During this first experimental phase, situation awareness was assessed using SAGAT (Situation Awareness Global Assessment Technique) questionnaire and visual behavior was analyzed with eye tracking. In the continuity of the video sequence, they have to step in the PPH situation and to provide care to the simulated patient. Performance of cares was evaluated and self-assessed as well as cognitive load.ResultsNo statistical difference between the residents and experienced anesthetists was observed on performance of simulated PPH management. The mean expected practice score was 76.9 ± 13.9%). Assessment of situation awareness (65 ± 7%), cognitive load (74.4 ± 11.3%) and theoretical knowledge of PPH (52.4 ± 3.5%) were also not statistically different between the two groups. Only results of self-assessed performance (respectively 66.1 ± 16.6 and 47.0 ± 20.8 for experts and residents) and eye-tracking data revealed that experts tended to get accurate evaluation of their performance and to monitor more the blood loss of the patient. Experts have in average 8.28% more fixating points than Novices and gazed the blood loss region longer (865 ms ± 439 vs. 717 ms ± 362).ConclusionsThis study pointed out the limits of classical assessment of performance, and human factors based on questionnaires to identify expertise in simulated PPH care. A neuroscientific approach with new technology like eye tracking could provide new objective and more sensitive insights on human factors in simulated medical emergency situations.

Assessing situation awareness in multitasking supervisory control using success rate of self-terminating search

The purpose of this study is to investigate measuring situation awareness (SA) by using the success rate of self-terminating visual search (STVS) in a multitasking supervisory control environment. Currently, many researchers have focused on developing unobtrusive SA metrics in a multitasking environment. Although some of the advanced eye-tracking metrics can estimate the operator's SA during a task, certain methodological issues remain for measuring SA in a supervisory control and multitasking environment. In this study, participants performed a distributed control task with the multi-attribute task battery. To measure the participant's SA, situation awareness global assessment technique (SAGAT) was used during the experiment. SA performance was compared to the success rate of STVS. An eye-head integrated tracking system was used to measure the rate of STVS based on the participants' eye movements and head positions in different monitoring conditions. The results showed that task-switching activity was influenced by salient features in a display. It causes a significant difference on SA between different monitoring conditions, and the difference is also monitored in the success rate of STVS. Furthermore, the study found that the attention level in visual searching influences situational awareness in a supervisory control environment. The findings from this study suggest that the success rate of STVS could be used as a new metric for measuring SA related to the visual perception of the elements in a multitasking supervisory control environment.

Effects of working memory, attention, and expertise on pilots’ situation awareness

The current study investigates individual differences that predict situation awareness (SA) in professional pilots. The aim of the study is twofold: to examine the roles of divided attention, inhibition, working memory, and expertise in predicting SA, and to demonstrate the relative contributions of these individual differences to online (Situation Awareness Present Method, SPAM) and offline (Situation Awareness Global Assessment Technique, SAGAT) SA measures. Thirty-six professional pilots completed a challenging flight scenario in a full-flight simulator. Divided attention, inhibition, working memory span, and expertise were measured using choice reaction time with dichotic listening, Stroop, and Automated Operation Span tasks, and flight hours in a full-flight simulator, respectively. Results indicated that offline and online SA measure were not correlated, supporting their concurrent use to obtain a comprehensive measure of SA. Offline SA scores were best predicted by working memory and level of expertise, while online SA scores were predicted by expertise, divided attention and inhibition. Results are discussed focusing on both theoretical contributions for defining and measuring SA and applications. Findings have implications for operators of critical domains and their interactions with automated systems, in which SA is crucial for performance and safety.

Measuring situation awareness and team effectiveness in pediatric acute care by using the situation global assessment technique

Situation awareness (SA) is an important human factor and necessary for effective teamwork and patient safety. Human patient simulation (HPS) with video feedback allows for a safe environment where health care professionals can develop both technical and teamwork skills. It is, however, very difficult to observe and measure SA directly. The Situation Global Assessment Technique (SAGAT) was developed by Endsley to measure SA during real-time simulation. Our objective was to measure SA among team members during simulation of acute pediatric care scenarios on the medical ward and its relationship with team effectiveness. Twenty-four pediatric teams, consisting of two nurses, one resident, and one consultant, participated in three acute care scenarios, using high-fidelity simulation. Individual SAGAT scores contained shared and complimentary knowledge questions on different levels of SA. Within each scenario, two “freezes” were incorporated to assess SA of each team members’ clinical assessment and decision-making. SA overlap within the team (team SA) was computed and compared to indicators of team effectiveness (time to goal achievement, consensus on primary problem, diagnosis, task prioritization, leadership, and teamwork satisfaction). In 13 scenarios (18%), the team failed to reach the primary goals within the prescribed time of 1200 s. There was no significant difference in failure of goal completion between the scripted scenarios; however, there was a significant difference between scenario 3 and the other scenarios in time to goal completion. In all three scenarios, SA overlap level 2 (consensus on primary problem during the first freeze and consensus on diagnosis during the second freeze) leads to significantly faster achievement of the predefined goals. There was a strong relationship between team SA on the primary problem and diagnosis and team SA on task prioritization. Consensus on leadership within the team was low. Teamwork satisfaction was more influenced by knowledge about the importance of the assigned task than outcome of the scenario.Conclusion: The use of SAGAT enables us to measure SA of team members during real-time simulation of acute care scenarios. Although there is no direct connection between team SA and goal achievement, SAGAT provides insight in differences in SA among team members, and the process of shared mental model formation. By measuring SA, issues that may improve team effectiveness (prioritizing tasks, enhancing shared mental models, and providing leadership) can be trained and assessed during medical team simulation, enhancing teamwork in health care settings.What is known?• Teamwork skills such as communication, leadership, and situational awareness have become increasingly recognized as essential for good performance in pediatric resuscitation. However, the assessment of pediatric team performance in these clinical situations has been traditionally difficult.• The Situation Awareness Global Assessment Technique (SAGAT) is a method of objectively and directly measuring SA during a team simulation using “freezes” at predetermined points in time with participants reporting on “what is going on” from their perspective on the situation.What is new?• We assessed SA, and its relationship with team effectiveness, in multidisciplinary pediatric teams performing simulated critical events in critically ill children on the medical ward using the SAGAT model, outside the emergency room setting.• In all three scenarios, consensus on the primary problem (shared mental model) leads to faster achievement of predefined goals. Consensus on leadership was overall low, without a significant impact on goal achievement.

The effects of passive mobile phone interaction on situation awareness and driving performance

The present study examines the effects of mobile phone dependency and passive mobile phone interaction on situation awareness and driving performance. A total of 37 participants drove in a fixed-base driving simulator that included rural roads followed by typical city roads with light traffic. As participants were driving, they received an audible text message notification to which they were unable to respond. After receiving the text message, participants’ situation awareness was assessed via the situation awareness global assessment technique (SAGAT). SAGAT probes occurred at 30 s and 10 s after the text message was delivered to determine the effect of text message delay on situation awareness. Results indicate that situation awareness is negatively affected for at least 10 s after the text message alert but not longer than 30 s. In addition to lowered situation awareness, participants’ mean speed increased during periods of distraction in the 10 s following a mobile phone notification. Low mobile phone dependency did not serve as a protective factor against distraction in that participants were similarly distracted irrespective of dependency level. The implications of passive mobile phone interaction, as well as possible solutions are discussed.

Proceed with caution: using verbal protocol analysis to measure situation awareness

Verbal protocol analysis (VPA) is often used to elicit information about the cognitive processes of operators as it provides rich data and can be used in naturalistic settings. Recently VPA has been used to investigate the acquisition and maintenance of situation awareness (SA), and to make comparisons between groups despite a lack of research regarding the efficacy of using VPA for this purpose. This train simulator experiment investigated whether VPA can effectively measure SA. Novice and expert participants were recorded on an audio device while talking aloud throughout the trials and their verbalisations were transcribed verbatim. A coding scheme developed from the transcripts was used to code the verbalisations. Results did not support the use of VPA as a measure of SA but did show that VPA detected differences in SA errors. Potential reasons for the conflicting findings between this experiment and those conducted by other researchers are discussed.Practitioner summary: This paper examined the validity of verbal protocol analysis (VPA) as a situation awareness (SA) measure. A repeated measures experiment was conducted using a train simulator. Normal VPA did not detect changes in SA but a measure of errors did. Caution should be used when using VPA to measure SA.Abbreviations: LETSSA: low-event task subjective situation awareness technique; SA: situation awareness; SAGAT: situation awareness global assessment technique; TPD: train performance display; VPA: verbal protocol analysis

An Adaptive System Based on Situation Awareness for Goal-Driven Management in Container Terminals

Lack of situation awareness when dealing with complex dynamic environments is recognized as one of the main causes of human errors that may lead to serious incidents, poor performance, etc. Thus, there is a need to define systems able to support operators in focusing their attention on active goals and, when really needed, switching them on more suitable goals, in a sort of continuous adaptation. This paper proposes an adaptive goal selection approach capable of suggesting the goal on which human operators should focus their attention to maintain an adequate level of situation awareness. The approach has been implemented in a Decision Support System for the management of operations in a port container terminal. An evaluation of the system applied to the container terminal of Salerno (Italy) has been realized by using the Situation Awareness Global Assessment Technique (SAGAT), a well-known methodology for evaluating SA in operational contexts. An evaluation of the performance has been executed to demonstrate that the approach helps to improve the performance of the logistics operations.

Situation awareness based on eye movements in relation to the task environment

The topic of situation awareness has received continuing interest over the last decades. Freeze-probe methods, such as the Situation Awareness Global Assessment Technique (SAGAT), are commonly employed for measuring situation awareness. The aim of this paper was to review validity issues of the SAGAT and examine whether eye movements are a promising alternative for measuring situation awareness. First, we outlined six problems of freeze-probe methods, such as the fact that freeze-probe methods rely on what the operator has been able to remember and then explicitly recall. We propose an operationalization of situation awareness based on the eye movements of the person in relation to their task environment to circumvent shortfalls of memory mediation and task interruption. Next, we analyzed experimental data in which participants (N = 86) were tasked to observe a display of six dials for about 10 min, and press the space bar if a dial pointer crossed a threshold value. Every 90 s, the screen was blanked and participants had to report the state of the dials on a paper sheet. We assessed correlations of participants’ task performance (% of threshold crossing detected) with visual sampling scores (% of dials glanced at during threshold crossings) and freeze-probe scores. Results showed that the visual-sampling score correlated with task performance at the threshold-crossing level (r = 0.31) and at the individual level (r = 0.78). Freeze-probe scores were low and showed weak associations with task performance. We conclude that the outlined limitations of the SAGAT impede measurement of situation awareness, which can be computed more effectively from eye movement measurements in relation to the state of the task environment. The present findings have practical value, as advances in eye-tracking cameras and ubiquitous computing lessen the need for interruptive tests such as SAGAT. Eye-based situation awareness is a predictor of performance, with the advantage that it is applicable through real-time feedback technologies.

Quantifying Distraction Potential of Augmented Reality Head-Up Displays for Vehicle Drivers

Background: A recent National Highway Traffic & Safety Administration (NHTSA) report states that 10% of fatal crashes and 18% of injury crashes were reported as distraction-affected crashes. In that same year, 3,179 people were killed and an estimated 431,000 injured in motor vehicle crashes involving distracted drivers, many of which involved secondary visual displays (NHTSA, 2016). Augmented reality (AR) head-up displays (HUD) promise to be less distractive than traditional in-vehicle displays since they do not take drivers’ eyes off the road (Gabbard, Fitch, & Kim, 2014). However, empirical studies have reported possible negative consequences of AR HUDs, in part, due to AR graphics’ salience (Sharfi & Shinar, 2014), frequent changes (Wolffsohn, McBrien, Edgar, & Stout, 1998), and visual clutter (Burnett & Donkor, 2012). Moreover, current in-vehicle display assessment methods which are based on eye-off-road time measures (NHTSA, 2012), cannot capture this unique challenge. Objective: This work aims to propose a new method for the assessment of AR HUDs by quantifying both positive (informing drivers) and negative (distracting drivers) consequences of AR HUDs which might not be captured by current in-vehicle display assessment methods. Method: We proposed a new way of quantifying the distraction potential of AR HUDs by measuring driver situation awareness with operational improvements on the situation awareness global assessment technique (Endsley, 2012) to suit AR usability evaluations. A human-subject experiment was conducted in a driving simulator to apply the proposed method and to evaluate two AR HUD interfaces for pedestrian collision warning. The AR warning interfaces were prototyped by the augmented video technique (Soro, Rakotonirainy, Schroeter, & Wollstdter, 2014). Twenty-four participants drove while interacting with different types of AR pedestrian collision warning interfaces (no warning, bounding box, and virtual shadow). Drivers’ situation awareness, confidence, and workload were measured and compared to the no warning condition. Results: Only one of the warning interface designs, the virtual shadow (Kim, Isleib, & Gabbard, 2016), improved driver situation awareness about pedestrians which were cued by the AR HUD, not affecting situation awareness about other environmental elements which were not augmented by the HUD. The experiment also showed drivers’ overconfidence bias while interacting with the bounding box which is another warning interface design. The empirical user study did not provide any evidence for reduced driver workload when AR warnings were given. Conclusion: Our initial human-subject study demonstrated a potential of the proposed method in quantifying both positive and negative consequences of AR HUDs on driver cognitive processes. More importantly, the experiment showed that AR interfaces can have both positive and negative consequences on driver situation awareness depending upon how we design perceptual forms of graphical elements. Application: The proposed assessment methods for AR HUDs can inform not only comparative evaluation among design alternatives but also assist in incrementally improving design iterations to better support drivers’ information needs, situation awareness, and in turn, performance, and safety.

Using Situation Awareness and Workload to Predict Performance in Submarine Track Management: A Multilevel Approach.

Examine the extent to which subjective workload and situation awareness (SA) can predict variance in performance at the between- and within-person levels of analysis in a simulated submarine track management task. SA and workload are crucial constructs in human factors that are conceptualized as states that change within individuals over time. Thus, a change in an individual's subjective workload or SA over the course of performing a task should be predictive of their subsequent performance (within-person effects). However, there is little empirical evidence for this. Participants monitored displays to track the behaviors of contacts in relationship to their own ship (Ownship) and landmarks. The Situational Awareness Global Assessment Technique measured SA, and the Air Traffic Workload Input Technique measured subjective workload. When a participant's subjective workload rating increased, their subsequent performance decreased, but there was no evidence for within-person effects of SA on performance. We replicated prior between-person level effects of SA; participants with higher SA performed better than those with lower SA. Change in an individual's subjective workload rating (but not SA) was predictive of their subsequent performance. Because an increase in SA should increase the extent to which operators hold the knowledge required to perform subsequent tasks, further research is required to examine SA effects on performance at the within-person level. Adapting automation is more likely to produce optimal outcomes if based on measurement of operator states that predict future task performance, such as workload.

PENGARUH PERHATIAN TERBAGI TERHADAP KESADARAN SITUASI

The increasing number of traffic accident is caused by the decreasing of driver situational awareness. Divided attention is one of several factors that affect situation awareness. This experiment was to study the effects of listening to the music or reading short message sent to participant’s mobile phone on participant’s situational awareness while they drove car simulator. Thirty subjects were randomly divided into two groups, namely: (a) music group. The subjects were asked to listen to Ed Sheeran pop music when they drove city car simulator, and (b) mobile phone group. In this condition the subjects received and replied short message on their mobile phones at third, fifth, eighth, and tenth minutes during driving car simulator. Situational awareness was measured by SAGAT (Situation Awareness Global Assessment Technique). The simulation was stopped when SAGAT questionnaire was distributed to the subjects at fifth and tenth minutes. The t test statistical analysis showed that there was a significant difference in situation awareness between the two groups (t= 5.750; p= .001). Music group showed higher situation awareness than mobile phone group. The use of mobile phone while driving is more distracting attention than listening to music.

Development and Implementation of a Situation Awareness Workshop to Advance Safe Practice in Novice Nurses.

The current study evaluated the effectiveness of a workshop designed to provide nurses (n = 6) with the means to improve their situation awareness. The nurses participated in a full-day workshop in which their situation awareness performance was measured before and after using the Situation Awareness Global Assessment Technique. Following the workshop, nurses demonstrated improvement in their perception of relevant cues, comprehension of what the combination of cues means, and projection of the patient's future status. The implications of situation awareness education for health-care professionals are discussed.

Simulation-based evaluation of an in-vehicle smart situation awareness enhancement system

Situation awareness (SA) constitutes a critical factor in road safety, strongly related to accidents. This paper describes the evaluation of a proposed SA enhancement system (SAES) that exploits augmented reality through a head-up display (HUD). Two SAES designs were evaluation (information rich vs. minimal information) using a custom-made simulator and the Situation Awareness Global Assessment Technique with performance and EEG measures. The paper describes the process of assessing the SA of drivers using the SAES, through a series of experiments with participants in a Cave Automatic Virtual Environment. The effectiveness of the SAES was tested in a within-group research design. The results showed that the information rich (radar-style display) was superior to the minimal (arrow hazard indicator) design and that both SAES improved drivers’ SA and performance compared to the control (no HUD) design.Practitioner Summary: Even though driver situation awareness is considered as one of the leading causes of road accidents, little has been done to enhance it. The current study demonstrates the positive effect of a proposed situation awareness enhancement system on driver situation awareness, through an experiment using virtual prototyping in a simulator.

Developing a Cognitive Battery for Top-Down Workload Assessment

Event Abstract Back to Event Developing a Cognitive Battery for Top-Down Workload Assessment Amanda E. Kraft1*, Matthias D. Ziegler1, Sophia Mayne-DeLuca1, Trevor Sands1, Alison M. Perez1, Jesse Mark2, Adrian Curtin2, Amanda Sargent2, Hasan Ayaz2, 3, 4 and William Casebeer1 1 Lockheed Martin (United States), United States 2 School of Biomedical Engineering, Science and Health Systems, Drexel University, United States 3 Department of Family and Community Health, School of Nursing, University of Pennsylvania, United States 4 General Pediatrics, Children's Hospital of Philadelphia, United States Computational models of workload are often specialized to specific types of tasks and do not transfer between tasks. Part of this is due to the reliance of building models based on similar training tasks that enable detection of comparable neural indicators. As such, many workload prediction systems currently rely on tasks of similar design and interaction targeting specific cognitive functions. To address this, we developed personalized models in our study from a battery of standard cognitive tasks and mapped them to a dynamic operational environment, requiring simultaneous use of a combination of these cognitive functions. We first discuss the design of the baseline battery of cognitive tasks and how the components translate to a more dynamic two-person mission. We selected and modified the following six standard cognitive workload tasks to comprise a 60-minute baseline battery assessment: Stop Signal Task, Conjunctive Continuous Performance Task (CCPT), Spatial Span Working Memory Task, Trail Making Task (TMT), Situation Awareness Global Assessment Technique (SAGAT), and Balloon Analogue Risk Task (BART). All tasks were adapted in at least two ways: (1) timing and number of trials to support use of functional neuroimaging with block design and enable 60-minute max across all 6 tasks; and (2) visually to reflect icons and styles expected in the final task. We then evaluate the effectiveness of the workload-based performance predictions for each cognitive function in guiding task reassignment in a two-person mission, as compared to reassignment based on performance alone. The two-person mission is composed of multiple interrelated subtasks with the overall goal of finding and identifying as many targets as possible in the time allowed. Each of the subtasks will require use of one or more cognitive functions, mapping to those tested in the baseline battery. By including workload indicators across the cognitive functions, we anticipate enabling reassignment prior to performance reduction, allowing for overall increase in performance while maintaining sustainable workload across the teammates. Acknowledgements This research was supported by the Air Force Research laboratory’s Human Performance Sensing BAA call 002. Keywords: baseline battery, baseline battery design, Workload Assessment, Cognitive Modeling, Performance assessment Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Oral Presentation Topic: Neuroergonomics Citation: Kraft AE, Ziegler MD, Mayne-DeLuca S, Sands T, Perez AM, Mark J, Curtin A, Sargent A, Ayaz H and Casebeer W (2019). Developing a Cognitive Battery for Top-Down Workload Assessment. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00028 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 02 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Ms. Amanda E Kraft, Lockheed Martin (United States), Bethesda, United States, amanda.e.kraft@lmco.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Amanda E Kraft Matthias D Ziegler Sophia Mayne-DeLuca Trevor Sands Alison M Perez Jesse Mark Adrian Curtin Amanda Sargent Hasan Ayaz William Casebeer Google Amanda E Kraft Matthias D Ziegler Sophia Mayne-DeLuca Trevor Sands Alison M Perez Jesse Mark Adrian Curtin Amanda Sargent Hasan Ayaz William Casebeer Google Scholar Amanda E Kraft Matthias D Ziegler Sophia Mayne-DeLuca Trevor Sands Alison M Perez Jesse Mark Adrian Curtin Amanda Sargent Hasan Ayaz William Casebeer PubMed Amanda E Kraft Matthias D Ziegler Sophia Mayne-DeLuca Trevor Sands Alison M Perez Jesse Mark Adrian Curtin Amanda Sargent Hasan Ayaz William Casebeer Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Nationwide critical infrastructure monitoring using a common operating picture framework

This paper describes the efforts involved in designing a common operating picture system for monitoring large-scale critical infrastructures. The design leverages the Joint Directors of Laboratories (JDL) data fusion model to enable the integration of different critical infrastructure systems with their dependency relations. The resulting Situational Awareness of Critical Infrastructure and Networks (SACIN) framework offers a platform that provides a common operating picture of a critical infrastructure. A generic data collection component customized to each source system generates events and facilitates JDL level 0 integration. An analysis component collects events and data to produce meaningful information about the current state and future impact estimates in accordance with JDL levels 1 to 3. A brokered architecture supports level 4 control by various components and a JDL level 5 user interface is offered via a web application. Interviews of infrastructure subject matter experts were conducted to obtain the situational awareness requirements. By applying key situational awareness oriented design principles to the situational awareness requirements, a user interface was created for organizing information based on operator situational awareness needs and supporting key cognitive mechanisms that transform data into high levels of situational awareness. Situational awareness measures were used to assess operator performance during critical infrastructure tasks – a “freeze-probe” recall approach (Situational Awareness Global Assessment Technique (SAGAT)), a post-trial subjective rating approach (Situational Awareness Rating Technique (SART)) and the System Usability Scale (SUS). The results indicate that the supply of attentional resources (SART supply) and overall SAGAT score best predict the performance levels of operators.

Can teamwork and situational awareness (SA) in ED resuscitations be improved with a technological cognitive aid? Design and a pilot study of a team situation display

Effective teamwork in ED resuscitations, including information sharing and situational awareness, could be degraded. Technological cognitive aids can facilitate effective teamwork. Objective: This study focused on the design of an ED situation display and pilot test its influence on teamwork and situational awareness during simulated resuscitation scenarios. Material and methods: The display design consisted of a central area showing the critical dynamic parameters of the interventions with an events time-line below it. Static information was placed at the sides of the display. We pilot tested whether the situation display could lead to higher scores on the Clinical Teamwork Scale (CTS), improved scores on a context-specific Situational Awareness Global Assessment Technique (SAGAT) tool, and team communication patterns that reflect teamwork and situational awareness. Results: Resuscitation teamwork, as measured by the CTS, was overall better with the presence of the situation display as compared with no situation display. Team members discussed interventions more with the situation display compared with not having the situation display. Situational awareness was better with the situation display only in the trauma scenario. Discussion: The situation display could be more effective for certain ED team members and in certain cases. Conclusions: Overall, this pilot study implies that a situation display could facilitate better teamwork and team communication in the resuscitation event.

Evaluating Situation Awareness: An Integrative Review.

Situation awareness (SA) refers to the conscious awareness of the current situation in relation to one's environment. In nursing, loss or failure to achieve high levels of SA is linked with adverse patient outcomes. The purpose of this integrative review is to examine various instruments and techniques used to measure SA among nurses across academic and clinical settings. Computerized database and ancestry search strategies resulted in 40 empirical research reports. Of the reports included in the review, 24 measured SA among teams that included nurses and 16 measured SA solely in nurses. Methods used to evaluate SA included direct and indirect methods. Direct methods included the Situation Awareness Global Assessment Technique and questionnaires. Indirect methods included observer rating instruments and performance outcome measures. To have a better understanding of how nurses' make decisions in complex work environments, reliable and valid measures of SA is crucial.

Examining the Feasibility and Predictive Validity of the SAGAT Tool to Assess Situation Awareness Among Medical Trainees.

Situational awareness (SA) describes a team's ability to perceive environmental elements, comprehend their meaning, and anticipate future events. Although SA is consistently described as a critical competency among surgical teams, there is a dearth of research identifying efficacious methods to assess and develop SA in such settings. The aim of this study was to investigate the feasibility of implementing an objective tool that has been used to measure SA in other intense and dynamic environments -the Situation Awareness Global Assessment Technique (SAGAT)-and to examine its ability to predict surgical trainee team performance. Ten team-training sessions were conducted involving 2 standardized high-fidelity trauma simulation scenarios. Teams consisted of 4 or 5 participants, and roles were randomly assigned. Team situational awareness was assessed using the SAGAT method, which involves intermittent freezes to probe trainee awareness of the situation. Team performance was assessed using the Mayo High-Performance Teamwork Scale. Hierarchical regression was used to examine SA-performance relationships for each scenario. Forty-three third-year medical students participated in the training sessions. Team SA ranged from 45% to 79% and 46% to 97% for the first and second scenarios, respectively. Additionally, team SA significantly predicted team performance for both the first scenario (F(1, 42)=19.57; P<0.001; R=0.30) and second scenario (F(1,42)=26.18, P<0.001; R=0.38). The SAGAT is a valid, reliable tool for assessing surgical trainee SA. Information provided by the SAGAT can help diagnose team performance problems, inform debriefing discussion points, and inform curriculum development endeavors.

Explicit or implicit situation awareness? Measuring the situation awareness of train traffic controllers

Previous research on situation awareness (SA) predominantly focused on its explicit, reasoned, conscious features rather than on the implicit, intuitive, unconscious aspects that are often identified with expert operators. This research investigated implicit levels of SA of train traffic controllers (TTCs) in order to contribute to the body of knowledge on rail human factors research and SA. A novel approach was used to uncover levels of implicit SA through a set of three analyses: (1) fairly low SAGAT values with correlations between SAGAT scores and multiple performance indicators; (2) negative correlations between work experience and SAGAT scores; and (3) structurally lower level-1 SA (perception) scores in comparison to level-2 SA (comprehension) scores in accordance with Endsley’s three-level model. Two studies were conducted: A pilot study – which focused on SA measurements with TTCs in a monitoring mode (N=9) – and the main study, which involved TTCs from another control center (N=20) and three different disrupted conditions. In the pilot study, SA was measured through the situation-awareness global assessment technique (SAGAT), perceived SA and observed SA, and performance was measured through punctuality and unplanned stops of trains before red signals. In the main study, SA was measured through SAGAT, and perceived SA and multiple performance indicators, such as arrival and departure punctuality and platform consistency, were assessed. In both studies, the set of three analyses showed consistent and persistent indications of the presence of implicit SA. Endsley’s three-level model and related SAGAT method can be constrained by the presence of these intuitive, unconscious processes and inconsistent findings on correlations between SAGAT scores and performance. These findings provide insights into the SA of TTCs in the Netherlands and can support the development of training programs and/or the design of a new traffic management system.