Measures Of Situation Awareness Research Articles

Accelerating Understanding of Human Response to Automation Failure

Firstly, I comment on the lack of support for the predictions of the lumberjack model to professionally qualified operators in high-fidelity work simulations (Jamieson & Skraaning, 2020a). I highlight the advantages that Bayesian statistics provide for qualifying the degree of evidence for the null hypotheses, issues concerning situation awareness measurement, and the alternative techniques available to study experts. Secondly, I comment on the innovative taxonomy of automation failure presented by Skraaning and Jamieson (2024), pointing out some issues with overlapping definitions and lack of cause-effect relationships. I then discuss the substantial opportunity this taxonomy presents to guide future research, such as the design of transparent automation. To conclude, I identify some other key problems regarding how we currently study human-automation teaming (e.g. presenting randomized automation failure unlinked to task context) and invite discussion from the research community on the relevance of computational modelling to this field of research.

Recognizing the situation awareness of forklift operators based on EEG techniques in a field experiment.

Lack of situation awareness (SA) is the primary cause of human errors when operating forklifts, so determining the SA level of the forklift operator is crucial to the safety of forklift operations. An EEG recognition approach of forklift operator SA in actual settings was presented in order to address the issues with invasiveness, subjectivity, and intermittency of existing measuring methods. In this paper, we conducted a field experiment that mimicked a typical forklift operation scenario to verify the differences in EEG states of forklift operators with different SA levels and investigate the correlation of multi-band combination features of each brain region of forklift operators with SA. Based on the sensitive EEG combination indexes, Support Vector Mechanism was used to construct a forklift operator SA recognition model. The results revealed that there were differences between forklift operators with high and low SA in the θ, α, and β frequency bands in zones F, C, P, and O; combined EEG indicators θ/β, (α + θ)/(α + β), and θ/(α + β) in zones F, P, and C were significantly correlated with SA; the recognition accuracy of the model reached 88.64% in the case of combined EEG indicators of zones C & F & P as input. It could provide a reference for SA measurement, contributing to the improvement of SA.

User-centered design in mobile human-robot cooperation: consideration of usability and situation awareness in GUI design for mobile robots at assembly workplaces

Abstract Mobile cobots can increase the potential for assembly work in industry. For human-friendly automation of cooperative assembly work, user-centered interfaces are necessary. The design process regarding user interfaces for mobile human-robot cooperation (HRC) shows large research gaps. In this article an exemplary approach is shown to design a graphical user interface (GUI) for mobile HRC at assembly workplaces. The design is based on a wireframe developed to support situation awareness. An interactive mockup is designed and evaluated. This is done in two iterations. In the first iteration, a user analysis is carried out using a quantitative survey with n = 31 participants to identify preferred input modalities and a qualitative survey with n = 11 participants that addresses touch interfaces. The interactive mockup is developed by implementing design recommendations of the usability standards ISO 9241 – 110, 112 and 13. A heuristic evaluation is conducted with n = 5 usability experts and the measurement of situation awareness with n = 30 end users. In the second iteration, findings from the preceding iteration are implemented in the GUI and a usability test with n = 20 end users is conducted. The process demonstrates a combination of methods that leads to high usability and situation awareness in mobile HRC.

Scientific Measurement of Situation Awareness in Operational Testing

Scientific Measurement of Situation Awareness in Operational Testing

Identifying firefighters' situation awareness requirements for fire and non-fire emergencies using a goal-directed task analysis

A firefighter's situation awareness (SA) is considered crucial to making effective tactical decisions and actions at the scene. Despite the importance of the firefighter's SA, there have been limited research efforts to understand what cues and information firefighters use to assess ongoing situations and predict future conditions. In addition to fire events, contemporary firefighters respond to an increasing volume of non-fire incidents. Thus, this study aims to identify firefighters' SA during three fire incidents (single house fire, vehicle fire, and passenger aircraft fire) and three non-fire incidents (medical emergency, hazardous materials, and urban search and rescue). A goal-directed task analysis was conducted via focus group discussions with eight career firefighters. Findings indicate that firefighters build their SA by processing various cues from hazards (e.g., fire, ignition source), humans (e.g., occupants, bystanders, drivers, passengers), spatial elements (e.g., building structure, location of hazards), and surrounding conditions (e.g., traffic, weather). Our findings provide insights into SA measurement, SA-oriented work processes, training for SA, and designing technologies to support firefighters' SA during all-hazard responses.

The Effects of Auditory Working Memory Task on Situation Awareness in Complex Dynamic Environments: An Eye-movement Study.

This study investigated the effect of auditory working memory task on situation awareness (SA) and eye-movement patterns in complex dynamic environments. Many human errors in aviation are caused by a lack of SA, and distraction from auditory secondary tasks is a serious threat to SA. However, it remains unclear how auditory working memory tasks affect SA and eye-movement patterns. Participants (n = 28) were randomly allocated to two groups and received different periods of visual search training (short versus long). They subsequently completed a situation awareness measurement task in three auditory secondary task conditions (without secondary task, auditory calculation task, and auditory 2-back task). Eye-movement data were collected during the situation awareness measurement task. The auditory 2-back task significantly reduced overall SA, Level 1 SA, dwell times, and total percentage of fixation time on task-related areas of interests in the SA measurement task. Overall SA and Level 3 SA were not reduced by the auditory 2-back task in individuals in the longer visual search training time condition. Auditory working memory load impairs SA in the perception and projection stage; however, greater experience can overcome impairment of SA in the projection stage. This study provided possible approaches to preventing loss of SA: (1) improving crew members' communication skills to ensure the accurate and clear transmission of information, reducing the difficulty of processing information, and (2) providing targeted cognitive training tailored to each pilot's level of experience.

Digital technology enhanced situation awareness for construction safety: Systematic review and future research directions

The dynamics of construction processes pose significant challenges for workers to perceive, understand, and anticipate hazards on sites. Maintaining an adequate level of situation awareness (SA) is critical for safety. Recent years have seen increasing research interests in SA and digital technology (DT) in the construction safety field. However, there lacks a systematic review that elucidates the impacts of DT on workers’ cognitive processes involved in hazard identification. To fill the research gap, the paper aims to conceptualize the role of SA and DT in construction safety management and provide a theoretical basis for the future development of technology-enabled situation-aware construction safety management systems. This study employed a five-step systematic review approach with a scientometric analysis to provide an unbiased and comprehensive overview of research concerning SA in construction safety. The results suggested that current research on the cognitive processes related to hazard identification has been fragmented and has caused theoretical confusion. There is a need to develop a cognitive framework for hazard identification. Four specific research directions were recommended: (1) develop and validate a real-time SA measurement, (2) identify SA information requirements, (3) develop a cognitive model of hazard identification, and (4) develop a technology-enhanced SA system. By synthesizing the current literature, this review paper identified and analyzed the theoretical roles of SA and DT in worker hazard identification, shedding light on the potential for developing a technology-enabled situation-aware construction safety management system. It also offered a valuable perspective to review and re-design safety management practices.

A Comprehensive Situation Awareness Measurement Method for Analyzing the Operators’ Situation Awareness of Multi-Module High Temperature Gas-Cooled Reactor Plants

For multi-modular nuclear power plants under the scheme of multiple nuclear steam supply system (NSSS) modules driving a single steam turbine, the NSSS modules are coupled tightly with each other by the common turbine, giving more complex normal operation with respect to the single-modular plants. To limit the operation cost of multi-modular plants, one operator is assigned to monitor and control two or more modules, whose feasibility should be verified. Combined with the characteristics of multi-module control rooms and multi-module running tasks, this paper designs a comprehensive situation awareness measurement method that combines SART, NASA-TLX (NASA Task Load Index), and eye movement tracking methods. The SART, NASA-TLX, and gaze entropy are adopted to measure the operators’ SA, and a series of accident handling experiments are performed on a full-scale simulator to gain enough data for analysis. The operators’ eye trajectories on the human–machine interface (HMI) during the experiments are all recorded for calculating the gaze entropy. Both the SART and NASA-TLX scales are filled by the operators after finishing the experiments. The experiment results show that the difference in operators’ workload and SA amongst all the experimental scenarios is limited, even between the toughest and tenderest scenarios, indicating the feasibility of one operator driving two NSSS modules simultaneously.

The effects of motivation and noise on situation awareness: A study based on SAGAT and EEG

The current research results have differences in the correlation between situation awareness (SA) and physiological data, and there is a lack of systematic research on the influence of motivation and noise on SA. This study investigated the correlation between SA and electroencephalogram (EEG) signals under the effects of motivation and noise.The independent variables were motivation (two levels: positive and negative) and noise (three levels: no noise, natural noise, and speech noise). And the dependent variable were the levels of SA in perception, comprehension and projection. Participants were required to observe traffic videos of three types of noise under two types of motivation and complete the Situation Awareness Global Assessment Technique (SAGAT) questionnaire after each video. We recorded the participants' EEG signals and calculated the average relative power of the delta, theta, alpha, and beta bands of each brain region.The results showed that the levels of perception and projection under negative motivation were higher than those under positive motivation. The level of perception was negatively correlated with the delta band power in the central lobe and theta power in the occipital lobe. The level of projection was negatively correlated with the delta band power in the frontal, central, parietal, and occipital lobes, positively correlated with the theta band power, and negatively correlated with the delta band power in the temporal lobe. Therefore, negative motivation can improve the individual's perception and projection levels. The delta and theta band power may reliably measure SA.This study can help improve team member's SA by establishing motivational rules and better develop physiological measures of SA.

Evaluating Flight Performance and Eye Movement Patterns Using Virtual Reality Flight Simulator

Efficient and economical performance evaluation of pilots has become critical to the aviation industry. With the development of virtual reality (VR) and the combination of eye-tracking technology, solutions to meet these needs are becoming a reality. Previous studies have explored VR-based flight simulators, focusing mainly on technology validation and flight training. The current study developed a new VR flight simulator to evaluate pilots' flight performance based on eye movement and flight indicators in a 3D immersive scene. During the experiment, 46 participants were recruited: 23 professional pilots and 23 college students without flight experience. The experiment results showed significant differences in flight performance between participants with and without flight experience, the former being higher than the latter. In contrast, those with flight experience showed more structured and efficient eye-movement patterns. These results of the differentiation of flight performance demonstrate the validity of the current VR flight simulator as a flight performance assessment method. The different eye-movement patterns with flight experience provide the basis for future flight selection. However, this VR-based flight simulator has shortcomings like motion feedback compared to traditional flight simulators. This flight simulator platform is highly flexible except for the apparent low cost. It can meet the diverse needs of researchers (e.g., measuring situation awareness, VR sickness, and workload by adding relevant scales).

Impact of a Simulation-Based Patient Safety Intervention on Self-Reported and Objective Measures of Situational Awareness

Impact of a Simulation-Based Patient Safety Intervention on Self-Reported and Objective Measures of Situational Awareness

Measuring situation awareness in health care providers: a systematic review of measurement properties using COSMIN methodology

BackgroundSituation awareness (SA) is a key factor in accountability and ensuring patient safety in health care. SA is an essential element to research on human factors in healthcare. It is essential to identify valid instruments for measuring this concept and assessing how it is affected by interventions and educational methods.MethodsThis systematic review aimed to assess the measurement properties of situation awareness instruments in health care providers’ (HCPS) using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. Four databases (Medline (through PubMed), Embase, Scopus, and Web of Science) were systematically searched. A manual search was also conducted on Google Scholar and the reference list of the included primary studies to supplement the electronic search. Studies aiming to determine the measurement properties of SA instruments or non-technical skills in HCPS were included. The overall results for each measurement property were reported as sufficient, insufficient, inconsistent, or indeterminate, and the quality of evidence was reported as high, moderate, low, or very low.ResultsA total of 25 studies and 15 instruments were included in the study. More than one measurement property was reported in some of the studies, and none of the studies presented all measurement properties. The most common measurement properties were content validity (12/25) and internal consistency (12/25). Cross‐cultural validity and responsiveness were not investigated in any study. Evidence quality for the measurement properties was not high in any of the 15 instruments.ConclusionsNone of the instruments can be recommended as the most suitable instrument, and all instruments were classified as promising instruments in need of further psychometric assessment. This systematic review proves the dire need for the development and psychometric evaluation of instruments to measure SA in HCPs in clinical settings.Systematic review registrationPROSPERO CRD42020147349.

Understanding situation awareness in SOCs, a systematic literature review

Situation awareness is shown through human factors research to be a valuable construct to understand and improve how humans perform while operating complex systems in critical environments. Within cyber security one such environment is the Security Operations Center (SOC). With the increasing threat of hybrid warfare, knowledge about situation awareness within SOC environments, where human error or low performance may be detrimental, must be developed. This paper reports on the results of a Systematic Descriptive Literature Review of the current research on situation awareness within SOCs. The goal of the paper is to analyze how situation awareness is understood in the current research. To achieve this goal three aspects of understanding were addressed: Theoretical foundations; levels of conceptualization; and measurement of situation awareness. Theoretical foundations in the literature were assessed by how situation awareness was defined and the presence of references to theoretical models of SA. The results show a clear trend of basing the research on Endsley's three level situation awareness model; this model has been developed into a domain specific formulation called “Cyber Situation Awareness”. Some parts of the literature, particularly in research aimed at developing tools for improving situation awareness, lack a theoretical foundation; some refer to alternative theoretical foundations of situation awareness like Stanton et al.’s Distributed Situation Awareness. Further, a balance between conceptualizations on the individual, group and system level has been identified. Within research aimed at developing tools for improving situation awareness there are some examples of specialized and precise measurements of situation awareness, but in general the research seems too reliant on indirect measures of situation awareness. The paper concludes with the proposition of connecting the systems-based theoretical perspective of distributed situation awareness into the research, utilizing a systems level conceptualization of situation awareness. This might prove to be a useful bridge between the human cognitive perspective of situation awareness and the development of the complex technical environment of critical importance that SOCs represent.

The Level of Measurement of Subjective Situation Awareness and Its Dimensions in the Situation Awareness Rating Technique (SART)

Situation awareness (SA), a measure of how well a person understands the situation, is frequently used to evaluate the safety and effectiveness of critical systems that depend on human behavior. While there are objective ways of measuring SA, subjective assessments, such as the SA rating technique (SART), are still widely used. However, it is not clear what the level of measurement is for SART-measured SA or its constituent dimensions This is a significant gap because the level of measurement determines what mathematics and statistics can be meaningfully used to synthesize and evaluate measures. This research uses a previously developed method for determining the level of measurement of psychometric ratings to evaluate the level of measurement of SART and its elements. Results show that all of the dimensions of SA can be treated as interval in most situations, but that each is on a separate interval scale. This result casts doubt on the validity of the formula SART uses to compute SA from its subcomponents. We ultimately discuss our results and explore future research directions.

Using eye movements to evaluate the effectiveness of the situation awareness rating technique scale in measuring situation awareness for smart manufacturing

Physiological indicators, including eye-tracking measures, may provide insight into human internal states in many domains, such as smart manufacturing. The Situation Awareness Rating Technique (SART) scale has been criticised for not assessing situation awareness (SA) accurately. In this study, we investigated the precision of the SART scale for assessing SA by comparing the scores to eye movement data. Thirty participants were recruited to complete a process plant monitoring task. Participants’ eye movements and SART scores were recorded and analysed. Our results moderately supported the idea that the SART scale did not accurately measure SA. We found that four dimensions in the SART scale need to be revised to reflect real SA, which may partially solve the divergence between objective and subjective SA measurements. Moreover, these findings provided solutions for designing a revised SART scale to measure the internal states of operators for smart manufacturing. Practitioner summary: Situation awareness (SA) is a critical component of decision-making and performance for smart manufacturing. The present study examines the relationships among eye movement, the SART scale, and SA for smart manufacturing in a refinery control room.

Team Situation Awareness, Cohesion, and Autonomic Synchrony.

This study evaluated the causal relationships among situation awareness (SA), cohesion, and autonomic synchrony (SE) within teams. SA is often a team effort and should be more accurate in better-functioning teams. Cohesive teams perform better overall, although the relationship appears reciprocal; the relationship to SA has not been considered previously. SE is a collective neurocognitive activity that has been connected to team coordination, communication, and performance in some circumstances. In this experiment, 71 undergraduates, organized into 16 teams, played two matches of a first-person shooter computer game and completed self-report measures of cohesion and SA. SE was determined through time series analysis of electrodermal responses using the driver-empath framework. Empaths and those who came from more synchronized teams reported less cohesion in the team. Granger causality regression showed reciprocal relations among SA, SE, and cohesion that were both positive and negative after controlling for match difficulty. The cohesion-SA relationship is similar to the reciprocal cohesion-performance relationship. SE plays an important and independent role in both the social and cognitive aspects of team behavior. It is possible, furthermore, that individuals who are more attuned to their co-workers reported a more accurate, and less obliging, social situation. Results are applicable to situations requiring teamwork in a dynamic environment.

Measurement of situation awareness in engine control room: approach for non-technical skill assessment in engine resource management

Measurement of situation awareness in engine control room: approach for non-technical skill assessment in engine resource management

Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving

Situation awareness (SA) is critical to improving takeover performance during the transition period from automated driving to manual driving. Although many studies measured SA during or after the driving task, few studies have attempted to predict SA in real time in automated driving. In this work, we propose to predict SA during the takeover transition period in conditionally automated driving using eye-tracking and self-reported data. First, a tree ensemble machine learning model, named LightGBM (Light Gradient Boosting Machine), was used to predict SA. Second, in order to understand what factors influenced SA and how, SHAP (SHapley Additive exPlanations) values of individual predictor variables in the LightGBM model were calculated. These SHAP values explained the prediction model by identifying the most important factors and their effects on SA, which further improved the model performance of LightGBM through feature selection. We standardized SA between 0 and 1 by aggregating three performance measures (i.e., placement, distance, and speed estimation of vehicles with regard to the ego-vehicle) of SA in recreating simulated driving scenarios, after 33 participants viewed 32 videos with six lengths between 1 and 20 s. Using only eye-tracking data, our proposed model outperformed other selected machine learning models, having a root-mean-squared error (RMSE) of 0.121, a mean absolute error (MAE) of 0.096, and a 0.719 correlation coefficient between the predicted SA and the ground truth. The code is available at <uri>https://github.com/refengchou/Situation-awareness-prediction</uri>. Our proposed model provided important implications on how to monitor and predict SA in real time in automated driving using eye-tracking data.

AAIM Recommendations to Improve Learner Transitions

xxx.

Triggering Alerts with an Eye-Tracking-Based Measurement of Situation Awareness

Triggering Alerts with an Eye-Tracking-Based Measurement of Situation Awareness