Predicting Situation Awareness Research Articles

An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems.

This study developed a fixation-related electroencephalography band power (FRBP) approach for situation awareness (SA) assessment in automated driving. Maintaining good SA in Level 3 automated vehicles is crucial to drivers' takeover performance when the automated system fails. A multimodal fusion approach that enables the analysis of the visual behavioral and cognitive processes of SA can facilitate real-time assessment of SA in future driver state monitoring systems. Thirty participants performed three simulated automated driving tasks. After each task, the Situation Awareness Global Assessment Technique (SAGAT) was deployed to capture their SA about key elements that could affect their takeover task performance. Participants eye movements and brain activities were recorded. Data on their brain activity after each eye fixation on the key elements were extracted and labeled according to the correctness of the SAGAT. Mixed-effects models were used to identify brain regions that were indicative of SA, and machine learning models for SA assessment were developed based on the identified brain regions. Participants' alpha and theta oscillation at frontal and temporal areas are indicative of SA. In addition, the FRBP technique can be used to predict drivers' SA with an accuracy of 88% using a neural network model. The FRBP technique, which incorporates eye movements and brain activities, can provide more comprehensive evaluation of SA. Findings highlight the potential of utilizing FRBP to monitor drivers' SA in real-time. The proposed framework can be expanded and applied to driver state monitoring systems to measure human SA in real-world driving.

Smart flood resilience: harnessing community-scale big data for predictive flood risk monitoring, rapid impact assessment, and situational awareness

Smart resilience is the beneficial result of the collision course of the fields of data science and urban resilience to flooding. The objective of this study is to propose and demonstrate a smart flood resilience framework that leverages heterogeneous community-scale big data and infrastructure sensor data to enhance predictive risk monitoring and situational awareness. The smart flood resilience framework focuses on four core capabilities that could be augmented by the use of heterogeneous community-scale big data and analytics techniques: (1) predictive flood risk mapping; (2) automated rapid impact assessment; (3) predictive infrastructure failure prediction and monitoring; and (4) smart situational awareness capabilities. We demonstrate the components of these core capabilities of the smart flood resilience framework in the context of the 2017 Hurricane Harvey in Harris County, Texas. First, we present the use of flood sensors for the prediction of floodwater overflow in channel networks and inundation of co-located road networks. Second, we discuss the use of social media and machine learning techniques for assessing the impacts of floods on communities and sensing emotion signals to examine societal impacts. Third, we describe the use of high-resolution traffic data in network-theoretic models for nowcasting of flood propagation on road networks and the disrupted access to critical facilities, such as hospitals. Fourth, we introduce how location-based and credit card transaction data were used in spatial analyses to proactively evaluate the recovery of communities and the impacts of floods on businesses. These analyses show that the significance of core capabilities of the smart flood resilience framework in helping emergency managers, city planners, public officials, responders, and volunteers to better cope with the impacts of catastrophic flooding events.

Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving

Situation awareness (SA) is critical to improving takeover performance during the transition period from automated driving to manual driving. Although many studies measured SA during or after the driving task, few studies have attempted to predict SA in real time in automated driving. In this work, we propose to predict SA during the takeover transition period in conditionally automated driving using eye-tracking and self-reported data. First, a tree ensemble machine learning model, named LightGBM (Light Gradient Boosting Machine), was used to predict SA. Second, in order to understand what factors influenced SA and how, SHAP (SHapley Additive exPlanations) values of individual predictor variables in the LightGBM model were calculated. These SHAP values explained the prediction model by identifying the most important factors and their effects on SA, which further improved the model performance of LightGBM through feature selection. We standardized SA between 0 and 1 by aggregating three performance measures (i.e., placement, distance, and speed estimation of vehicles with regard to the ego-vehicle) of SA in recreating simulated driving scenarios, after 33 participants viewed 32 videos with six lengths between 1 and 20 s. Using only eye-tracking data, our proposed model outperformed other selected machine learning models, having a root-mean-squared error (RMSE) of 0.121, a mean absolute error (MAE) of 0.096, and a 0.719 correlation coefficient between the predicted SA and the ground truth. The code is available at <uri>https://github.com/refengchou/Situation-awareness-prediction</uri>. Our proposed model provided important implications on how to monitor and predict SA in real time in automated driving using eye-tracking data.

A new methodology for assessing human contributions to occurrences (MAHCO) in Air Traffic Management utilising a Bayesian hierarchical predictive coding approach to the brain, and the benefits for just culture

The traditional approaches to classifying Human Error have provided a strong and beneficial approach to understanding the roles of humans in safety occurrences. However, as the demand for human performance has increased and with the rise of more automation, the line between man and machine continues to become increasingly blurred. Traditional Human Error analysis techniques are no longer sufficient for extracting the most valuable learning data from occurrences. Furthermore, there is a continuing need to strengthen Safety and Just culture. One means to achieving this is through developing ever greater trust in the ability of organisations to accurately investigate and report occurrences. This paper outlines a new 10 step approach to investigating human contribution to occurrences which is based on a version of predictive situational awareness in the brain that is derived from hierarchical predictive coding approaches. Utilising statistically based induction, the steps allow investigators to make an objective reconstruction of sub-conscious human error analysis and integrate it with traditional subjective feedback on conscious decisions made. Examples of system data that can provide further evidence to support the reconstructions is also discussed. This approach has been tested on more than 50 human factors related occurrences in Air Traffic Control in 2021/2022 and the results are introduced. It is further argued that using a refined approach and educating operators on the approach can significantly benefit Just Culture and lead to increases in trust and frequency of reporting because it will allow for the removal of biases by individual investigators. Future research should focus on validating the approach against current methods and testing it for occurrences further away from front line operators such as in non-safety related decision making.

Using Eye-tracking Data to Predict Situation Awareness in Real Time during Takeover Transitions in Conditionally Automated Driving

Situation awareness (SA) is critical to improving takeover performance during the transition period from automated driving to manual driving. Although many studies measured SA during or after the driving task, few studies have attempted to predict SA in real time in automated driving. In this work, we propose to predict SA during the takeover transition period in conditionally automated driving using eye-tracking and self-reported data. First, a tree ensemble machine learning model, named LightGBM (Light Gradient Boosting Machine), was used to predict SA. Second, in order to understand what factors influenced SA and how, SHAP (SHapley Additive exPlanations) values of individual predictor variables in the LightGBM model were calculated. These SHAP values explained the prediction model by identifying the most important factors and their effects on SA, which further improved the model performance of LightGBM through feature selection. We standardized SA between 0 and 1 by aggregating three performance measures (i.e., placement, distance, and speed estimation of vehicles with regard to the ego-vehicle) of SA in recreating simulated driving scenarios, after 33 participants viewed 32 videos with six lengths between 1 and 20 s. Using only eye-tracking data, our proposed model outperformed other selected machine learning models, having a root-mean-squared error (RMSE) of 0.121, a mean absolute error (MAE) of 0.096, and a 0.719 correlation coefficient between the predicted SA and the ground truth. The code is available at https://github.com/refengchou/Situation-awareness-prediction. Our proposed model provided important implications on how to monitor and predict SA in real time in automated driving using eye-tracking data.

A hybrid deep learning model for predictive flood warning and situation awareness using channel network sensors data

AbstractThe objective of this study is to create and test a hybrid deep learning (DL) model, FastGRNN‐FCN (fast, accurate, stable and tiny gated recurrent neural network‐fully convolutional network), for urban flood prediction and situation awareness using channel network sensors data. The study used Harris County, Texas, as the testbed, and obtained channel sensor data from three historical flood events (e.g., 2016 Tax Day Flood, 2016 Memorial Day Flood, and 2017 Hurricane Harvey Flood) for training and validating the hybrid DL model. The flood data are divided into a multivariate time series and used as the model input. Each input comprises nine variables, including information of the studied channel sensor and its predecessor and successor sensors in the channel network. Precision‐recall curve and F‐measure are used to identify the optimal set of model parameters. The optimal model with a weight of 1 and a critical threshold of 0.59 are obtained through 100 iterations based on examining different weights and thresholds. The test accuracy and F‐measure eventually reach 97.8% and 0.792, respectively. The model is then tested in predicting the 2019 Imelda Flood in Houston and the results show an excellent match with the empirical flood. The results show that the model enables accurate prediction of the spatial–temporal flood propagation and recession and provides emergency response officials with a predictive flood warning tool for prioritizing the flood response and resource allocation strategies.

Predictive Cyber Situational Awareness and Personalized Blacklisting

Cybersecurity adopts data mining for its ability to extract concealed and indistinct patterns in the data, such as for the needs of alert correlation. Inferring common attack patterns and rules from the alerts helps in understanding the threat landscape for the defenders and allows for the realization of cyber situational awareness, including the projection of ongoing attacks. In this article, we explore the use of data mining, namely sequential rule mining, in the analysis of intrusion detection alerts. We employed a dataset of 12 million alerts from 34 intrusion detection systems in 3 organizations gathered in an alert sharing platform, and processed it using our analytical framework. We execute the mining of sequential rules that we use to predict security events, which we utilize to create a predictive blacklist. Thus, the recipients of the data from the sharing platform will receive only a small number of alerts of events that are likely to occur instead of a large number of alerts of past events. The predictive blacklist has the size of only 3% of the raw data, and more than 60% of its entries are shown to be successful in performing accurate predictions in operational, real-world settings.

A New Model for Predicting Infectious Disease Outbreaks

This project integrated tools and hybrid methodologies historically used for early warning, intelligence, counter space, public health, informatics, and medical surveillance applications. A multidiscipline team assembled and explored non-medical prediction and analytical techniques that successfully predict critical events for low probability but high-regret national and global scenarios. The team then created novel approaches needed to fill nuanced and unique gaps for the infectious disease prediction challenge. The team adopted and applied those proven procedures to determine which would be efficacious in foretelling infectious disease outbreaks around the world. One outcome of that effort was a successful two-year development and validation project designated ‘RAID’ (Risk Awareness Framework for Infectious Diseases), which focused on malaria prediction. The project’s objective was to maximize the warning (prediction) window of impending malaria epidemic outbreaks with sufficient time to allow meaningful preventive intervention before widespread human infection. It is generally recognized the more protracted the prediction window extends before an event, the more time available for health authorities to muster and deploy resources, which lessen morbidity, mortality, and harmful economic effects. Also, the value of early warning for an imminent epidemic must have mitigation options, or the warning window would have no beneficial impact on health outcomes. Finally, early notice is preferable over surprise epidemics, as unexpected waves of patients seeking acute care can easily overwhelm most local medical systems, as history repeatedly teaches. This cliché keeps repeating, with recurring Ebola epidemics and the recent COVID-19 pandemic as prominent exemplars. Predictive lead times need to be adequate for an intervention to be relevant. RAID’s focus on malaria prediction met these criteria from a relevant clinical and humanitarian perspective. Subsequent papers will address successful external generalization of these methods in predicting other similar infectious diseases. The model presented in this manuscript supports the conclusion that an additional two weeks advance notice could be available to public health authorities utilizing these techniques. This foreknowledge would allow the deployment of limited health resources into areas where they would do the most good and just in time. The geographical specificity was examined down to 5 km x 5 km grid squares overlaid anywhere in the world. Most of the model’s input data were derived from remote sensing satellite sources that could combine with historical WHO (World Health Organization) or nation-reported existential pathogen loads to improve model accuracy; however, such data harmonization is not required. If ground sensors were integrated into the modeling, the confidence of the risk of infection would logically improve. The model provides a successful global risk assessment via commercially available remote space sensors, even without ground sensing. RAID provides a necessary and useful preliminary means to predictive situational awareness. This improved predictive awareness is sufficiently granular to identify last chance windows for public health interventions globally. This need will become even more pronounced as infectious diseases evolve biologically and migrate geographically at ever-increasing rates.

Effects of working memory, attention, and expertise on pilots’ situation awareness

The current study investigates individual differences that predict situation awareness (SA) in professional pilots. The aim of the study is twofold: to examine the roles of divided attention, inhibition, working memory, and expertise in predicting SA, and to demonstrate the relative contributions of these individual differences to online (Situation Awareness Present Method, SPAM) and offline (Situation Awareness Global Assessment Technique, SAGAT) SA measures. Thirty-six professional pilots completed a challenging flight scenario in a full-flight simulator. Divided attention, inhibition, working memory span, and expertise were measured using choice reaction time with dichotic listening, Stroop, and Automated Operation Span tasks, and flight hours in a full-flight simulator, respectively. Results indicated that offline and online SA measure were not correlated, supporting their concurrent use to obtain a comprehensive measure of SA. Offline SA scores were best predicted by working memory and level of expertise, while online SA scores were predicted by expertise, divided attention and inhibition. Results are discussed focusing on both theoretical contributions for defining and measuring SA and applications. Findings have implications for operators of critical domains and their interactions with automated systems, in which SA is crucial for performance and safety.

Bayesian network for the prediction of situation awareness errors

A new method is proposed to predict situation awareness errors in training simulations. It is based on Endsley's model and the eight 'situation awareness demons' that she described. The predictions are determined thanks to a Bayesian network and noisy-or nodes. A maturity model is introduced to come up with the initialisation problem. The NASA behavioural competency model is also used to take individual differences into account.

A Predictive Model of Situation Awareness with ACT-R

Objective: The aim of this study is to model all levels of situation awareness (SA), which would be able to predict situation awareness quantitatively.BR Background: When measuring situation awareness, directly measuring SA methods such as SAGAT and SART have been utilized. Several approaches (cognitive modeling approaches) were introduced to model SA but level 3 SA was not completed. For real-life situation, however, it is necessary to detect the problematic level of SA rather than overall SA. Therefore, we proposed a new model of all levels of SA in this study.BR Method: In order to model all levels of SA, this study chose factors in ACT-R architecture through literature review. ATC (Air Traffic Control)-related simulation task was video-taped to analyze human behaviors in order to model all levels of SA including level 3.BR Results: As a result, regression analyses show that cognitive activities (neural activations) represented for all levels of SA were highly correlated with SAGAT.BR Conclusion: In conclusion, neural activations in ACT-R could be proved to be effective to model all levels of SA.BR Application: Our SA model could be used to predict all levels of SA quantitatively without directly measuring the SA of operators.

Cyber-physical security metric inference in smart grid critical infrastructures based on system administrators' responsive behavior

To protect complex power-grid control networks, efficient security assessment techniques are required. However, efficiently making sure that calculated security measures match the expert knowledge is a challenging endeavor. In this paper, we present EliMet, a framework that combines information from different sources and estimates the extent to which a control network meets its security objective. Initially, EliMet passively observes system operators' online reactive behavior against security incidents, and accordingly refines the calculated security measure values. To make the values comply with the expert knowledge, EliMet actively queries operators regarding those states for which sufficient information was not gained during the passive observation. Finally, EliMet makes use of the estimated security measure values for predictive situational awareness by ranking potential cyber-physical contingencies that the security administrators should plan for upfront. Our experimental results show that EliMet can optimally make use of prior knowledge as well as automated inference techniques to minimize human involvement and efficiently deduce the expert knowledge regarding individual states of that particular system.

Knowing What You Know: The Role of Meta-Situation Awareness in Predicting Situation Awareness

To help operators handle increasing task demands, automated systems are increasingly being used in modern work settings. However, a potential consequence of such systems is the out of the loop performance syndrome, which refers to the reduced ability of operators working with high levels of automation support to take over manual control in the event of an automation failure (e.g., Wiener & Curry, 1980). One of the main factors considered responsible for this is a loss of operator situation awareness (SA; Endsley & Kiris, 1995). In order to improve SA and consequently improve performance, it becomes important to better understand the process of SA and the underlying factors that predict SA (e.g., Durso & Sethumadhavan, 2008). The role of meta-SA in predicting SA when individuals control air traffic with automation support is investigated in this paper. Results have practical implications for the development of SA training programs.

Predicting Situation Awareness from Team Communications

Given the importance of Situation Awareness (SA) in military operations, there is a critical need for a realtime, unobtrusive tool that objectively and reliably measures warfighters' SA in both training and operations. Just as the requirement for improved access to SA measures has become vital, it is now commonplace for military team communications to be mediated by technology, hence easily captured and available for analysis. We believe that team communications can be used to derive SA measures. To address this issue, we are developing the Automated Communications Analysis of Situation Awareness (ACASA) system. ACASA combines the explanatory capacity of the SA construct with the predictive and computational power of TeamPrints, to assess team and shared SA as well as other cognitive processes. TeamPrints is a system that combines computational linguistics and machine learning techniques coupled with Latent Semantic Analysis (LSA) to analyze team communication. In this paper, we present the findings from an exploratory evaluation of how well TeamPrints predicts SA from the team communications arising during a military training exercise.

An Evaluation of Task Network Modeling for Use in Simulating Situation Awareness

To achieve its mission of regulating the commercial nuclear power industry, the U.S. Nuclear Regulatory Commission (NRC) periodically commissions research studies to investigate potential safety issues. These include operational studies involving operator and crew performance. A main objective of these research projects is to provide information and knowledge that can be used in the development of regulation, regulatory guidance, and improved methods for collecting and analyzing personnel performance data to assist in regulatory inspections, investigations, and evaluations. The Halden Man-Machine Laboratory (HAMMLAB) in Halden, Norway recently conducted a study using professional nuclear power plant operators to investigate the effect of different alarm systems and display types on operator performance. Dependent variables in the Halden Alarm study included a mix of plant performance measures, operator task performance measures, operator cognitive processes, and subjective evaluations. Operator cognitive process measures included measures of Situation Awareness (SA) using an adaptation of the Situation Awareness Global Assessment Technique (SAGAT). Through the NRC, Micro Analysis and Design was allowed access to the work being performed at Halden in order to evaluate the use of Task Network Modeling (TNM) to simulate human behavior in a nuclear power plant. The alarm system, a turbine trip scenario and data from Halden were used as the basis for the model developed in this project. For part of this project, we also developed a model to simulate SA that was representative of how it was measured in the Halden Alarm study. This model was then integrated into our model that represented a disturbance scenario at a nuclear power plant. Data generated by the task network simulation was then compared to data collected from the nuclear power plant simulator at the HAMMLAB. Statistical tests showed no significant differences between the SA scores of actual operators and those SA scores predicted by the model. Thus, results indicate that a simple representation of SA within TNM can be used to accurately predict situation awareness given this measuring technique. Implications of this finding and recommendations for improving the model are discussed.