Level Of Situation Awareness Research Articles

Towards enhancing major emergency initial response training: a comparative study of STCW and OPITO standards

The maritime industry has been characterized as a high-risk domain owing to various risks associated with its complicated operations. This aspect imposes significant importance on the human element role, especially during unanticipated events. When major emergencies occur on ships, the response of the ship’s Master and his supporting team becomes crucial in controlling the situation and preventing the escalation of threat. In recognition of this fact, the IMO Standards of Training, Certification and Watchkeeping (STCW) 95/2010 convention has developed a range of emergency response competencies. However, data on accidents and incidents in shipping reveal that the human element is still considered the primary factor contributing to most cases of ship loss. Furthermore, several cases have been reasoned for improper and ineffective management of the emergency. This might raise serious concerns regarding the adequacy of IMO model courses and whether they provide learners with the required level of knowledge, skills, situational awareness, and confidence to appropriately respond should a real major emergency occurs onboard. With the aim of enhancing the current level of maritime emergency management training, the authors of this study intended to identify possible gaps in current STCW training criteria by conducting a comparison with similar emergency response training standards established by the Offshore Petroleum Industry Training Organization (OPITO). The Qualitative analysis was used to compare three case studies of emergency-related STCW courses with two OPITO courses that have similar objectives, in terms of their aims, scopes, principles, outlines, number of delegates, outcomes, decision-making processes and assessment criteria, in order to assess whether the OPITO emergency response training provides more extensive scope for ensuring the optimum management of an extended range of emergency events rather than the currently delivered STCW courses. The study is concluded by identifying possible gaps and proposing specific modifications to the STCW standard which can be marked as a first step in the direction of establishing effective training for managing and controlling major emergencies onboard ships.

The Influence of Presentation Frames of Visualization Information for Safety on Situational Awareness under a Three-Level User-Interface Design.

To explore the influence of the construction and presentation frames of visualization information for safety (VIS) on people's situation awareness (SA), we designed a three-level user interface (UI) of VIS based on the three-stage SA theory, including perception (SA1), comprehension (SA2), and projection (SA3). Then, 166 subjects were recruited and divided into three groups to participate in the experiment, in which SA was measured by the situation-present-assessment method (SPAM) and situation-awareness-rating technique (SART), and eye-movement data were recorded. The results show that the level-3 UI design could effectively improve the subjects' SA levels. Although the increase in VIS displayed caused by the higher UI level led to a decrease in the perception-stage score of SA, the level-3 UI fully considered the three stages of human information processing, and helped improve the SA of the subjects; the overall SA score measured using the SART method was not significant, but the result was consistent with the SPAM. There was a framing effect on the presentation of VIS, and subjects perceived different degrees of risk under different presentation frames; that is, less risk under the positive frame, more risk under the negative frame, and a higher level of SA under the positive frame compared with the negative frame. To some extent, the nearest-neighbor-index (NNI) algorithm could be utilized to quantify subjects' eye-tracking fixation mode. While the frames were guided by the high-level interface and the positive presentation frame, the distribution of the subjects' gaze points was more discrete; they could grasp the relevant information more comprehensively and had a relatively high level of SA. To some extent, this study can provide a reference for the design and optimization of the VIS presentation interface.

Enhanced Situational Awareness for DER-Rich Distribution Systems Under Pre- and Post-Outage

Situational awareness (SA) is critical to properly operating active power distribution systems during normal and outage conditions. Appropriate SA tools should provide an accurate estimate of system voltage and current variables, the operational network topology, and the power injections from distributed energy sources (DERs), including behind-the-meter (BTM) photovoltaics (PVs). Obtaining an accurate SA, especially by estimating network topology and gross load demand, is increasingly challenging with the proliferation of BTM DERs with intermittent power generation. Moreover, the SA required for distribution system restoration is even more challenging after a medium to a prolonged outage due to inaccurate estimates of cold load pickup (CLPU) and switch statuses. This paper proposes an integrated real-time model update (RTMU) module for SA enhancement to help distribution system operators (DSOs) understand the power system conditions in dynamic and DER-rich environments. The proposed RTMU consists of several modules to obtain the required level of SA for operational decision-making. It includes estimators for a) BTM PV power, b) network topology, and c) CLPU. The proposed approaches leverage multiple data resources, deep learning approaches, and domain knowledge of the power system to provide the required level of SA to DSOs. The dependencies among these modules are actively leveraged to enhance SA under normal conditions and during power outages. We demonstrate and analyze the performance of the proposed RTMU on a modified IEEE 123-node feeder and a utility distribution system from the Western United States.

Long-term prediction of safety parameters with uncertainty estimation in emergency situations at nuclear power plants

The correct situation awareness (SA) of operators is important for managing nuclear power plants (NPPs), particularly in accident-related situations. Among the three levels of SA suggested by Ensley, Level 3 SA (i.e., projection of the future status of the situation) is challenging because of the complexity of NPPs as well as the uncertainty of accidents. Hence, several prediction methods using artificial intelligence techniques have been proposed to assist operators in accident prediction. However, these methods only predict short-term plant status (e.g., the status after a few minutes) and do not provide information regarding the uncertainty associated with the prediction. This paper proposes an algorithm that can predict the multivariate and long-term behavior of plant parameters for 2 h with 120 steps and provide the uncertainty of the prediction. The algorithm applies bidirectional long short-term memory and an attention mechanism, which enable the algorithm to predict the precise long-term trends of the parameters with high prediction accuracy. A conditional variational autoencoder was used to provide uncertainty information about the network prediction. The algorithm was trained, optimized, and validated using a compact nuclear simulator for a Westinghouse 900 MWe NPP.

Analysis of Physical Workload and Situational Awareness at the Apron Movement Control (AMC) Unit at Husein Sastranegara International Airport Bandung

Efforts to develop human resources in line with the company's vision and mission are not an easy job, companies are required to be able to prepare a workforce that is able to carry out tasks in work effectively and efficiently. Obstacles that often occur in a company in efforts to develop human resources are high physical workload or workload and lack of situational awareness. This research uses a type of research with a qualitative descriptive method. To be able to get answers from the research, the researcher made a research design design. Qualitative research is research that is used to investigate, find, describe, and explain the quality of influences that cannot be explained, measured or described through a quantitative approach. The results of the study show that the physical workload of the AMC unit at Husein Sastranegara Airport in Bandung is very high due to the very wide coverage of the supervisory area, as well as the great responsibility in supervising and controlling all movements on the air side, with a large number of personnel. experienced a shortage of officers/executors as field workers, where the AMC unit only had 2 officers/executors, therefore with this great responsibility the workload on the AMC unit was also very high. Regarding the condition and form, the level of Situational awareness possessed by AMC officers is already at level 3, because officers can function themselves in the right conditions and at the right time in the surrounding environment in an effective way

Opportunities and Directions for the Evolution of Command and Control Systems in the Context of Multi-domain Operations

The military conflicts of recent times have highlighted the need for modern military operations to focus on multi-domain operations, which requires a transformation of the command and control system, by creating mobile and flexible headquarters that can provide relevant information at any time, even during redeployments and redeployments. Cloud computing can be used to integrate data collection tools, ensuring fast and accurate analysis of large amounts of incoming data. A private 5G network can provide a secure, high-speed and reliable communication environment for cloud-based command and control systems. This network will enable real-time information transfer, facilitate rapid decision-making processes and enable the deployment of autonomous vehicles and drones. The authors have also explored capabilities that support the deployment of autonomous vehicles and drones, enabling commanders to conduct intelligence and surveillance more effectively. By leveraging technology and advanced communications systems, commanders can lead their teams in a dynamic environment while providing a high level of situational awareness. This integration of information and mobility allows commanders to react quickly and decisively, ultimately increasing their effectiveness on the battlefield.

Formalizing Distributed Situation Awareness in Multi-Agent Networks

The version of distributed situation awareness (DSA) used in this article originated in human factors/ergonomics. Typically, this has involved the study of human operators working in teams and has used various forms of concept maps to qualitatively describe the information that team members use. In this article, we apply the concept of DSA to multi-agent teams (where team members might be human or automation) and extend the concept using the formal properties of Bayesian belief networks. In particular, we show how the Bayesian belief network can define DSA and how such a network can (using expectation maximization) adapt, even on the basis of limited information. The approach was considered in terms of situation awareness levels of perception (i.e., sensor state values conversion to belief) and projection (in terms of prediction and mission missing values estimation accuracy).

Integration of Machine Learning-Based Attack Detectors into Defensive Exercises of a 5G Cyber Range

Cybercrime has become more pervasive and sophisticated over the years. Cyber ranges have emerged as a solution to keep pace with the rapid evolution of cybersecurity threats and attacks. Cyber ranges have evolved to virtual environments that allow various IT and network infrastructures to be simulated to conduct cybersecurity exercises in a secure, flexible, and scalable manner. With these training environments, organizations or individuals can increase their preparedness and proficiency in cybersecurity-related tasks while helping to maintain a high level of situational awareness. SPIDER is an innovative cyber range as a Service (CRaaS) platform for 5G networks that offer infrastructure emulation, training, and decision support for cybersecurity-related tasks. In this paper, we present the integration in SPIDER of defensive exercises based on the utilization of machine learning models as key components of attack detectors. Two recently appeared network attacks, cryptomining using botnets of compromised devices and vulnerability exploit of the DoH protocol (DNS over HTTP), are used as the support use cases for the proposed exercises in order to exemplify the way in which other attacks and the corresponding ML-based detectors can be integrated into SPIDER defensive exercises. The two attacks were emulated, respectively, to appear in the control and data planes of a 5G network. The exercises use realistic 5G network traffic generated in a new environment based on a fully virtualized 5G network. We provide an in-depth explanation of the integration and deployment of these exercises and a complete walkthrough of them and their results. The machine learning models that act as attack detectors are deployed using container technology and standard interfaces in a new component called Smart Traffic Analyzer (STA). We propose a solution to integrate STAs in a standardized way in SPIDER for the use of trainees in exercises. Finally, this work proposes the application of Generative Adversarial Networks (GANs) to obtain on-demand synthetic flow-based network traffic that can be seamlessly integrated into SPIDER exercises to be used instead of real traffic and attacks.

Security Evaluation of Situational Awareness in Cyberspace Based on Artificial Neural Network-Back Propagation

Computer network security has become increasingly controversial among many businesses as a result of the rise in cyber threats. Artificial neural network (ANN) is mature research in this field, whereas the traditional algorithm is slower, in feedback error, and has the disadvantage of easy convergence to local extreme value. To guide against these threats, in this paper, ANN-back propagation (BP) algorithm is used to establish the relationship between the level of cyber security situational awareness (CSA) and the perceptual parameters and quantitatively evaluate the situational awareness. This study established the ANN-back propagation (BP) to make the relationship between the level of cyber security situational awareness (CSA) and the perceptual parameters, which evaluates situational awareness. The ANN-BP with variable step size learning strategy and simulated annealing method is used for optimization to build a virtual network environment. The proposed model offers better precision, improved sensitivity, and higher (0.987%) accuracy.

Decentralized Control of a Heterogeneous Human–Robot Team for Exploration and Patrolling

We present a decentralized connectivity-maintenance control framework for a heterogeneous human–robot team. The algorithm is able to manage a team composed of an arbitrary number of mobile robots (drones and ground robots in our case) and humans, for collaboratively achieving exploration and patrolling tasks. Differently from other works on the subject, here the human user physically becomes part of the team, moving in the same environment of the robots and receiving information about the team connectivity through wearable haptics or audio feedback. Although human explores the environment, robots move so as to keep the team connected via a connectivity-maintenance algorithm; at the same time, each robot can also be assigned with a specific target to visit. We carried out three human subject experiments, both in virtual and real environments. Results show that the proposed approach is effective in a wide range of scenarios. Moreover, providing either haptic or audio feedback for conveying information about the team connectivity significantly improves the performance of the considered tasks, although users significantly preferred receiving haptic stimuli w.r.t. the audio ones. Note to Practitioners—Exploration, patrolling, and search-and-rescue are highly dynamic and unstructured scenarios. When considering the operative conditions of such environments, the benefits of multirobot systems are evident. Most tasks can be carried out faster and more robustly by a team of robots with respect to a single unit. There are also situations explicitly requiring the presence of a multirobot team, e.g., using one drone for surveillance of the ground team and one ground mobile robot for carrying supplies. Of course, if the operator(s) in charge of the operation could share the same environment of the robots (i.e., be together with the robots in the field), they would be provided with a level of situational awareness that no teleoperation technology can match as of today. This work presents a framework for controlling heterogeneous teams composed of one human operator and an arbitrary number of aerial and ground mobile robots. The operator moves together with the robotic team and, at the same time, he or she receives meaningful information about the status of the formation. The algorithm only uses the relative position of the drones and humans with respect to each other, and all computations are designed in a decentralized fashion. Decentralization avoids relying on any absolute positioning system (e.g., GPS) or centralized command centers. These features make the proposed framework ready for deployment in different high-impact applications, such as in surveillance, search-and-rescue, and disaster response scenarios.

EEG Feature Analysis Related to Situation Awareness Assessment and Discrimination

In order to discriminate situation awareness (SA) levels on the basis of SA-sensitive electroencephalography (EEG) features, the high-SA (HSA) group and low-SA (LSA) groups, which are representative of two SA levels, were classified according to the situation awareness global assessment technology (SAGAT) scores measured in the multi-attribute task battery (MATB) II tasks. Furthermore, three types of EEG features, namely, absolute power, relative power, and slow-wave/fast-wave (SW/FW), were explored using spectral analysis. In addition, repeated analysis of variance (ANOVA) was conducted in three brain regions (frontal, central, and parietal) × three brain lateralities (left, middle, and right) × two SA groups (LSA and HSA) to explore SA-sensitive EEG features. The statistical results indicate a significant difference between the two SA groups according to SAGAT scores; moreover, no significant difference was found for the absolute power of four waves (delta (δ), theta (θ), alpha (α), and beta (β)). In addition, the LSA group had a significantly lower β relative power than the HSA group in central and partial regions. Lastly, compared with the HSA group, the LSA group had higher θ/β and (θ + α)/(α + β) in all analyzed brain regions, higher α/β in the parietal region, and higher (θ + α)/β in all analyzed regions except for the left and right laterality in the frontal region. The above SA-sensitive EEG features were fed into principal component analysis (PCA) and the Bayes method to discriminate different SA groups, and the accuracies were 83.3% for the original validation and 70.8% for the cross-validation. The results provide a basis for real-time assessment and discrimination of SA by investigating EEG features, thus contributing to monitoring SA decrement that might lead to threats to flight safety.

Comparing Levels and Types of Situational-Awareness based Agent Transparency in Human-Agent Collaboration

Increasing agent transparency is an ongoing challenge for Human-Agent Collaboration (HAC). Chen et al. proposed the three level SAT framework to improve Agent Transparency and users’ Situational Awareness (SA) by informing about (1) what the agent is doing, (2) why the agent is doing it and (3) what the agent will do next. Explanations can be descriptive (informing the user decision-making process) or prescriptive (guiding the user toward a pre-determined choice). To study these differences, we conducted a 3 (SA level) x 2 (explanation types) online between-group user experiment (n=180) where we designed six visual explanations and tested their impact on task performance, reliance, reported trust, cognitive load and situational awareness in a goal-oriented HAC interactive task. We found that SA level 1 explanations led to better task performance, while SA level 2 explanations increased trust. Moreover, descriptive explanations had a more positive impact on participants compared to prescriptive explanations.

A multi-level analytic framework for disaster situational awareness using Twitter data

During a natural disaster, mining messages from social media platforms can facilitate local agencies, rescue teams, humanitarian aid organizations, etc., to track the situational awareness of the public. However, for different stakeholders, the concerns about people’s situational awareness in a natural disaster event are different. Therefore, I developed a Twitter-based analytic framework to take perception-level situational awareness, humanitarian-level situational awareness, and action-level situational awareness into consideration. Specifically, perception-level situational awareness mainly reflects people’s perception of the ongoing natural disaster event (i.e., if people are discussing the disaster event). Decision-makers can rapidly have a big picture of severely impacted regions. Humanitarian-level situational awareness represents tweets that are associated with the humanitarian categories based on the definition from the United Nations Office for the Coordination of Humanitarian Affairs. The detection of humanitarian-level situational awareness can help response teams understand the specific situations and needs of local communities. In terms of the action-level situational awareness, I extracted noun-verb pairs in each tweet to explicitly represent the specific event described in a given tweet, so that the response teams can quickly act on the situation case by case. Moreover, to shed light on disaster resilience and social vulnerability, I further examined the demographic characteristics of three levels of situational awareness. I empirically demonstrated the analytic framework using geo-tagged tweets during 2018 Hurricane Michael.

Using the Endsley Model to Evaluate Simulation-Based Situation Awareness Training for Medical and Nursing Students in India: A Qualitative Analysis.

Situation awareness (SA) training is a vital part of healthcare training, and opportunities to provide SA training to healthcare workers are limited in low- and middle-income countries. We aimed to analyze undergraduate medical and nursing students' perception of their understanding of SA through an interprofessional obstetric neonatal emergency simulation workshop (ONE-Sim) and subsequently evaluate their perceived changes in SA understanding using the Endsley model ( Hum Factors 1995;37(1):32-64). Feedback on SA before and after the workshop was collected through questionnaire-based surveys. Thematic analysis was performed, with themes emerging from an inductive analysis followed by a deductive analysis using the Endsley model. The themes emerging from the inductive analysis included environmental awareness, evolving knowledge, skill development, and applicability to practice. These aligned with the 3 levels of SA in the Endsley model in the deductive analysis suggesting that participants transformed their perception, comprehension, and projection of SA after the workshop. Simulation-based education enhanced SA perception in obstetric and neonatal emergencies for medical and nursing students in a low- and middle-income country, and the Endsley model is a feasible framework to measure learner perceived changes in SA understanding through simulation-based education.

Modeling eye movement in dynamic interactive tasks for maximizing situation awareness based on Markov decision process

For complex dynamic interactive tasks (such as aviating), operators need to continuously extract information from areas of interest (AOIs) through eye movement to maintain high level of situation awareness (SA), as failures of SA may cause task performance degradation, even system accident. Most of the current eye movement models focus on either static tasks (such as image viewing) or simple dynamic tasks (such as video watching), without considering SA. In this study, an eye movement model with the goal of maximizing SA is proposed based on Markov decision process (MDP), which is designed to describe the dynamic eye movement of experienced operators in dynamic interactive tasks. Two top-down factors, expectancy and value, are introduced into this model to represent the update probability and the importance of information in AOIs, respectively. In particular, the model regards sequence of eye fixations to different AOIs as sequential decisions to maximize the SA-related reward (value) in the context of uncertain information update (expectancy). Further, this model was validated with a flight simulation experiment. Results show that the predicted probabilities of fixation on and shift between AOIs are highly correlated (R = 0.928 and R = 0.951, respectively) with those of the experiment data.

Investigating explanations in conditional and highly automated driving: The effects of situation awareness and modality

With the level of automation increases in vehicles, such as conditional and highly automated vehicles (AVs), drivers are becoming increasingly out of the control loop, especially in unexpected driving scenarios. Although it might be not necessary to require the drivers to intervene on most occasions, it is still important to improve drivers’ situation awareness (SA) in unexpected driving scenarios to improve their trust in and acceptance of AVs. In this study, we conceptualized SA at the levels of perception (SA L1), comprehension (SA L2), and projection (SA L3), and proposed an SA level-based explanation framework based on explainable AI. Then, we examined the effects of these explanations and their modalities on drivers’ situational trust, cognitive workload, as well as explanation satisfaction. A three (SA levels: SA L1, SA L2 and SA L3) by two (explanation modalities: visual, visual + audio) between-subjects experiment was conducted with 340 participants recruited from Amazon Mechanical Turk. The results indicated that by designing the explanations using the proposed SA-based framework, participants could redirect their attention to the important objects in the traffic and understand their meaning for the AV system. This improved their SA and filled the gap of understanding the correspondence of AV’s behavior in the particular situations which also increased their situational trust in AV. The results showed that participants reported the highest trust with SA L2 explanations, although the mental workload was assessed higher in this level. The results also provided insights into the relationship between the amount of information in explanations and modalities, showing that participants were more satisfied with visual-only explanations in the SA L1 and SA L2 conditions and were more satisfied with visual and auditory explanations in the SA L3 condition. Finally, we found that the cognitive workload was also higher in SA L2, possibly because the participants were actively interpreting the results, consistent with a higher level of situational trust. These findings demonstrated that properly designed explanations, based on our proposed SA-based framework, had significant implications for explaining AV behavior in conditional and highly automated driving.

Doing the right thing, in the right place, with the right people, at the right time; a study of the development of judgment and decision making in mid-career outdoor instructors

ABSTRACT In recent years, increased interest in the Professional judgment and Decision Making (PJDM) of outdoor instructors has focused on experts. This paper builds on earlier work, exploring the components and development of PJDM in mid-career outdoor instructors. The study conducted and thematically analysed semi-structured interviews with nine senior instructors working for Outward Bound Trust (the Trust) in the UK. Findings included two over-arching themes; (1) a Hahnian approach to development, in line with the philosophies that underpin the Trust’s work, and (2) practical wisdom: the synergy of contextual comprehension and appropriate options. We conclude that these mid-career outdoor instructors’ decision making is dependent on high levels of situational awareness, situational demands, and availability of appropriate options as a course of action. We further identify a philosophical chain between the Trust’s ethos, instructor development, and adaptive expertise. These skills develop through a process akin to a cognitive apprenticeship within the Trust’s community of practice.

Remote Training of Neurointerventions by Audiovisual Streaming

BackgroundRemote access of trainees to training centers via video streaming (tele-observership, e‑fellowship) emerges as an alternative to acquire knowledge in endovascular interventions. Situational awareness is a summary term that is also used in surgical procedures for perceiving and understanding the situation and projecting what will happen next. A high situational awareness would serve as prerequisite for meaningful learning success during tele-observerships. We hypothesized that live perception of the angiographical procedures using streaming technology is feasible and sufficient to gain useful situational awareness of the procedure.MethodsDuring a European tele-observership organized by the European Society of Minimally Invasive Neurological Therapy (ESMINT) and its trainee association (EYMINT), a total of six neurointerventional fellows in five countries observed live cases performed by experienced neurointerventionalists (mentors) in six different high-volume neurovascular centers across Europe equipped with live-streaming technology (Tegus Medical, Hamburg, Germany). Cases were prospectively evaluated during a 12-month period, followed by a final questionnaire after completion of the course.ResultsA total of 102/161 (63%) cases with a 1:1 allocation of fellow and mentor were evaluated during a 12-month period. Most frequent conditions were ischemic stroke (27.5%), followed by embolization of unruptured aneurysms (25.5%) and arteriovenous malformations (AVMs) (15.7%). A high level of situational awareness was reported by fellows in 75.5% of all cases. After finishing the program, the general improvement of neurointerventional knowledge was evaluated to be extensive (1/6 fellows), substantial (3/6), and moderate (2/6). The specific fields of improvement were procedural knowledge (6/6 fellows), technical knowledge (3/6) and complication management (2/6).ConclusionOnline streaming technology facilitates location-independent training of complex neurointerventional procedures through high levels of situational awareness and can therefore supplement live hands-on-training. In addition, it leads to a training effect for fellows with a perceived improvement of their neurointerventional knowledge.

An Analysis of Driver Gaze Behaviour at Roundabouts

Determining driver situational awareness is one of the most difficult tasks in driver performance profiling. Perception, being the first level of driver situational awareness, is perhaps the only objectively quantifiable factor for driver inattention detection. In the context of this paper, driver perception is defined as their ability to understand their operating environment and peripheral contextual information. One of the most reliable means of determining driver perception is through eye tracking and gaze concentration. In this study, a set of carefully chosen in-vehicle gaze targets are used to determine driver gaze concentration on/off the road. The time drivers spend looking at these predefined gaze targets has been used to characterise driver visual behaviour at roundabouts during normal driving. In total, 24 drivers completed a 22 km route, including two 5-exit two-lane roundabouts in urban and suburban traffic. This study shows that on the approach to roundabouts, driver gaze direction and gaze concentration was often diverted away from the direction of travel of the vehicle and for some drivers it took a significant amount of time to return their gaze towards the direction of travel of the vehicle. The study is based on the premise that a driver not looking at key regions in a scene cannot perceive the contents of those regions with sufficient detail for safe driving. It is not possible to determine a driver’s level of understanding of a scene, unless the parameters of the scene as well as the driver’s response is determined in real time.

The Situation Awareness Framework for Explainable AI (SAFE-AI) and Human Factors Considerations for XAI Systems

Recent advances in artificial intelligence (AI) have drawn attention to the need for AI systems to be understandable to human users. The explainable AI (XAI) literature aims to enhance human understanding and human-AI team performance by providing users with necessary information about AI system behavior. Simultaneously, the human factors literature has long addressed important considerations that contribute to human performance, including how to determine human informational needs, human workload, and human trust in autonomous systems. Drawing from the human factors literature, we propose the Situation Awareness Framework for Explainable AI (SAFE-AI), a three-level framework for the development and evaluation of explanations about AI system behavior. Our proposed levels of XAI are based on the informational needs of human users, which can be determined using the levels of situation awareness (SA) framework from the human factors literature. Based on our levels of XAI framework, we also suggest a method for assessing the effectiveness of XAI systems. We further detail human workload considerations for determining the content and frequency of explanations as well as metrics that can be used to assess human workload. Finally, we discuss the importance of appropriately calibrating user trust in AI systems through explanations along with other trust-related considerations for XAI, and we detail metrics that can be used to evaluate user trust in these systems.