Objective Measures Of Situation Awareness Research Articles

AAIM Recommendations to Improve Learner Transitions

xxx.

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM.

To examine evidence of sensitivity, predictiveness, and methodological concerns regarding direct, objective measures of situation awareness (SA). The ability to objectively measure SA is important to the evaluation of user interfaces and displays, training programs, and automation initiatives, as well as for studies that seek to better understand SA in both individuals and teams. A number of methodological criticisms have been raised creating significant confusion in the research field. A meta-analysis of 243 studies was conducted to examine evidence of sensitivity and predictiveness, and to address methodological questions regarding Situation Awareness Global Assessment Technique (SAGAT), Situation Present Assessment Technique (SPAM), and their variants. SAGAT and SPAM were found to be equally predictive of performance. SPAM (64%) and real-time probes (73%) were found to have significantly lower sensitivity in comparison to SAGAT (94%). While SAGAT was found not to be overly memory reliant nor intrusive into operator performance, SPAM resulted in problems with intrusiveness in 40% of the studies examined, as well as problems with speed-accuracy tradeoffs, sampling bias, and confounds with workload. Concerns about memory reliance, the utility of these measures for assessing Team SA, and other issues are also addressed. SAGAT was found to be a highly sensitive, reliable, and predictive measure of SA that is useful across a wide variety of domains and experimental settings. Direct, objective SA measurement provides useful and diagnostic insights for research and design in a wide variety of domains and study objectives.

The role of metacognition in the relationship between objective and subjective measures of situation awareness

The present paper is concerned with the relationship between subjective and objective measures of situation awareness (SA) within the context of a realistic command and control (C2) scenario. A 2-day scenario involved a joint military–civilian C2 exercise relating to the (crisis) management of a large event held in a metropolitan area. The role of participants comprised both the acquisition and maintenance of SA of each crisis event and the coordination of an appropriate course of action. Two measures of SA were collected at the end of each day–Situation Awareness Rating Technique (SART) and Quantitative Assessment of Situation Awareness (QUASA). No correlation was found between QUASA confidence and accuracy scores. However, SART SA, SART S(upply) and SART U(nderstanding) scores showed a negative correlation with QUASA accuracy scores. These results are interpreted in light of Koriat et al.'s (Koriat, A., et al., 2008. Information-based and experience-based metacognitive judgement: evidence from subjective confidence. In: J. Dunlosky and R.A. Bjork, eds. Handbook of metamemory and memory. New York: Taylor & Francis, 117–135) framework of information- and experience-based metacognitive judgements and practical implications for the subjective measurement of SA are discussed. SA is a critical real-world concept, yet its measurement remains ambiguous. The relationship between subjective and objective SA measures was examined within the context of a realistic C2 scenario. Using theories of metacognition, unique insights were gained into how subjective SA plays a strategic role in the acquisition of SA.

Situation Awareness in Driving While Using Adaptive Cruise Control and a Cell Phone

This research investigated the effects of an adaptive cruise control (ACC) system, and cell phone use in driving, on a direct objective measure of situation awareness (SA). Subjects drove a virtual car in a medium-fidelity driving simulation and performed a following task. Half of the subjects were required to respond to cell phone calls and all subjects completed trials with and without use of the ACC system. SA was measured using a simulation freeze technique and SA queries on the driving situation. Results indicated use of the ACC system to improve driving task SA under normal driving conditions, and cell phone conversations degraded SA. Results also revealed the ACC system to improve safe driving headway distance. Although the deviations in headway distance from an optimum were greater during cell phone conversations, this did not prove to be significant in terms of performance under normal driving conditions.

Situation awareness and workload in driving while using adaptive cruise control and a cell phone

Little work has empirically examined the cognitive construct of situation awareness (SA) in driving tasks involving the use of advanced in-vehicle automated technologies and personal communication devices. This research investigated the effects of an adaptive cruise control (ACC) system, and cell phone use in driving, on a direct and objective measure of SA, and assessed the competition of multiple driving and communication tasks for limited mental resources in terms of driving performance. Eighteen participants drove a virtual car in a driving simulation and performed a following task involving changes in speed and lateral position. Half of the participants were required to respond to cell phone calls and all completed trials with and without use of the ACC system. Task performance was measured in terms of lane deviations and speed control in tracking a lead vehicle, as well as headway distance in the following task. SA was measured using a simulation freeze technique and SA queries on the driving situation. Subjective workload was measured using a uni-dimensional mental workload rating. Results indicated use of the ACC system to improve driving task SA under typical driving conditions, and to reduce driver mental workload. However, the cell phone conversation caused deleterious effects on driving SA and increased driver mental load. The cell phone conversation (secondary task) competed for limited mental resources of drivers, leading to less attention to, and accurate knowledge of, the driving situation. Results also revealed the ACC system to improve driving performance along multiple dimensions; however, the cell phone did not have an effect. The latter result may be attributed to a short duration of the cell phone conversations during the experiment. This study has implications for the implementation of in-vehicle automation to support driver SA under normal driving conditions and regulations on the use of cell phones while driving. Relevance to Industry The results of this study have relevance to the introduction of advanced automation in commercial vehicles for supporting driver SA and regulation of cell phone use in driving. The study brings to light the critical role of attention-demanding distracter tasks, such as cell phone conversation while driving and using in-vehicle automation.

Objective measures of situation awareness in a simulated medical environment.

One major limitation in the use of human patient simulators is a lack of objective, validated measures of human performance. Objective measures are necessary if simulators are to be used to evaluate the skills and training of medical practitioners and teams or to evaluate the impact of new processes or equipment design on overall system performance. Situation awareness (SA) refers to a person's perception and understanding of their dynamic environment. This awareness and comprehension is critical in making correct decisions that ultimately lead to correct actions in medical care settings. An objective measure of SA may be more sensitive and diagnostic than traditional performance measures. This paper reviews a theory of SA and discusses the methods required for developing an objective measure of SA within the context of a simulated medical environment. Analysis and interpretation of SA data for both individual and team performance in health care are also presented.

Comparison of Two Situation Awareness Metrics: SAGAT and SA-SWORD

The role of situation awareness (SA) in aircrew safety and survivability continues to be of considerable interest to the aviation community. Various methods of measuring SA have been proposed. Among these are SAGAT and SA-SWORD. The former is an objective measure of SA, while the latter is based on operators' subjective comparisons. This paper reports the findings of a study that compared these two metrics in a simulated low-level ingress scenario. It was found that the type of SA metric used did not affect performance measures collected. While independent variables manipulated had large and statistically significant effects upon SA-SWORD data, this was not true for SAGAT data and the two SA measures were not significantly correlated. Advantages and disadvantages of the two techniques are discussed, and lessons learned from their application in this study are reported.

Measuring Driver Adaptation to In-Vehicle Information Systems: Disassociation of Subjective and Objective Situation Awareness Measures

This study examined a range of design parameters for an in-vehicle signing and warning system, using a low-fidelity driving simulator and a variety of driver performance measures. A series of probe questions provided an objective measure of situation awareness, and drivers' confidence ratings in these answers provided a subjective measure of situation awareness. The correspondence between the accuracy of the answers to the probe questions and subjective confidence in the answers to the probe questions provides a measure of meta-situation awareness. Meta-situation awareness is a necessary condition for the validity of any subjective measure of situation awareness. Analysis of subjective and objective measures of situation awareness showed that they were both sensitive to design parameters, but that they responded in different ways. In addition, they were poorly calibrated and were only slightly correlated. This indicates a poor level of meta-situation awareness. Poor meta-situation awareness argues for great caution in using subjective measures of situation awareness. The results also suggest that meta-situation awareness may be a useful concept in explaining human performance.

Predictive Utility of an Objective Measure of Situation Awareness

Situation awareness (SA) can generally be defined as the pilot's mental model of the world around him. The ability of the pilot to maintain situation awareness has been recognized in the pilot community as crucial to mission success and survivability. Little empirical work has been done to explore this construct, however, as measurement techniques have been lacking. The Situation Awareness Global Assessment Technique (SAGAT) was developed and validated as an objective measure of a pilot's situation awareness to meet this need. An investigation of the relationship between situation awareness and performance was conducted using SAGAT. The study evaluated the utility of situation awareness in a manned fighter sweep tactical mission using experienced pilot subjects. A total of 90 trials were conducted. The results support the importance of situation awareness for mission effectiveness. Blue Team subjects were almost twice as likely to later kill an aircraft when they had SA of that aircraft, as measured by SAGAT, than when they did not. The effect of rules of engagement, mission scenario, aircraft and weapons systems on the relationship between SA and performance is discussed.