Operator Situation Awareness Research Articles

Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues

The human factors literature on intelligent systems was reviewed in relation to the following: efficient human supervision of multiple robots, appropriate human trust in the automated systems, maintenance of human operator's situation awareness, individual differences in human-agent (H-A) interaction, and retention of human decision authority. A number of approaches-from flexible automation to autonomous agents-were reviewed, and their advantages and disadvantages were discussed. In addition, two key human performance issues (trust and situation awareness) related to H-A teaming for multirobot control and some promising user interface design solutions to address these issues were discussed. Some major individual differences factors (operator spatial ability, attentional control ability, and gaming experience) were identified that may impact H-A teaming in the context of robotics control.

Malware Target Recognition of Unknown Threats

Organizations traditionally use signature-based commercial antivirus products as a frontline defense against malware, but advanced persistent threats craft custom malicious tools to achieve their objectives. Organizations safeguarding sensitive information have difficulty in identifying new malware threats among millions of benign executables using only signature-based antivirus systems. This paper extends a performance-based malware target recognition architecture that currently uses only static heuristic features. Experimental results show that this architectural component achieves an overall test accuracy of 98.5% against a malware set collected from operational environments, while three commercial antivirus products combine for a detection accuracy of only 60% with their most sensitive settings. Implementations of this architecture will enable organizations to self-discover new malware threats, providing enhanced situation awareness for cyberspace operators in hostile threat environments.

Design and Evaluation of an Exploration Assistant for Human Deep Space Risk Mitigation

Abstract This paper describes the development of a Virtual Camera (VC) system to improve astronaut and mission operations situation awareness while exploring other planetary bodies. It is claimed that the advanced interaction media capability of the VC can improve situation awareness as the distribution of human space exploration roles change in deep space exploration. It can minimize the risk of astronauts exploring unknown reaches of the solar system with limited previous knowledge of the area under exploration. It can be thought of as a tour guide with captured expertise to aid exploration. It provides a collaborative tool so that ground-based expert knowledge can be captured and be easily assessable in the remote deep space environment. A tablet PC-based interactive database application has been developed and tested for usability and capability to improve situation awareness. The method of testing will be described as well as testing results.

Visual feedback system to reduce errors while operating roof bolting machines

ProblemOperators of roof bolting machines in underground coal mines do so in confined spaces and in very close proximity to the moving equipment. Errors in the operation of these machines can have serious consequences, and the design of the equipment interface has a critical role in reducing the probability of such errors. MethodsAn experiment was conducted to explore coding and directional compatibility on actual roof bolting equipment and to determine the feasibility of a visual feedback system to alert operators of critical movements and to also alert other workers in close proximity to the equipment to the pending movement of the machine. The quantitative results of the study confirmed the potential for both selection errors and direction errors to be made, particularly during training. ResultsSubjective data confirmed a potential benefit of providing visual feedback of the intended operations and movements of the equipment. ImpactThis research may influence the design of these and other similar control systems to provide evidence for the use of warning systems to improve operator situational awareness.

Empirical research on an ecological interface design for improving situation awareness of operators in an advanced control room

The purpose of this study is to validate whether an ecological interface design (EID) improves operators’ situation awareness in an advanced control room of a nuclear power plant (NPP). EID is defined as an approach to interface design that was introduced specifically for complex socio-technical, real-time, and dynamic systems. The EID technology has not yet been adapted to the nuclear power industry due to lack of empirical studies. Especially in a situational awareness aspect, many researchers have predicted that the EID will support operators to detect unanticipated events. Just a few studies, however, unveiled the positive effect of the EID display on human performance using a full scoped simulator.In this study, to investigate whether an EID improves operators’ situational awareness, we developed an EID prototype for nuclear power operations and a partial scoped dynamic mockup to validate the effectiveness of the EID prototype. Three experienced operators were involved as subjects in our study and they were fully well trained for using the EID prototype. We compared two types of situations in terms of situation awareness. One is mimic based information display and the other is a mimic plus EID based information display. The result of our study revealed that a mimic plus EID based information display is more effective than a mimic based information display in terms of situation awareness. This study is significant in that the EID as an emerging technology is adoptable to a digitalized control room in an aspect of improving operators’ situation awareness.

Adaptable and Adaptive Automation for Supervisory Control of Multiple Autonomous Vehicles

Supervisory control of multiple autonomous vehicles raises many issues concerning the balance of system autonomy with human interaction for optimal operator situation awareness and system performance. An unmanned vehicle simulation designed to manipulate the application of automation was used to evaluate participants’ performance on image analysis tasks under two automation control schemes: adaptable (level of automation directly manipulated by participant throughout trials) and adaptive (level of automation adapted as a function of participants’ performance on four types of tasks). The results showed that while adaptable automation increased workload, it also improved change detection, as well as operator confidence in task-related decision-making.

SIMULTANEOUS VISUALIZATION OF DIFFERENT UTILITY NETWORKS FOR DISASTER MANAGEMENT

Abstract. Cartographic visualizations of crises are used to create a Common Operational Picture (COP) and enforce Situational Awareness by presenting and representing relevant information. As nearly all crises affect geospatial entities, geo-data representations have to support location-specific decision-making throughout the crises. Since, Operator's attention span and their working memory are limiting factors for the process of getting and interpreting information; the cartographic presentation has to support individuals in coordinating their activities and with handling highly dynamic situations. The Situational Awareness of operators in conjunction with a COP are key aspects of the decision making process and essential for coming to appropriate decisions. Utility networks are one of the most complex and most needed systems within a city. The visualization of utility infrastructure in crisis situations is addressed in this paper. The paper will provide a conceptual approach on how to simplify, aggregate, and visualize multiple utility networks and their components to meet the requirements of the decision-making process and to support Situational Awareness.

Situation Awareness in the NextGen Air Traffic Management System

The Next Generation Air Transportation System (NextGen) is expected to revolutionize how air traffic will be managed and significantly enhance the capacity of the National Airspace System in the United States. Because NextGen is a system that requires a high level of interaction between human operators and technology, there is a need to assess how NextGen tools and concepts of operation will impact operator situation awareness (SA). The goal of this study is to highlight the importance of SA to operator performance in complex systems. The defended view is that SA is situated, which means that operators rely in fundamental ways on interactions with external tools to maintain their understanding of a situation, minimizing internal memory storage of task relevant information. Consequently, SA metrics must be conducted in the context where operators have access to their displays and tools. The article ends with a discussion of how the situated SA approach can be used to explain the acquisition of shared SA in distributed teams and to guide applied research relating to the evaluation and development of NextGen concepts and tools.

A Baseline Evaluation of Situation Awareness For Electric Power System Operation Supervisors

High levels of situation awareness (SA) are critical for successful power transmission and distribution (T&D) operations. Robust and efficient operations within power T&D require that system operators maintain high levels of situation awareness to fully understand the current state of the complex and dynamically changing power grid system, and to successfully project future system states in order to respond in a timely manner. This paper discusses the results of an evaluation study conducted at an electric power control room facility to form a baseline understanding of the impact that current control room conditions have on system operator SA. An objective measure of operator SA was collected during the study through the use of the Situation Awareness Global Assessment Technique (SAGAT), which was modified to accommodate the real-time operational demands of the control room. The study also gathered subjective SA measures for the system operator position. The insights gleaned from this analysis can provide guidance when generating recommendations for modifying or redesigning control center displays to support operator situation awareness.

Performance-based Adaptive Automation for Supervisory Control

Supervisory control of multiple autonomous vehicles raises many issues concerning the balance of system autonomy with human interaction for optimal operator situation awareness and system performance. An unmanned vehicle simulation designed to manipulate the application of automation was used to evaluate participants’ performance on image analysis tasks under two automation conditions: static (level of automation remained constant throughout trials) and adaptive (level of automation adapted as a function of performance on five types of tasks). The results showed that performance-based adaptation of the image task autonomy level improved performance on this task, as well as other tasks. Additionally, participants preferred the adaptive automation condition and felt that it reduced their cognitive workload and aided performance. Research issues are identified to further evaluate performance-based adaptation for supervisory control.

Knowing What You Know: The Role of Meta-Situation Awareness in Predicting Situation Awareness

To help operators handle increasing task demands, automated systems are increasingly being used in modern work settings. However, a potential consequence of such systems is the out of the loop performance syndrome, which refers to the reduced ability of operators working with high levels of automation support to take over manual control in the event of an automation failure (e.g., Wiener & Curry, 1980). One of the main factors considered responsible for this is a loss of operator situation awareness (SA; Endsley & Kiris, 1995). In order to improve SA and consequently improve performance, it becomes important to better understand the process of SA and the underlying factors that predict SA (e.g., Durso & Sethumadhavan, 2008). The role of meta-SA in predicting SA when individuals control air traffic with automation support is investigated in this paper. Results have practical implications for the development of SA training programs.

THE USE OF AUGMENTED REALITY IN COMMAND AND CONTROL SITUATION AWARENESS

Augmented reality is not completely new technology, but rather an interestingcombination of existing technologies, facilitated by the rapid progress made onultra-portable devices such as smart cellular telephones. It has the potential toenhance the situation awareness of military operators. This study identifies possibleuses of augmented reality in command and control applications with specificattention to situation awareness in the South African context. Applications across thedifferent command and control functions, as well as at the different levels of militaryoperations are considered. The article concludes with some concept definitions foraugmented reality applications.

Investigation of automation deployment in the main control room of nuclear power plants by using adaptive automation

AbstractRecently, with the progress of computer technology, the digitalization of human–system interface (HSI) in the main control room (MCR) of advanced nuclear power plants (NPPs) has become an important issue. A higher level of automation means that the computers execute the greater part of the task. The frequency, however, of automation‐induced system failures, including human errors, prompts us to more closely investigate the influences of automation. The objective of this study is to measure the effects of adaptive automation on the primary task performance, operator situation awareness (SA), and workload in an advanced MCR of an advanced NPP to reduce the number of failures and improve human performance. This study selects automatic mode and semi‐automatic mode to compose five different types of automation allocations. The experiment scenario is a reactor shutdown task. A personal computer transient analyzer (PCTran) is used to simulate the HSI of an MCR. Twelve participants are required to perform the experiment scenario on the automation allocation types. The dependent variables were the primary task performance; the subjective workload was measured using an NASA‐Task Load Index (NASA‐TLX) questionnaire; and three levels of SA were estimated by SA queries. Experimental results revealed that the automation allocations made up of higher ratio automatic modes led to superior primary task performance, lower operator workload, and lower frequency of human errors. In the high ratio automatic mode allocation, the operators have the lowest SA queries score. This study provides additional information for future digital HSI design in the MCRs of advanced NPPs. © 2011 Wiley Periodicals, Inc.

Automation-Level Transference Effects in Simulated Multiple Unmanned Aerial Vehicle Control

Supervisory control of multiple unmanned aerial vehicles (UAVs) raises many questions concerning the balance of system autonomy with human interaction for effective operator situation awareness and system performance. The reported experiment used a UAV simulation environment to evaluate two applications of autonomy levels across two primary control tasks: allocation (assignment of sensor tasks to vehicles) and router (determining vehicles’ flight plans). In one application, the autonomy level was the same across these two tasks. In the other, the autonomy levels differed, one of the two tasks being more automated than the other. Trials also involved completion of other mission-related secondary tasks as participants supervised three UAVs. The results showed that performance on both the primary tasks and many secondary tasks was better when the level of automation was the same across the two sequential primary tasks. These findings suggest that having the level of automation similar across closely coupled tasks reduces mode awareness problems, which can negate the intended benefits of a fine-grained application of automation. Several research issues are identified to further explore the impact of automation-level transference in supervisory control applications involving the application of automation across numerous tasks.

Moonlight in Miami: a field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise

This article explores human-robot interaction during a 16-hr, high-fidelity urban search and rescue disaster response drill with teleoperated robots. This article examines operator situation awareness and technical search team interaction using communication analysis. It analyzes situation awareness, team communication, and the interaction of these constructs using a systematic coding scheme designed for this research. The findings indicate that operators spent significantly more time gathering information about the state of the robot and the state of the environment than they did navigating the robot. Operators had difficulty integrating the robot's view into their understanding of the search and rescue site. They compensated for this lack of situation awareness by communicating with team members at the site, attempting to gather information that would provide a more complete mental model of the site. They also worked with team members to develop search strategies. The article concludes with suggestions for design and future research.

Functional versus Schematic Overview Displays: Impact on Operator Situation Awareness in Process Monitoring

It is important for console operators to maintain good situation awareness while monitoring process operations. A key component that helps operators achieve better situation awareness is how the information about the process is displayed on their console. Most oil and gas refineries still utilize traditional schematic displays and connecting lines between equipments to present process information. In this study, we designed a functional layout for an overview display. This layout shows the same process parameters as a traditional schematic display, but in a different format. Eighteen plant operators viewed both displays to monitor a crude unit process and their situation awareness was measured. Results indicated that operators' situation awareness was significantly higher when they monitored the process on a functional display compared with a schematic display. The implications of the findings for continuous process control are discussed.

Enhanced Process Control – Maximizing Coal Handling Preparation Plant Productivity

The benefits of various enhanced methodologies in process control are well documented and include safer operation, reduced environmental impact, and improved efficiencies, qualities, and throughputs [1, 2]. Numerous applications of these techniques exist in the fields of petrochemical, oil and gas, pulp and paper, and mineral processing. However, the application of these has been slow to filter through into coal preparation [3]. Variability associated with plant feed rates and the coal seam(s) often results in problems in achieving plant throughput and the desired quality. Since plants are generally designed to average feed specifications and as the feed material qualities change (more or less ash, fines, etc.), the plant operators and control system must compensate accordingly. This article will present two different case studies demonstrating the benefits achievable through improvements in process control techniques. The performance of industrial plants is significantly affected by control loop configuration and the effectiveness of the operator and the interface in use. The first case study will highlight the benefits of prototyping and simulating enhanced control schemes before implementation on the actual plant. The second case study demonstrates how to improve operator situation awareness through enhanced graphics techniques and process alarm rationalization.

BeAware!—Situation awareness, the ontology-driven way

Information overload is a severe problem for human operators of large-scale control systems as, for example, encountered in the domain of road traffic management. Operators of such systems are at risk to lack situation awareness, because existing systems focus on the mere presentation of the available information on graphical user interfaces—thus endangering the timely and correct identification, resolution, and prevention of critical situations. In recent years, ontology-based approaches to situation awareness featuring a semantically richer knowledge model have emerged. However, current approaches are either highly domain-specific or have, in case they are domain-independent, shortcomings regarding their reusability. In this paper, we present our experience gained from the development of BeAware!, a framework for ontology-driven information systems aiming at increasing an operator's situation awareness. In contrast to existing domain-independent approaches, BeAware!'s ontology introduces the concept of spatio-temporal primitive relations between observed real-world objects thereby improving the reusability of the framework. To show its applicability, a prototype of BeAware! has been implemented in the domain of road traffic management. An overview of this prototype and lessons learned for the development of ontology-driven information systems complete our contribution.

Human Factors of Precision Taxiing under Two Levels of Automation

New air traffic control concepts and automation tools designed to improve the capacity of the National Airspace System are continually researched, including those that test early prototypes and their impact on the operators to adequately perform the task at hand. This study investigated the workload, situation awareness and other human factors issues related to using a prototype surface automation tool (“GoSAFE-Ground Operations Situation Awareness and Efficiency Tool”) for managing precision taxi operations, with increased traffic relative to current day operations. Our team of controller participants working four air traffic positions handled airport traffic using GoSAFE under two different levels of automation: (1) Mixed voice/datalink [partially equipped] and (2) Datalink only [fully equipped]. The study found statistically significant differences on workload due to automation level. The impact of automation was also investigated for the four air traffic control positions. Results on one situation awareness measure were largely consistent with the workload findings. While realizing these differences, workload and situation awareness levels remained manageable and reasonable across all conditions and positions. Our controller participants also reported acceptability and feasibility ratings and provided feedback on the usability of the user interface. Results suggested that the GoSAFE automation helped the participants cope with the differences in their respective areas of responsibility and an increased volume of traffic, while maintaining adequate information resources required for effective air traffic control operations. The study-participants did have difficulty with some of the interface characteristics of the automation tool, such as use of color and data block size. As with any prototype surface automation tool, additional research and development needs to address these types of issues in order to insure effective use of the tool. This research will make it possible to ascertain the future impact of the tool on other system characteristics such as throughput and safety.

Evaluating a Human-Robot Interface for Exploration Missions

The research reported in this paper concerns the design, implementation, and experimental evaluation of a Human-Robot Interface for stationary remote operators, implemented for a PC computer. The GUI design and functionality is described. An Autonomy Management Model has been implemented and explained. We have conducted user evaluation, making two set of experiments, that will be described and the resulting data analyzed. The conclusions give an insight on the most important usability concerns, regarding the operator situational awareness. The scalability of the interface is also experimentally studied.