Adaptive Automation Research Articles

N-SEEV in SOAS: Predicting Time to Notice for Multi-Modal Cockpit Alerting Events

This paper presents the current work to extend the N-SEEV model of visual attention to both the auditory and tactile modalities in support a cockpit adaptive automation system for pilot spatial disorientation. Cockpit countermeasure systems use visual, auditory and tactile modalities to communicate problems to the pilot. The SOAS spatial orientation aiding system uses all three modalities to support a disoriented pilot. The system initiates increasingly intrusive countermeasure as it determines that the probability and severity of a detected disorientation is increasing. N-SEEV has been included within SOAS to support the change to countermeasure levels based on the prediction of the pilot noticing countermeasure onset. Previous versions of N-SEEV could predict a time-to-notice for the onset of a visual cockpit countermeasure. In this work, N-SEEV is extended to include noticing predictions for both the auditory and tactile modes.

The effects of day-to-day variability of physiological data on operator functional state classification

The application of pattern classification techniques to physiological data has undergone rapid expansion. Tasks as varied as the diagnosis of disease from magnetic resonance images, brain-computer interfaces for the disabled, and the decoding of brain functioning based on electrical activity have been accomplished quite successfully with pattern classification. These classifiers have been further applied in complex cognitive tasks to improve performance, in one example as an input to adaptive automation. In order to produce generalizable results and facilitate the development of practical systems, these techniques should be stable across repeated sessions. This paper describes the application of three popular pattern classification techniques to EEG data obtained from asymptotically trained subjects performing a complex multitask across five days in one month. All three classifiers performed well above chance levels. The performance of all three was significantly negatively impacted by classifying across days; however two modifications are presented that substantially reduce misclassifications. The results demonstrate that with proper methods, pattern classification is stable enough across days and weeks to be a valid, useful approach.

Explicit control of adaptive automation under different levels of environmental stress

This article examines the effectiveness of three different forms of explicit control of adaptive automation under low- and high-stress conditions, operationalised by different levels of noise. In total, 60 participants were assigned to one of three types of automation design (free, prompted and forced choice). They were trained for 4 h on a highly automated simulation of a process control environment, called AutoCAMS. This was followed by a 4-h testing session under noise exposure and quiet conditions. Measures of performance, psychophysiology and subjective reactions were taken. The results showed that all three modes of explicit control of adaptive automation modes were able to attenuate the negative effects of noise. This was partly due to the fact that operators opted for higher levels of automation under noise. It also emerged that forced choice showed marginal advantages over the two other automation modes. Statement of Relevance: This work is relevant to the design of adaptive automation since it emphasises the need to consider the impact of work-related stressors during task completion. During the presence of stressors, different forms of operator support through automation may be required than under more favourable working conditions.

Adaptive Automation, Level of Automation, Allocation Authority, Supervisory Control, and Adaptive Control: Distinctions and Modes of Adaptation

This paper revisits several concepts and model frameworks that have been in the literature of human-machine interaction and control engineering for up to 50 years. The purposes of the revisit are as follows: 1) to sharpen distinctions between adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control engineering as the terms are currently used in the literature; 2) to define modes of human supervisory adaptation from the control engineering perspective; and 3) to suggest comparative taxonomies for adaptive automation in direct control and in supervisory control.

Adaptive Aiding of Human-Robot Teaming

In many emerging civilian and military operations, human operators are increasingly being tasked to supervise multiple robotic uninhabited vehicles (UVs) with the support of automation. As 100% automation reliability cannot be assured, it is important to understand the effects of automation imperfection on performance. In addition, adaptive aiding may help counter any adverse effects of static (fixed) automation. Using a high-fidelity multi-UV simulation involving both air and ground vehicles, two experiments examined the effects of automation reliability and adaptive automation on human-system performance with different levels of task load. In Experiment 1, participants performed a reconnaissance mission while assisted with an automatic target recognition (ATR) system whose reliability was low, medium, or high. Overall human-robot team performance was higher than with either human or ATR performance alone. In Experiment 2, participants performed a similar reconnaissance mission with no ATR, static automation, or with adaptive automation keyed to task load. Participant trust and self-confidence were higher and workload was lower for adaptive automation compared with the other conditions. The results show that human-robot teams can benefit from imperfect static automation even in high task load conditions and that adaptive automation can provide additional benefits in trust and workload.

Investigation of automation deployment in the main control room of nuclear power plants by using adaptive automation

AbstractRecently, with the progress of computer technology, the digitalization of human–system interface (HSI) in the main control room (MCR) of advanced nuclear power plants (NPPs) has become an important issue. A higher level of automation means that the computers execute the greater part of the task. The frequency, however, of automation‐induced system failures, including human errors, prompts us to more closely investigate the influences of automation. The objective of this study is to measure the effects of adaptive automation on the primary task performance, operator situation awareness (SA), and workload in an advanced MCR of an advanced NPP to reduce the number of failures and improve human performance. This study selects automatic mode and semi‐automatic mode to compose five different types of automation allocations. The experiment scenario is a reactor shutdown task. A personal computer transient analyzer (PCTran) is used to simulate the HSI of an MCR. Twelve participants are required to perform the experiment scenario on the automation allocation types. The dependent variables were the primary task performance; the subjective workload was measured using an NASA‐Task Load Index (NASA‐TLX) questionnaire; and three levels of SA were estimated by SA queries. Experimental results revealed that the automation allocations made up of higher ratio automatic modes led to superior primary task performance, lower operator workload, and lower frequency of human errors. In the high ratio automatic mode allocation, the operators have the lowest SA queries score. This study provides additional information for future digital HSI design in the MCRs of advanced NPPs. © 2011 Wiley Periodicals, Inc.

Individual differences in response to automation: The five factor model of personality.

This study examined the relationship of operator personality (Five Factor Model) and characteristics of the task and of adaptive automation (reliability and adaptiveness-whether the automation was well-matched to changes in task demand) to operator performance, workload, stress, and coping. This represents the first investigation of how the Five Factors relate to human response to automation. One-hundred-sixty-one college students experienced either 75% or 95% reliable automation provided with task loads of either two or four displays to be monitored. The task required threat detection in a simulated uninhabited ground vehicle (UGV) task. Task demand exerted the strongest influence on outcome variables. Automation characteristics did not directly impact workload or stress, but effects did emerge in the context of trait-task interactions that varied as a function of the dimension of workload and stress. The pattern of relationships of traits to dependent variables was generally moderated by at least one task factor. Neuroticism was related to poorer performance in some conditions, and all five traits were associated with at least one measure of workload and stress. Neuroticism generally predicted increased workload and stress and the other traits predicted decreased levels of these states. However, in the case of the relation of Extraversion and Agreeableness to Worry, Frustration, and avoidant coping, the direction of effects varied across task conditions. The results support incorporation of individual differences into automation design by identifying the relevant person characteristics and using the information to determine what functions to automate and the form and level of automation.

Development of multi-level adaptive control and scheduling solutions for shop-floor automation in reconfigurable manufacturing systems

The proposed work outlines the production system (re)configuration problem from the real-time system automation perspective. A multi-level approach to support the development of adaptive automation systems in reconfigurable manufacturing systems is introduced. Starting from the hardware resources selected to face the production demand, the developed approach enables the identification and modelling of automation tasks, as well as the design and implementation of the distributed self-adaptive control solutions based on IEC 61499. A deeper focus on a dynamic algorithm to schedule automation tasks over time is outlined. Finally, preliminary analyses are illustrated with reference to a robotic cell operating in a pilot assembly line.

Effects of steering demand on lane keeping behaviour, self-reports, and physiology. A simulator study

In this study a driving simulator was used to determine changes in mental effort in response to manipulations of steering demand. Changes in mental effort were assessed by using subjective effort ratings, physiology, and the standard deviation of the lateral position. Steering demand was increased by exposure to narrow lane widths and high density oncoming traffic while speed was fixed in all conditions to prevent a compensatory reaction. Results indicated that both steering demand factors influence mental effort expenditure and using multiple measures contributes to effort assessment. Application of these outcomes for adaptive automation is envisaged.

Adaptive Automation Effects on Operator Performance during a Reconnaissance Mission with an Unmanned Ground Vehicle

We simulated a generic military crew station and examined the workload and performance of robotics operators when interacting with a ground robot in the two modes of robotic autonomy, teleoperation or semi-autonomous. We examined the effect of autonomy and invocation strategies on performance. The operator had either full teleoperation (manual) or semiautonomy (static) regardless of task load. In a third condition, the robots autonomy changed based on task load (adaptive). The operator had to identify hostile targets during the mission and maintain situation awareness (SA) of his local environment and the overall mission via a SA map. Results showed that when task load increased from low to high, participants' SA performance was better in the adaptive and static automation conditions than the manual condition; their threat detection performance degradation was less in manual and adaptive than in the static condition. On the other hand, when task load shifted from high to low, threat detection performance was better in the adaptive than the other two conditions.

Guidance for Human-System Interfaces to Automatic Systems

Automation is ubiquitous in modern complex systems, and commercial nuclear- power plants are no exception. Automation is applied to a wide range of functions, including monitoring and detection, situation assessment, response planning, and response implementation. Automation has become a “team player” supporting personnel in nearly all aspects of system operation. In light of its increasing use and importance in new- and future-plants, guidance is needed to conduct safety reviews of the operator's interface with automation. The objective of this research was to develop such guidance. We first characterized the important HFE aspects of automation, including six dimensions: Levels, functions, processes, modes, flexibility, and reliability. Next, we reviewed literature on the effects of all of these aspects of automation on human performance, and on the design of human-system interfaces (HSIs). Then, we used this technical basis established from the literature to identify general principles for human-automation interaction and to develop review guidelines. The guidelines consist of the following seven topics: Automation displays, interaction and control, automation modes, automation levels, adaptive automation, error tolerance and failure management, and HSI integration.

Using Transcranial Doppler Sonography to Measure Cognitive Load in a Command and Control Task

Previous work has explored the possibility of measuring the functional state of the operator to drive the implementation of physiological adaptive aiding. Transcranial Doppler Sonography (TCD), which has shown promise as an index of cognitive resource utilization in vigilance or sustained attention tasks, may provide a time and cost efficient alternative to traditional measures used to assess operator functional state. In the current study, participants performed a command and control simulation under varying levels of task load: a low task load condition in which enemy threats incurred at a steady pace, and a high workload trial in which the number of enemy threats increased unpredictably at two points within the scenario. Reaction time to engage and destroy enemies, and the efficiency of protection of a no-fly zone, were superior in the low than in the high load condition. Furthermore, an automated decision aid facilitated better performance in both task load conditions. As the demands of the task increased unpredictably in the high task load condition, cerebral blood flow velocity (CBFV) increased in a similar manner for the first task load transition, but not for the second. Results suggest that the TCD measure may be useful in monitoring the dynamic changes of operator workload in unpredictable environments, but additional studies are needed to validate its use for physiologically-driven adaptive automation.

Comparison of Multiple Physiological Sensors to Classify Operator State in Adaptive Automation Systems

Automating tasks alleviates operator resources to be delegated to other demands, but the cost is often situation awareness. In contrast, complete manual control of a system opens the door for greater human error. Therefore, an ideal situation would require the development of an adaptive system in which automation can be triggered based on performance of a particular task, time spent on the task, or perhaps physiological response. The latter pertains to the goal for this particular study. Electroencephalogram (EEG), electrocardiogram (ECG), and eye tracking measures were recorded during six multi-tasking scenarios to assess if any one single measure is best suited for future implementation as an automation invocation. EEG showed the greatest potential for that purpose.

Adaptive Automation Based on an Object-Oriented Task Model: Implementation and Evaluation in a Realistic C2 Environment

Staffing reduction initiatives and more complicated military operations lead to a higher cognitive workload in command and control (C2) environments. Extending automation with adaptive capabilities can aid the human in overcoming cognitive workload challenges. At present, most adaptive automation research has focused on laboratory experiments and only limited research has aimed to implement and validate adaptive automation in a real-world setting. The objective of the present study was to investigate the effects of adaptive automation in precisely such a setting, extending the scientific knowledge base of adaptive systems with an evaluation of a real-world adaptive task. Implementing adaptive automation in a real-world C2 setting required extending current adaptive automation theories with an object-oriented task model and a hybrid triggering mechanism. The extended model was evaluated with eight naval officers using a high-fidelity C2 environment and showed an overall efficiency effect of 60%. Furthermore, no negative side effects of adaptive automation have been found, and the data show that the scenarios were manipulated correctly. In addition, the positive efficiency effects appear most strongly in the more complicated asymmetrical scenarios (65%). This latter conclusion shows that adaptive automation can be a valuable contribution to future C2 systems.

Human Supervisory Control Challenges in Network-Centric Operations

Network-centric operations (NCOs), envisioned for future command and control systems in military and civilian settings, must be supported by sophisticated automated systems, so human-computer interactions are an important aspect of overall system performance. This chapter identifies 10 human supervisory control challenges that could significantly impact operator performance in NCOs: information overload, attention allocation, decision biases, supervisory monitoring of operators, distributed decision making through team coordination, trust and reliability, the role of automation, adaptive automation, multimodal technologies, and accountability. NCOs will bring increases in the number of information sources, volume of information, and operational tempo with significant uncertainty, all of which will place higher cognitive demands on operators. Thus, it is critical that NCO research focuses not only on technological innovations but also on the strengths and limitations of human-automation interaction in a complex system.

Real-Time Adaptive Automation System Based on Identification of Operator Functional State in Simulated Process Control Operations

This paper proposes a new framework for the online monitoring and adaptive control of automation in complex and safety-critical human-machine systems using psychophysiological markers relating to humans under mental stress. The starting point of this framework relates to the assessment of the so-called operator functional state using psychophysiological measures. An adaptive fuzzy model linking heart-rate variability and task load index with the subjects' optimal performance has been elicited and validated offline via a series of experiments involving process control tasks simulated on an automation-enhanced Cabin Air Management System. The elicited model has been used as the basis for an online control system via the predictions of the system performance indicators corresponding to the operator stressful state. These indicators have been used by a fuzzy decision maker to modify the level of automation under which the system may operate. A real-time architecture has been developed as a platform for this approach. It has been validated in a series of human volunteer studies with promising improvement in performance.

A STEP compliant knowledge based schema to support shop-floor adaptive automation in dynamic manufacturing environments

On the factory shop-floor adaptive automation and scheduling plays a sensitive role for production systems’ agility, representing the soft layer between the factory physical resources and business processes. A fundamental requirement for adaptive shop-floor solutions is to incorporate a data structure that comprehensively integrates product, process and physical resource information. This paper presents a novel data structure entitled SKEM – STEP compliant Knowledge Engine for Manufacturing, conceived as an abstract, extensible, scalable, multi-granular, integrated and evolving platform. A software tool based on SKEM is tested through a case study to illustrate the benefits of its application for a dynamic manufacturing factory environment.

Adaptive Automation in a Naval Combat Management System

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> There is a continuing trend of letting fewer people deal with larger amounts of information in more complex situations using highly automated systems. In such circumstances, there is a risk that people are overwhelmed by information during intense periods or, on the other hand, do not build sufficient situational awareness during periods of slack to deal with situations where human intervention becomes necessary. A number of studies show encouraging results in increasing the efficiency of human–machine systems by making the automation adapt itself to the human needs. Current literature shows no examples of adaptive automation in real operational settings, however. We introduce a fine-grained adaptation methodology based on well-established concepts that is easy to comprehend and likely to be accepted by the end user. At the same time, we let the machine operate like a virtual team member in that it continuously builds its own view of the situation independent from the human. Working agreements between human and machine provide lower and upper bounds of automation that are in advance determined by the end user so that unwanted appropriation of responsibility by the machine is avoided. The framework is domain neutral and therefore thought to be applicable across a wide range of complex systems, both military and civilian. It gives researchers an architecture that they can use in their own work to get adaptive automation up and running quickly and easily. </para>

Current Concepts and Trends in Human-Automation Interaction

The purpose of this panel was to provide a general overview and discussion of some of the most current and controversial concepts and trends in human-automation interaction. The panel was composed of eight researchers and practitioners. The panelists are well-known experts in the area and offered differing views on a variety of different human-automation topics. The range of concepts and trends discussed in this panel include: general taxonomies regarding stages and levels of automation and function allocation, individualized adaptive automation, automation-induced complacency, economic rationality and the use of automation, the potential utility of false alarms, the influence of different types of false alarms on trust and reliance, and a system-wide theory of trust in multiple automated aids.

The Effects of the Adaptability and Reliability of Automation on Performance, Stress and Workload

One way in which non-human agents manifest in complex systems is through the use of adaptive automation. In this case, the degree of involvement of a computer-based agent can be controlled dynamically as a function of environmental conditions and operator state. The current study examined the effects of specific aspects of adaptive automation, reliability and adaptability (the ability of the system to adapt appropriately to changes in task demand), on human performance, stress, and workload.