The effects of poor automation design

Abstract

A significant degree of the effort from cognitive ergonomists in the early stages of design and development is concerned with reducing workload during the operation and supervision of systems and equipment. There is a implicit belief that reductions in the workload attached to specific areas will result in improved performance in other areas. Rasmussen (1999) has only recently argued that adjustments to tasks which decrease workload can paradoxically result in reduced performance because operators are subject to cognitive disengagement. It has been argued that operators can become disengaged from the task or they are more willing to take significantly greater risks as they perceive themselves to have greater control over the process they are managing. It is likely that automation which is introduced to reduce workload will act as a catalyst for such risk-taking behaviour because it serves to buffer the operator from any entropic elements of the environment in many pre-conceived areas of operation. Unusual events and highly dynamic incidents, which overwhelm or overtake the automation will create a sense of panic or extreme anxiety because the operator will have reduced system management ability, as a result of inadvertent de-skilling. The automation may prevent or encourage failures in early event detection, or negative process trends, because the identifiable features characteristic of such events are not displayed or their visible significance is moderated by automation intervention. In addition, the automation of many systems will contribute to reduced cue awareness related to developing failures and this will result in one of the most commonly reported problems of mis-identification the incident type, which in turn undermines the management of the event. Thus, the operator will experience unusual events as appearing out of the blue, so called de-compensation problems, and will struggle frantically to identify the causes when they do occur. It is argued that operators can have significantly better performance in systems which manage workload with contextual and temporal sensitivity. Paradoxically increases and decreases in the level of interaction can produce positive performance benefits. This approach is illustrated with the outline of a new pilot's assistant for use in a simulation of an attack helicopter. A major thrust of the work is that automation induces two major types of cognitive errors. First, the poorly designed automation results in interruption errors because the operator is forced to task-switch at inappropriate times in response to automation and agent-based requests for process management. Secondly, the more serious errors are with regard to prospective memory because the operator does not receive cues that refresh the intention to act with regard to future process interventions. This results in serious errors of omission and commission because operator's natural flow of behaviour is disrupted. Such errors are well documented in the current literature regarding automation.