Levels of automation in a simulated failure detection task

Abstract

Systems increasingly provide operators with the ability to move between different levels of automation from entirely manual modes, over partly automated modes (in which some task components are performed by the operator) and up to fully automated modes. It is still unclear what determines the relative efficiency of different levels of automation. An experimental system was developed to study these issues. In the system operators had to detect faulty items, based on partial information. Two levels of automation were tested: aided detection (operators received cues from a failure detection system), and approval (operators could change decisions of the failure detection system). An experiment assessed operators' performance with the different levels of automation and with low and high validity failure detection systems. Results showed that operators adjusted their response to the diagnostic value of the automation. Also, operators responded more strongly to low-validity cues in the approval condition than in the aided detection condition. These results point to some of the complex issues that need to be considered when choosing the appropriate level of automation for a system.