Predicted Failure Alerting in a Supervisory Control Task Does Not Always Enhance Performance

Abstract

Many emerging technologies mandate supervisory control of automation, with operators monitoring and intervening within systems that change dynamically over time. Providing decision aiding to these operators at critical moments has the potential to improve performance. The current study employed a supervisory control task to examine the effects of supplying a secondary aid (an alert to a predicted automation failure) to task performance. The aid signified potential anomalies in automated planning, and was either present throughout training, or only after some task experience was obtained. The aid occasionally changed operator decision-making, but did not consistently improve task performance. Crucially, the presence of the aid did not improve operators’ abilities to reject bad automated plans. Overall these results highlight a critical issue for the development of detection systems to effectively support future supervisory control activities, and the implications are discussed.