Characterizing User Responses to Failures in Aerial Autonomous Systems

Abstract

Users are often the last barrier in the detection and correction of abnormal behavior in autonomous systems, so understanding what can be expected from users in various contexts is crucial to the performance of such systems. This letter presents the first study characterizing a user ability to timely report and correct autonomous system failures in the context of small Unmanned Aerial Vehicles (sUAVs). The study aims to explore the complex tradespace which designers will encounter when developing effective transitions of control in sUAVs. We have analyzed these tradeoffs in terms of accuracy and response time while manipulating several key contextual elements including subtlety of failure, transitions of control, introduction of noise in autonomous paths, and amount of user training. Results indicate that: 1) increased accuracy is achieved under longer deadlines without large delays in responses, 2) increased noise, user training, and user responsibility for correction lead to increased reporting times, but only increased user responsibility increased accuracy, 3) users, particularly those with additional training, wanted to remain engaged in failure correction even when such interactions were not requested, 4) asking users to fix failures that they did not report resulted in increased response times and reduced accuracy.