The importance of real-time visual information for the remote supervision of an autonomous agricultural machine

Abstract

“Supervised autonomy” is a term that best describes the autonomous agricultural machines currently being envisioned. Such machines will be able to work autonomously, but require human supervision. The topic of interface design, however, has not received sufficient attention by designers. The goal of this study was to investigate the importance of live video for remote supervision of autonomous agricultural machines. The study was conducted using an autonomous agricultural machine control interface simulator, which provided information of machine status using graphical indicators (which interpreted and displayed sensor information) and live video (which was presented as seen by the camera). The participants of the study performed the role of the supervisor of an autonomous agricultural machine. The importance of live video was assessed by comparing the participant’s performance during trials with and without video. Information about the gaze direction was obtained with an eye-tracking system. The results showed that graphical indicators are the preferred source of machine status information, and live video is a secondary source. At the beginning of the experiment, participants split their attention evenly between the graphical indicators and the live video, but by the end of the experiment, their focus was on the graphical indicators 70% of the time. More than 75% of the participants indicated that the live video helped them to understand machine functions better and they felt more secure when the video footage was present. The participants suggested that live video should be available either full time or on demand. Control interface designers should consider including live video on the interface for autonomous agricultural machines to provide additional decision-making support to the supervisor.