A situational understanding enhancer based on augmented visual prompts for teleoperation using a multi-monitor system

Abstract

This study proposes an operator support system that enhances understandability of multiple monitor for teleoperated disaster response robots. The autonomous environmental camera control system we previously developed could provide operators with various visual information suited to the situation. To enable operators to adequately recognize environmental situations, multiple camera images are projected on multiple monitors, but this may increase the cognitive load of operators. The proposed situational understanding enhancer thus supports selective visual attention, situational awareness, and action decision, based on augmented visual prompts (AVPs). To support both work facilitation and danger avoidance, the AVPs include (i) monitor frame, (ii) obscuration, (iii) end-point vertical arrow, and (iv) distance arrow, with variable color and flashing effect. The system adaptively draws visual attention by changing the types and parameters of the AVPs according to the situations. Debris-removal tasks were conducted using a virtual reality simulator to compare the systems with and without AVP support. The results showed that the AVP support reduced the frequent changes in point of gaze, provided a sense of depth, and drew attention to potential collisions.