Time Delay Mitigation in Aerial Telerobotic Operations Using Heterogeneous Stereo-Vision Systems

Abstract

This paper investigates the use of a heterogeneous stereo-vision system to mitigate the effects of time delays in a drone-based visual interface presented to a human operator. Time delays in the display for a telerobotic interface refer to the time difference between the operator’s input action and the corresponding visible outcome. In human/machine interfaces, time delays can arise due to computation, telecommunication, and mechanical limitations. These delays can degrade the performance of the human/machine system. A heterogeneous stereo-vision predictive algorithm is presented that can reduce the negative effects of time delays in the operator’s display. The heterogeneous stereo-vision system consists of an omnidirectional camera and a pan/tilt/zoom camera. Two predictive display setups were developed that modify the delayed video imagery that would otherwise be presented to the operator in a way that provides an almost immediate visual response to the operator’s control actions. The usability of the system is determined through human performance testing with and without the predictive algorithms. The results indicate that the predictive algorithm allows more efficient, accurate, and user-friendly operation.