Stereoscopic First Person View System for Drone Navigation

Abstract

Ground control of unmanned aerial vehicles (UAV) is key to the advancement of this technology for commercial purposes. Not only because handover situations will require humans to take control over the drones when autonomous systems fail. But also, because training deep neural network based control systems require extensive real world data, and capturing meaningful and diverse data is a bottleneck. This axiom is even more prominent for the case of unmanned flying robots where there is no simple solution to capture optimal navigation footage. In such scenarios, improving the ground control and developing better autonomous systems are two sides of the same coin. To improve the ground control experience, and thus the quality of the footage, we propose to upgrade on-boad teleoperation systems to a fully immersive setup that provides operators with a stereoscopic First Person View through a Virtual Reality (VR) Head Mounted Display (HMD). We tested users (n=7) by asking them to fly our drone on the field. Test flights showed that operators flying our system can take off, fly and land successfully while wearing VR headsets. Additionally, we ran two experiments with pre-recorded videos of the flights and walks to a wider set of participants (n=69 and n=20) to compare the proposed technology to the experience provided by current drone FPV solutions that only include monoscopic vision. Our immersive stereoscopic setup enables higher accuracy depth perception, which has clear implications for achieving better teleoperation and unmanned navigation. Our studies show comprehensive data on the impact of motion and simulator sickness in case of stereoscopic setup. We present the device specifications as well as the measures that improve teleoperation experience and reduce induced simulator sickness. Our approach provides higher perception fidelity during flights, which leads to a more precise better teleoperation and ultimately translates into better flight data for training deep UAV control policies.