A Stereo Vision-Based Support System for Tele-Operation of Unmanned Vehicle

Abstract

This research provides a stereoscopic vision system on an unmanned vehicle system (UVS) for disaster inspection. Taiwan is a disaster-prone area where earthquakes, typhoon, floods and debris flow happen frequently. These disasters usually cause terrain transformation and building collapse that raise the risk for human inspection. However, authorities need instant onsite data right after disaster happened to deploy appropriate rescue missions and distribute resources. Instead of sending bunch of sensor into the field with unmanned vehicles and retrieve a huge amount of numeric data for time-consuming post analysis, an instinct and cognitive method for quick understanding and reconstruction of onsite situation is needed. Therefore, we developed a stereoscopic vision system which can be easily integrated with all UVS. In order to enhance the cognition of the operator, we applied two methods: (1) stereoscopic vision for head mount display, (2) head tracking data for gimbal control. We used two camera sources to simulate human vision and optimized the perspective to suit human eyes’ field-of-view (FOV). By displaying the optimized image on the head mount display, the operator would have a realistic first-person view of the UVS. We also developed control logic for 3-axis gimbals that can interpret the head tracking data into gimbal motion so operator can move camera’s direction by turning heads. The integration of these two methods provides a enhanced visual aid that can be adapted to almost every UVS nowadays. In the future research, we will deploy the proposed system on common UVS to validate the improvement of human cognition.