<title>Color-feature-based environmental synchronization in VR-aided teleoperation</title>

Abstract

In teleoperation situations where virtual environments are employed and fine operation is needed, it is crucial to dynamically keep the virtual environment consistent with the real remote environment. This is especially important when the remote site is at a great distance, such as in a space station, and therefore large time delays exist during the process of teleoperation. In this paper, we propose an automatic calibration method which dynamically determines the difference in 3D position and orientation between virtual and real environments by using a new color image matching technique which is based on gradients of both gray levels and color information. During the process of model building, significant color features in the real environment, either natural or specially prepared, are picked up and mapped onto the corresponding environment model positions. During the process of teleoperation, color images are taken by a camera mounted on a manipulator. These images are analyzed and features are extracted and matched with those in the model in real time. The 3D poses and positions of the camera in the real environment are calculated and then compared with those in the virtual environment in order to determine differences between them. Feature correspondences are determined based on color attributes and geometric relations. A simplified closed-form solution for 3D location of a 4-DOF mobile camera is given. Experimental results show the effectiveness of this dynamic calibration approach.