Stereo Vision System for Remotely Operated Robots

Abstract

Remotely operated robots, functioning in hazardous and time critical environments, have significant requirements for control and visual information (Davids (2002), Murphy (2004)). The control systems are supposed to guarantee a precise timely response in order to prevent fatal scenarios in bomb disposal operations or in life rescue missions. Significant role to these operating scenarios play the concurrent visual information provided to the remote operators by the on-board mounted cameras. Visual information (Fong & Thorpe (2001), Desouza & Kak (2002)) is often displayed in one or more monitors depending on the number of on-board mounted cameras. In sophisticated and multi-tasking robots more than one operators are performing certain actions. Especially, in case of robots with grippers and robotic arms, one operator might be dedicated only with the maneuvering and controlling of the robotic arms or grippers. In these operation scenarios, the dedicated user must be focused only on this task and furthermore should have the best visual understanding of the working field. The widely used equipment and gear for these assignments is a Head Mounted Display (HMD) and an attached head tracker. The HMD projects visual feedback of the remote robot in front of operator eyes. A single camera feedback projection in both eyes is not so significant since the result in operator’s perception is the same as being watched from a single monitor. Thus, a pair of cameras are used instead, in order to provide a real stereo feedback to the operator’s HMD, thus enhancing his visual perception and improving the sense of depth (Willemsen et al. (2004)). Consequently, operators can judge situations and perform actions more efficiently based on the qualitative information of the synchronized stereo video streams. The use of a head tracker expands the operator’s functions while it offers a hands-free ability to remotely control the pose of the robot head. The inertial measurement devices used for the head tracking usually contain rate gyroscopes (gyros) and accelerometers. The measurements of the inertial sensor can be processed and transmitted as control signals to the remote robot. Many different interfaces have been presented in literature recently. Amethod of robot teleoperation that allows a human operator to control a robot manipulator is presented in (Kofman et al. (2005)). It uses a non-contacting, vision-based, human-robot interface for both the communication of the human motion to the robot and for feedback of the robot motion to the human operator. However, this visual feedback does not give the operator the depth visual information, which is necessary for this critical task. In (Bluethmann et al. (2003)), a sophisticated anthropomorphic robot is developed for space operations. It is comprised of a stereo 4