Stereo Vision and Its Application to Robotic Manipulation

Abstract

A robot is expected to provide a service to us in our daily-life environment. Thus, it is easy to find robots that can achieve a task, such as home cleaning performed by the iRobot Roomba1 or entertainment provided by the Sony Aibo. Unlike the situation in a plant, the daily-life environment changes sequentially. Before performing a task, it is necessary to observe the environment. Since the motion and manipulation done by a robot occur in a 3D world, gathering 3D information and not 2D information is inevitable. For example, such information helps us achieve semi-automatic robot programming (Ikeuchi & Suehiro (1994); Kuniyoshi et al. (1994)). There are many kinds of devices to obtain 3D information. They are roughly classified into two types: active sensors and passive sensors. What distinguishes these two types is the capacity of the sensor to output energy (e.g., emit light) to the outer world. As an example of active sensors, a laser sensor measures the distance using the duration since the light is emitted until it captures the reflected light. Among the passive sensors, stereo vision is the simplest device to obtain 3D information. A stereo vision system employs at least two separate imaging devices, as in the case of human vision. Generally, the active sensors are better in accuracy than the passive sensors2. Although this may mean that human stereo vision suffers from inaccuracy of the obtained 3D information, we unconsciously employ prior knowledge to compensate for this. Les us consider estimation of a 6-DOF object trajectory, which is required in various kinds of robotic manipulation, such as pick-and-place and assembly. To reduce the inaccuracy in the estimation, some constraints about the trajectory are necessary. For example, if it is previously known that the trajectory is a straight line, we reduce the inaccuracy by minimally deforming the trajectory to align it to the line. We will introduce prior knowledge into an actual robot application. In this chapter, we will first present an overview of the stereo vision system and a method for localization using 3D data (Section 2). We will describe a method for using the contact relation (Section 3) as prior knowledge; in real world, rigid objects do not penetrate each other. Also,