<title>Three-dimensional geometrical model building for visual robot control</title>

Abstract

This paper addresses an approach to the problem of determining the 3D location of points of an object in the environment of a moving camera mounted on a robot arm, based on a monocular image sequence obtained by the camera. These points can be either endpoints of the line segments or other feature points. The robot arm's velocity and position are assumed to be known via the robot arm controller. The motion model of the camera incorporates the robot arm dynamics. The resulting model is a linear time-varying one. This model overcomes the common assumption of a constant velocity camera motion between consecutive image frames. The motion of the 3D points in the camera reference frame is maintained by tracking between frames. This is done recursively using the extended Kalman filter (EKF). The 3D motion stereo equations which are derived serve as the measurement model for the corresponding EKF without the need to solve them explicitly. The resulting measurement equations are linear time-varying ones with multiplicative noise. The 3D location of points of the selected object are then updated recursively using the EKF in conjunction with different views of the object. These models are particularly suitable for the EKF implementation. Correspondence between two 2D image points in consecutive frames of the same 3D scene point is constrained by statistical distance produced by the EKF. Simulation results are presented to illustrate the approach.