Image-based visual servoing with depth estimation

Abstract

For the depth estimation problem in the image-based visual servoing (IBVS) control, this paper proposes a new observer structure based on Kalman filter (KF) to recover the feature depth in real time. First, according to the number of states, two different mathematical models of the system are established. The first one is to extract the depth information from the Jacobian matrix as the state vector of the system. The other is to use the depth information and the coordinate point information of the two-dimensional image plane as the state vector of the system. The KF is used to estimate the unknown depth information of the system in real time. And an IBVS controller gain adjustment method for 6-degree-of-freedom (6-DOF) manipulator is obtained using fuzzy controller. This method can obtain the gain matrix by taking the depth and error information as the input of the fuzzy controller. Compared with the existing works, the proposed observer has less redundant motion while solving the Jacobian matrix depth estimation problem. At the same time, it will also be beneficial to reducing the time for the camera to reach the target. Conclusively, the experimental results of the 6-DOF robot with eye-in-hand configuration demonstrate the effectiveness and practicability of the proposed method.