A Practical Extrinsic calibration method for joint depth and color sensors

Abstract

In recent years, with the development of navigation applications, depth and color sensor pairs have been widely used in many onboard systems of vision and robotics, for example, Automated Guided Vehicle (AGV). To effectively use the 3D data from the depth sensor and 2D data from the color sensor, extrinsic calibration is a fundamental problem. However, the existing calibration methods are mostly target-based or complicated, which could not fit onboard systems well. A flexible and practical approach is still expected. This paper presents an easy-to-use calibration method based on the line of two intersecting planes (e.g., floor and wall), which can be easily found in artificial environments. Make a planar motion of a set of depth and color sensors on the floor. Since the set's motion is relative to the line, we could consider the set static and the line to be moving with reference to the set. Lots of coplanar intersection points of the lines can be obtained from the depth sensor (3D) and color sensor (2D) together. Using these 3D-2D corresponding point pairs, a homography relationship about the floor between the depth sensor and color sensor can be computed. The extrinsic parameters between these two sensors can be solved in a closed-form. Our method is verified on both simulated and real data, and the experiments show that the method can perform well in terms of robustness and accuracy.