Simultaneous Calibration of Multiple Revolute Joints for Articulated Vision Systems via SE(3) Kinematic Bundle Adjustment

Abstract

We propose a vision-based approach to calibrate kinematic structure of low degree-of-freedom (DoF) articulated systems. Standard hand-eye calibration yields excellent eye-to-hand relations by explicitly estimating end-effector mounted camera poses from the Perspective-n-Point (PnP) problem of a single calibration rig. However, these methods struggle when the ideal kinematic model is unknown or inaccurate, which are typical in customized serial chains such as articulated vision systems. Inspired by bundle adjustment in structure-from-motion methods, we proposed an approach, dubbed KBA, to simultaneously localize multiple joint axes at the kinematic level instead of in the 3D space. To achieve this, we design a robust multi-checkerboard detector to enlarge the kinematic coverage of end-effector samples, which are usually limited by sensor field of view. The overall optimization problem is formulated and solved using Lie group SE(3)—suitable for integration into simultaneous localization and mapping systems. In addition to simulated scenes, the experimental results with articulated monocular (AMV) and binocular vision (ABV) confirm that the proposed method not only is applicable to real serial kinematic chains but also enables challenging 3D vision tasks such as stereo reconstruction in motion.