Caliber: Camera Localization and Calibration Using Rigidity Constraints

Abstract

This article presents a camera calibration system, Caliber, and the underlying pose estimation problem it solves, which we call sensor localization with rigidity (SL-R). SL-R is a constraint-satisfaction-like problem that finds a set of poses satisfying certain constraints. These constraints include not only relative pose constraints such as those found in SLAM and motion estimation problems, but also rigidity constraints: the notion of objects that are rigidly attached to each other so that their relative pose is fixed over time even if that pose is not known a priori. We show that SL-R is NP-hard, but give an inference-based algorithm that works well in practice. SL-R enables Caliber, a tool to calibrate systems of cameras connected by rigid or actuated links, using image observations and information about known motions of the system. The user provides a model of the system in the form of a kinematic tree, and Caliber uses our SL-R algorithm to generate an estimate for the rigidity constraints, then performs nonlinear optimization to produce a solution that is locally least-squares optimal in terms of reprojection error. In this way, Caliber is able to calibrate a variety of setups that would have previously required special-purpose code to calibrate. We demonstrate Caliber in a number of different scenarios using both synthetic and experimental data.