Joint radiometric calibration and feature tracking system with an application to stereo

Abstract

To capture the full brightness range of natural scenes, cameras automatically adjust the exposure value which causes the brightness of scene points to change from frame to frame. Given such a video sequence, we introduce a system for tracking features and estimating the radiometric response function of the camera and the exposure difference between frames simultaneously. We model the global and nonlinear process that is responsible for the changes in image brightness rather than adapting to the changes locally and linearly which makes our tracking more robust to the change in brightness. We apply our system to perform structure-from-motion and stereo to reconstruct a texture-mapped 3D surface from a video taken in a high dynamic range environment.