Physics-based Efficient Full Projector Compensation using only Natural Images.

Abstract

Achieving practical full projector compensation requires the projection display to adapt quickly to textured projection surfaces and unexpected movements without interrupting the display procedure. A possible solution to achieve this involves using a projector and an RGB camera and correcting both color and geometry by directly capturing and analyzing the projected natural image content, without the need for additional patterns. In this study, we approach full projector compensation as a numerical optimization problem and present a physics-based framework that can handle both geometric calibration and radiometric compensation for a Projector-camera system (Procams), using only a few sampling natural images. Within the framework, we decouple and estimate the Procams' factors, such as the response function of the projector, the correspondence between the projector and camera, and the reflectance of projection surfaces. This approach provides an interpretable and flexible solution to adapt to the changes in geometry and reflectance caused by movements. Benefitting from the physics-based scheme, our method guarantees both accurate color calculation and efficient movement and reflectance estimation. Our experimental results demonstrate that our method surpasses other state-of-the-art end-to-end full projector compensation methods, with superior image quality, reduced computational time, lower memory consumption, greater geometric accuracy, and a more compact network architecture. The data and source code are accessible at https://github.com/kylin-leo/FullProjectorCompensation.