Ray geometry in non-pinhole cameras: a survey

Abstract

A pinhole camera collects rays passing through a common 3D point and its image resembles what would be seen by human eyes. In contrast, a non-pinhole (multi-perspective) camera combines rays collected by different viewpoints. Despite their incongruity of view, their images are able to preserve spatial coherence and can depict, within a single context, details of a scene that are simultaneously inaccessible from a single view, yet easily interpretable by a viewer. In this paper, we thoroughly discuss the design, modeling, and implementation of a broad class of non-pinhole cameras and their applications in computer graphics and vision. These include mathematical (conceptual) camera models such as the General Linear Cameras and real non-pinhole cameras such as catadioptric cameras and projectors. A unique component of this paper is a ray geometry analysis that uniformly models these non-pinhole cameras as manifolds of rays and ray constraints. We also model the thin lens as a ray transform and study how ray geometry is changed by the thin lens for studying distortions and defocusing. We hope to provide mathematical fundamentals to satisfy computer vision researchers as well as tools and algorithms to aid computer graphics and optical engineering researchers.