Three-dimensional integral imaging in photon-starved environments with high-sensitivity image sensors.

Abstract

Imaging in poorly illuminated environments using three-dimensional (3D) imaging with passive imaging sensors that operate in the visible spectrum is a formidable task due to the low number of photons detected. 3D integral imaging, which integrates multiple two-dimensional perspectives, is expected to perform well in the presence of noise, as well as statistical fluctuation in the detected number of photons. In this paper, we present an investigation of 3D integral imaging in low-light-level conditions, where as low as a few photons and as high as several tens of photons are detected on average per pixel. In the experimental verification, we use an electron multiplying charge-coupled device (EM-CCD) and a scientific complementary metal-oxide-semiconductor (sCMOS) camera. For the EM-CCD, a theoretical model for the probability distribution of the pixel values is derived, then fitted with the experimental data to determine the camera parameters. Likewise, pixelwise calibration is performed on the sCMOS to determine the camera parameters for further analysis. Theoretical derivation of the expected signal-to-noise-ratio is provided for each image sensor and corroborated by the experimental findings. Further comparison between the cameras includes analysis of the contrast-to-noise ratio (CNR) as well as the perception-based image quality estimator (PIQE). Improvement of image quality metrics in the 3D reconstructed images is successfully confirmed compared with those of the 2D images. To the best of our knowledge, this is the first experimental report of low-light-level 3D integral imaging with as little as a few photons detected per pixel on average to improve scene visualization including occlusion removal from the scene.