Integrated MPCAM: Multi-PSF learning for large depth-of-field computational imaging

Abstract

Large DOF (depth-of-field) imaging with high SNR (signal-noise-ratio) is useful for applications such as machine vision and medical imaging. In traditional optical systems, DOF extension is always implemented at the cost of SNR. In this paper, we present a MPCAM (Multi-PSF Camera) system highly integrated with AF (auto-focus) function to realize both large DOF and high SNR imaging. MPCAM based on MPGAN (Multi-PSF Generative Adversarial Network) is first proposed to automatically extract multiple PSFs (point spread functions) and realize high fidelity image reconstruction by features fusion. The proposed end-to-end generative image fusion network is flexible and can be designed with different input dimensions for a given AF application, which is vital to circumvent the trade-off between DOF and SNR. We build a dataset containing 5000 raw images tailored to the proposed network by an off-the-shelf camera. Results show that our MPCAM system can produce images with average higher values than raw images over 4.625, and 0.061 in PNSR (peak signal to noise ratio), and SSIM (structure similarity) metrics, respectively. Moreover, compared to the classic and latest image fusion methods, the results also verify that our method has achieved comparable or even better performance. Due to its advance in high SNR and large DOF imaging, this novel, portable and inexpensive system is suitable for computational applications such as microscopic pathological diagnosis, domain-specific computational imaging and smartphone photography. The implementation code of MPGAN and dataset are available from https://www.kaggle.com/datasets/ktd970903/multi-psf-camera.