Real-Time Light Field Denoising Using a Novel Linear 4-D Hyperfan Filter

Abstract

Four-dimensional (4-D) light fields (LFs) enable novel imaging technologies, which are traditionally based on two-dimensional images. In most of these applications, denoising of LFs is required as a preprocessing technique before any subsequent processing. We propose a real-time LF denoising method using a novel 4-D linear and shift-invariant hyperfan filter. The proposed method exploits sparsity of the spectrum of a LF and the 4-D hyperfan filter is implemented in the 4-D mixed-domain (i.e.,two-dimensional space and two-dimensional frequency) leading to significant reductions in computational and memory complexities. A software implementation of the proposed method provides better or comparable denoising performance for grayscale and color LFs with respect to the metrics peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) compared to previously reported linear LF denoising methods, while reducing the processing time approximately by 66% and 31% for grayscale and color LFs, respectively. Furthermore, we propose a semi-systolic hardware architecture for the proposed denoising method, and implement on a field-programmable gate array (FPGA). The FPGA implementation implies a throughput of 25 LFs/s for LFs of size 11×11×625×434 and provides approximately 13 dB improvement in PSNR and 0.7 improvement in SSIM for grayscale LFs verifying the suitability for real-time processing.