Lightweight Neural Network for Enhancing Imaging Performance of Under-Display Camera

Abstract

Under-Display Camera (UDC) is an emerging feature of cellphone. This technology makes full-screen cellphones possible by hiding the front-facing camera below the display panel, which is in contrast to the conventional designs that place the camera in a bezel or punch-hole on the screen border. However, this novel imaging paradigm also causes degradation. The display panel attenuates and diffracts incoming light, so the images captured by UDC contain multiple artifacts, such as blurring, color shift, and low intensities. This paper proposes a lightweight deep learning approach to restore UDC images in a blind setting. The restoration network uses cross-scale modulation to exploit complementary information from multi-scale representations and capture the self-similarity across scales, aiming to find the cues for recovering distortion-free images. To facilitate the deployment of this scheme across mobile devices, especially on those with limited memory space and computing power, we compress the restoration network by reducing architectural redundancy. An adaptive distillation algorithm is designed to exploit knowledge from a pre-trained full-size model. The proposed work also interprets the behavior of the neural network in utilizing local and non-local information to restore UDC images. The proposed algorithm is evaluated over three datasets of the images captured by the cameras below different types of display panels. The results of comparative experiments demonstrate that our algorithm shows comparable or superior performance to the competing ones that are much heavier in parameter amount and computational complexities.