Low-light images enhancement and denoising network based on unsupervised learning multi-stream feature modeling

Abstract

Inspired by the generative adversarial networks EnlightenGAN, this paper proposes a novel low-light images Enhancement and Denoising model based on unsupervised learning Multi-Stream feature modeling (MSED). The model has two stages: generator network and discriminator network. Generator network includes global and local feature modeling network. First, Swin Transformer Block is innovatively introduced in the global feature modeling of generator network stage. It makes the interaction between the image and the convolutional kernel related to the image content. Its shift window mechanism can model the global feature dependency of the input image with less memory consumption, and extract the color, texture, and shape features of the image, thereby effectively suppressing noise and artifacts. Second, in the local feature modeling, a multi-scale image and feature fusion branch is added. It not only extracts reduced features from large-scale low-light images, but also extracts features from multiple downsampled low-light images, and then combines the two through attention and DSFF. By utilizing the complementary information of the reduced features and downsampled images, various underexposure/ overexposure phenomena caused by low-light images can be effectively avoided. In the discriminator network stage, the deep/shallow feature aggregation module is added to enhance the discrimination ability, and the inconsistencies are suppressed by learning the contradictory information of spatial filter, so that the shallow representation of information and the deep semantic information can guide each other. Thanks to the synergy of the above three innovative work, compared with many existing advanced low-light images enhancement models, MSED can achieve SOTA level performance on several public datasets. However, when dealing with low-light images with blurry content caused by rapid motion, MSED still cannot effectively restore their detailed information.