Normal-light Image Research Articles

UPT-Flow: Multi-scale transformer-guided normalizing flow for low-light image enhancement

Low-light images often suffer from information loss and RGB value degradation due to extremely low or nonuniform lighting conditions. Many existing methods primarily focus on optimizing the appearance distance between the enhanced image and the normal-light image, while neglecting the explicit modeling of information loss regions or incorrect information points in low-light images. To address this, this paper proposes an Unbalanced Points-guided multi-scale Transformer-based conditional normalizing Flow (UPT-Flow) for low-light image enhancement. We design an unbalanced point map prior based on the differences in the proportion of RGB values for each pixel in the image, which is used to modify traditional self-attention and mitigate the negative effects of areas with information distortion in the attention calculation. The Multi-Scale Transformer (MSFormer) is composed of several global-local transformer blocks, which encode rich global contextual information and local fine-grained details for conditional normalizing flow. In the invertible network of flow, we design cross-coupling conditional affine layers based on channel and spatial attention, enhancing the expressive power of a single flow step. Without bells and whistles, extensive experiments on low-light image enhancement, night traffic monitoring enhancement, low-light object detection, and nighttime image segmentation have demonstrated that our proposed method achieves state-of-the-art performance across a variety of real-world scenes. The code and pre-trained models will be available at https://github.com/NJUPT-IPR-XuLintao/UPT-Flow.

BGFlow: Brightness-guided normalizing flow for low-light image enhancement

Low-light image enhancement poses significant challenges due to its ill-posed nature. Recently, deep learning-based methods have attempted to establish a unified mapping relationship between normal-light images and their low-light versions but frequently struggle to capture the intricate variations in brightness conditions. As a result, these methods often suffer from overexposure, underexposure, amplified noise, and distorted colors. To tackle these issues, we propose a brightness-guided normalizing flow framework, dubbed BGFlow, for low-light image enhancement. Specifically, we recognize that low-frequency sub-bands in the wavelet domain carry significant brightness information. To effectively capture the intricate variations in brightness within an image, we design a transformer-based multi-scale wavelet-domain encoder to extract brightness information from the multi-scale features of the low-frequency sub-bands. The extracted brightness feature maps, at different scales, are then injected into the brightness-guided affine coupling layer to guide the training of the conditional normalizing flow module. Extensive experimental evaluations demonstrate the superiority of BGFlow over existing deep learning-based approaches in both qualitative and quantitative assessments. Moreover, we also showcase the exceptional performance of BGFlow on the underwater image enhancement task.

Degrade for upgrade: Learning degradation representations for real-world low-light image enhancement

Most existing low-light image enhancement approaches predominantly focus on directly modeling the mapping between low-light images and normal-light images, but models trained on the single fixed data set often fail to achieve satisfactory results in complex real-world scenes. To address the limitations posed by limited data, this paper proposes a joint framework to learn spatially adaptive degradation and enhancement representations of bright-to-dark images and dark-to-bright images. Firstly, we design an illumination degradation model based on cycle consistent loss to learn how to degrade normal-light images into low-light reference images. The degradation model can synthesize a large number of normal/degraded-light pairs, which is beneficial for the training of the second-stage enhancement model. Subsequently, leveraging the self-luminance information of images, we construct a self-attention-based light enhancement model. The enhancement model is trained on mixed data, which combines both real normal/low-light image pairs and synthetic normal/degraded-light image pairs. In this way, the enhancement model can learn more features from a diverse set of images, enhancing its robustness. Extensive experiments on low-light image enhancement, low-light object detection, and nighttime image segmentation have demonstrated that our proposed method outperforms existing supervised and unsupervised methods across a variety of real-world scenes.

Illumination-Aware Low-Light Image Enhancement with Transformer and Auto-Knee Curve

Images captured under low-light conditions suffer from several combined degradation factors, including low brightness, low contrast, noise, and color bias. Many learning-based techniques attempt to learn the low-to-clear mapping between low-light and normal-light images. However, they often fall short when applied to low-light images taken in wide-contrast scenes because uneven illumination brings illumination-varying noise and the enhanced images are easily over-saturated in highlight areas. In this article, we present a novel two-stage method to tackle the problem of uneven illumination distribution in low-light images. Under the assumption that noise varies with illumination, we design an illumination-aware transformer network for the first stage of image restoration. In this stage, we introduce the Illumination-aware Attention Block featured with Illumination-aware Multi-head Self-attention, which incorporates different scales of illumination features to guide the attention module, thereby enhancing the denoising and reconstruction capabilities of the restoration network. In the second stage, we innovatively introduce a cubic auto-knee curve transfer with a global parameter predictor to alleviate the over-exposure caused by uneven illumination. We also adopt a white balance correction module to address color bias issues at this stage. Extensive experiments on various benchmarks demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively.

LELD: Learn enhancement by learning degradation

Enhancing low-light images improves both the visibility and quality of the images. Existing methods primarily focus on the enhancement process and heavily rely on the supervised learning strategy, where low/normal-light image pairs are used as the training dataset. In this paper, we propose a novel method called Learn Enhancement by Learning Degradation (LELD) to achieve efficient light adjustment and scene fidelity. We use a carefully designed degradation network (DNet) to guide the enhancement network (ENet). Specifically, the role of DNet is transforming normal-light images into low-light images. For better generalization ability, we employ an unsupervised learning strategy and a generative adversarial network framework. The training is totally dependent on unpaired datasets. Inspired by Retinex theory, we propose a fidelity loss to maintain color and detail during the degradation process. The ENet exhibits a straightforward architecture and achieves efficient enhancement. Experimental results demonstrate the advantages of our method over state-of-the-art methods in terms of visual quality and enhancement efficiency.

Low-light images enhancement via a dense transformer network

This paper proposes a dense network composed of an improved Transformer network, which successfully restores low-light images to high-quality normal-light images, alleviating issues such as low brightness, high noise, and missing critical information in low-light images. The entire network architecture is based on the improved Transformer network and builds a dense network with a combination of long and short connections. While retaining the self-attention mechanism of the Transformer network, it achieves multi-level fusion and utilization of shallow and deep features, providing the network with rich image features and enabling the restoration of low-light images to high-quality normal-light images. Additionally, a spatial-domain and frequency-domain combined loss function is designed, considering both pixel-level and frequency domain losses, effectively constraining the image restoration process and avoiding spectral biases. Lastly, a multi-scale hybrid gate feedforward network is designed to replace the traditional feedforward network in the Transformer, facilitating feature selection and forward propagation. These designs effectively enhance the richness of meaningful image features, alleviate spectral biases, and improve the visual quality of low-light images. Experimental results demonstrate the superiority of our method over state-of-the-art networks on various typical image enhancement datasets. Taking the most commonly used low-light dataset LOLv1 as an example, our method achieves improvements of 1.3% and 3.07% in PSNR and SSIM, respectively, compared to the best-performing network, showing favorable qualitative and quantitative evaluation results. The proposed method effectively addresses the issue of insufficiently realistic results in low-light image restoration, providing a reliable reference for practical applications.

Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data.

Deep learning-based low-light image enhancement (LLIE) is a task of leveraging deep neural networks to enhance the image illumination while keeping the image content unchanged. From the perspective of training data, existing methods complete the LLIE task driven by one of the following three data types: paired data, unpaired data and zero-reference data. Each type of these data-driven methods has its own advantages, e.g., zero-reference data-based methods have very low requirements on training data and can meet the human needs in many scenarios. In this paper, we leverage pure Gaussian noise to complete the LLIE task, which further reduces the requirements for training data in LLIE tasks and can be used as another alternative in practical use. Specifically, we propose Noise SElf-Regression (NoiSER) without access to any task-related data, simply learns a convolutional neural network equipped with an instance-normalization layer by taking a random noise image, N(0,σ2) for each pixel, as both input and output for each training pair, and then the low-light image is fed to the trained network for predicting the normal-light image. Technically, an intuitive explanation for its effectiveness is as follows: 1) the self-regression reconstructs the contrast between adjacent pixels of the input image, 2) the instance-normalization layer may naturally remediate the overall magnitude/lighting of the input image, and 3) the N(0,σ2) assumption for each pixel enforces the output image to follow the well-known gray-world hypothesis [1] when the image size is big enough. Compared to current state-of-the-art LLIE methods with access to different task-related data, NoiSER is highly competitive in enhancement quality, yet with a much smaller model size, and much lower training and inference cost. In addition, the experiments also demonstrate that NoiSER has great potential in overexposure suppression and joint processing with other restoration tasks.

Cycle-Retinex: Unpaired Low-Light Image Enhancement via Retinex-Inline CycleGAN

Low-light image enhancement aims to recover normal-light images from the images captured under dim environments. Most existing methods could just improve the light appearance globally whereas failing to handle other degradation such as dense noise, color offset and extremely low-light. Moreover, unsupervised methods proposed in recent years lack reliable physical model as the basis, thus universality is greatly limited. To address these problems, we propose a novel low-light image enhancement method via Retinex-inline cycle-consistent generative adversarial network named Cycle-Retinex, whose training is totally dependent on unpaired datasets. Specifically, we organically combine Retinex theory with CycleGAN, by which we decouple low-light image enhancement task into two sub-tasks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> illumination map enhancement and reflectance map restoration. Retinex theory helps CycleGAN simplify low-light image enhancement problem and CycleGAN provides synthetic paired images to guide the training of Retinex decomposition network. We further introduce a self-augmented method to address the color distortion and noise problem, thus making the network learn to enhance low-light images adaptively. Extensive experiments show that the proposed method can achieve promising results. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mummmml/Cycle-Retinex</uri> .

HPCDNet: Hybrid position coding and dual-frquency domain transform network for low-light image enhancement.

Low-light image enhancement (LLIE) improves lighting to obtain natural normal-light images from images captured under poor illumination. However, existing LLIE methods do not effectively utilize positional and frequency domain image information. To address this limitation, we proposed an end-to-end low-light image enhancement network called HPCDNet. HPCDNet uniquely integrates a hybrid positional coding technique into the self-attention mechanism by appending hybrid positional codes to the query and key, which better retains spatial positional information in the image. The hybrid positional coding can adaptively emphasize important local structures to improve modeling of spatial dependencies within low-light images. Meanwhile, frequency domain image information lost under low-light is recovered via discrete wavelet and cosine transforms. The resulting two frequency domain feature types are weighted and merged using a dual-attention module. More effective use of frequency domain information enhances the network's ability to recreate details, improving visual quality of enhanced low-light images. Experiments demonstrated that our approach can heighten visibility, contrast and color properties of low-light images while better preserving details and textures than previous techniques.

LL-WSOD: Weakly supervised object detection in low-light

Weakly supervised detection performs significantly lower than the fully supervised methods due to the lack of detailed and precise annotations. Especially, its performance deteriorates more severely in low-light conditions with the lack of low-light datasets. To overcome these issues, we propose a new Low-Light Weakly Supervised Object Detection (LL-WSOD) framework. First, we propose a Progressive Low-light NoiseModule (PLNM) to train the model progressively with low light using the common datasets with normal light, greatly reducing the training difficulty. Next, a Residual Self-refinement Low-light Rebuild Module (ResSLRM) is proposed to allow convolutional neural networks to learn sharper features by rebuilding low-light features into normal-light images. Finally, a Pseudo Boundingbox Assisted Learning Module (PBALM) is designed to perform better low-light training using salient priors. The results show that the proposed LL-WSOD algorithm effectively detects objects under low-light conditions and achieves great results on the real low-light dataset ExDark.

Dark2Light: multi-stage progressive learning model for low-light image enhancement.

Due to severe noise and extremely low illuminance, restoring from low-light images to normal-light images remains challenging. Unpredictable noise can tangle the weak signals, making it difficult for models to learn signals from low-light images, while simply restoring the illumination can lead to noise amplification. To address this dilemma, we propose a multi-stage model that can progressively restore normal-light images from low-light images, namely Dark2Light. Within each stage, We divide the low-light image enhancement (LLIE) into two main problems: (1) illumination enhancement and (2) noise removal. Firstly, we convert the image space from sRGB to linear RGB to ensure that illumination enhancement is approximately linear, and design a contextual transformer block to conduct illumination enhancement in a coarse-to-fine manner. Secondly, a U-Net shaped denoising block is adopted for noise removal. Lastly, we design a dual-supervised attention block to facilitate progressive restoration and feature transfer. Extensive experimental results demonstrate that the proposed Dark2Light outperforms the state-of-the-art LLIE methods both quantitatively and qualitatively.

Cartoon-texture guided network for low-light image enhancement

Recovering normal-exposure images from low-light images is a challenging task. Recent works have built a great deal of deep learning methods to address this task. Nevertheless, most of them treat cartoon and texture components in the same way, resulting in a loss of details. Recent effort, i.e. unfolding total variation network (UTVNet), is proposed, which recovers normal-light image by roughly decomposing the image into a noise-free smoothing layer and a detail layer using total variation (TV) regularization, and then processes the two components in different ways. However, its enhanced image exhibits color distortion owing to the limited representation ability of the TV model. To address this limitation, we design a cartoon-texture guided network named CatNet for low-light image enhancement. CatNet uses a cartoon-guided normalizing flow to retain cartoon information and an elaborated frequency domain attention mechanism in U-Net denoted as FAU-Net to recover texture information. Concretely, the ground-truth image is decomposed into cartoon and texture components to guide the corresponding recovery modules training, respectively. We also design a hybrid loss in the spatial and frequency domains to train the CatNet. Compared to state-of-the-art methods, our method gets better results, obtaining richer colors and more details. The source code and datasets have been made publicly available at https://github.com/shibaoshun/CatNet.

SAGAN: Deep semantic-aware generative adversarial network for unsupervised image enhancement

Low-light image enhancement (LLIE) is a common pretext task for computer vision, which aims to adjust the luminance of the low-light image to obtain the normal-light image. At present, unsupervised LLIE has been developed. However, its performance is limited due to the lack of sufficient semantic information and guidance from a strict discriminator. In this work, a semantic-aware generative adversarial network is proposed to alleviate the above limitations. We use the pre-trained VGG model on ImageNet to extract the prior semantic information, which is organically fed into the generator to refine its feature representation, and develop an adaptive image fusion strategy working on the output layer of the generator. Further, to improve the discriminator’s capacity of supervising generator, we design the dual-discriminator with dense connection and two image quality-driven priority queues with time-aware. The quantitative and qualitative experiments on four testing datasets demonstrate the competitiveness of the proposed model and the effectiveness of each component. Our code is available at: https://github.com/Shecyy/SAGAN.

N-LoLiGan: Unsupervised low-light enhancement GAN with an N-Net for low-light tunnel images

Because of low luminance in tunnels, target features in images are not salient, which makes the detection of tunnel surface defects challenging. Most deep-learning-based low-light enhancement methods suffer from contrast distortion and insufficient brightness in extremely dark scenes, as well as artifacts and overexposure in tunnel scenes with large light and dark differences. To alleviate these problems, we propose a novel unsupervised generative adversarial network (GAN), called the N-shape low-light enhancement GAN (N-LoLiGan), which can be trained using unpaired low-/normal-light images and is proven to generalize sufficiently on tunnel images with complex lighting. Experimental results demonstrate that the proposed N-LoLiGan outperforms several state-of-the-art low-light image enhancement methods in terms of visual quality and two no-reference image quality metrics. Additionally, when the proposed method is used as a preprocessing step for the tunnel multi-target detection task, it yields a higher detection rate (88.9% and 97.23%) than other methods.

Learning Wasserstein Contrastive Color Histogram Representation for Low-Light Image Enhancement

The goal of low-light image enhancement (LLIE) is to enhance perception to restore normal-light images. The primary emphasis of earlier LLIE methods was on enhancing the illumination while paying less attention to the color distortions and noise in the dark. In comparison to the ground truth, the restored images frequently exhibit inconsistent color and residual noise. To this end, this paper introduces a Wasserstein contrastive regularization method (WCR) for LLIE. The WCR regularizes the color histogram (CH) representation of the restored image to keep its color consistency while removing noise. Specifically, the WCR contains two novel designs including a differentiable CH module (DCHM) and a WCR loss. The DCHM serves as a modular component that can be easily integrated into the network to enable end-to-end learning of the image CH. Afterwards, to ensure color consistency, we utilize the Wasserstein distance (WD) to quantify the resemblance of the learnable CHs between the restored image and the normal-light image. Then, the regularized WD is used to construct the WCR loss, which is a triplet loss and takes the normal-light images as positive samples, the low-light images as negative samples, and the restored images as anchor samples. The WCR loss pulls the anchor samples closer to the positive samples and simultaneously pushes them away from the negative samples so as to help the anchors remove the noise in the dark. Notably, the proposed WCR method was only used for training, and was shown to achieve high performance and high speed inference using lightweight networks. Therefore, it is valuable for real-time applications such as night automatic driving and night reversing image enhancement. Extensive evaluations on benchmark datasets such as LOL, FiveK, and UIEB showed that the proposed WCR method achieves superior performance, outperforming existing state-of-the-art methods.

Cyclic Generative Attention-Adversarial Network for Low-Light Image Enhancement.

Images captured under complex conditions frequently have low quality, and image performance obtained under low-light conditions is poor and does not satisfy subsequent engineering processing. The goal of low-light image enhancement is to restore low-light images to normal illumination levels. Although many methods have emerged in this field, they are inadequate for dealing with noise, color deviation, and exposure issues. To address these issues, we present CGAAN, a new unsupervised generative adversarial network that combines a new attention module and a new normalization function based on cycle generative adversarial networks and employs a global-local discriminator trained with unpaired low-light and normal-light images and stylized region loss. Our attention generates feature maps via global and average pooling, and the weights of different feature maps are calculated by multiplying learnable parameters and feature maps in the appropriate order. These weights indicate the significance of corresponding features. Specifically, our attention is a feature map attention mechanism that improves the network's feature-extraction ability by distinguishing the normal light domain from the low-light domain to obtain an attention map to solve the color bias and exposure problems. The style region loss guides the network to more effectively eliminate the effects of noise. The new normalization function we present preserves more semantic information while normalizing the image, which can guide the model to recover more details and improve image quality even further. The experimental results demonstrate that the proposed method can produce good results that are useful for practical applications.

Detachable image decomposition and illumination mapping search for low-light image enhancement

Low-light image enhancement is a fundamental low-level computer vision task that has a wide range of applications, such as night monitoring, all-weather automatic driving, back-light imaging, and so on. In this paper, we propose a novel four-stage detachable framework for low-light image enhancement, where each module can be trained and fine-tuned separately. More importantly, the proposed framework can effectively be adapted to unseen datasets. In the first stage, we propose an image decomposition network that can extract a pair of a reflectance map and an illumination map given an image with arbitrary lighting conditions. The proposed loss function ensures that images of the same scene with different lighting conditions share the same reflectance map. Instead of using gamma correction, we establish an automatic searching scheme in the second stage to find an explicit mapping relationship between illumination maps in different lighting conditions. In the third stage, we use an efficient and effective unsupervised training method to find the best parameter set for the mapping. Finally, a normal-light image is obtained by composing together its reflectance map and transformed illumination map. A series of experiments are conducted on several popular datasets, and the results demonstrate the superiority of our proposed method on unseen datasets. Our framework surpasses other state-of-the-art methods and is applicable to enhance images in a wide range of lighting conditions.

Event-guided low light image enhancement via a dual branch GAN

In the low light conditions, images are corrupted by low contrast and severe noise, but event cameras capture event streams with clear edge structures. Therefore, we propose an Event-Guided Low Light Image Enhancement method using a dual branch generative adversarial networks and recover clear structure with the guide of events. To overcome the lack of paired training datasets, we first synthesize three datasets containing low-light event streams, low-light images, and the ground truth normal-light images. Then, in the generator network, we develop an end-to-end dual branch network consisting of a image enhancement branch and a gradient reconstruction branch. The image enhancement branch is employed to enhance the low light images, and the gradient reconstruction branch is utilized to learn the gradient from events. Moreover, we develops the attention based event-image feature fusion module which selectively fuses the event and low-light image features, and the fused features are concatenated into the image enhancement branch and gradient reconstruction branch, which respectively generate the enhanced images with clear structure and more accurate gradient images. Extensive experiments on synthetic and real datasets demonstrate that the proposed event guided low light image enhancement method produces visually more appealing enhancement images, and achieves a good performance in structure preservation and denoising over state-of-the-arts.

CUCN: continuously updated connection network for low-light image enhancement

Low-light (LL) images make subsequent computer vision tasks difficult due to its low contrast. To solve this problem, the LL image enhancement problem is regarded as narrowing the gap between LL images and normal-light images in the process of iterative learning. A continuously updated connection network (CUCN) that is inspired by the recent development of convolutional neural networks is proposed. The proposed CUCN is composed of a continuously updated 4-units module (CU4UM), a feature fusion module (FFM), and a color enhancement module (CEM). The CU4UM adopts a connection method to combine four U-shaped structures with continuously updated parameters. To retain the global and local details while brightening an image, the FFM effectively combines the features of different layers in different U-shaped structures and assigns appropriate weights to each channel. The CEM takes the residuals of multiple features of different scales and performs iterative weighting to obtain more natural color information. The proposed method is evaluated on different public datasets, and the results show that the proposed CUCN method is superior to other state-of-the-art methods in terms of both subjective and objective metrics.

Low-Light Image Enhancement by Combining Transformer and Convolutional Neural Network

Within low-light imaging environment, the insufficient reflected light from objects often results in unsatisfactory images with degradations of low contrast, noise artifacts, or color distortion. The captured low-light images usually lead to poor visual perception quality for color deficient or normal observers. To address the above problems, we propose an end-to-end low-light image enhancement network by combining transformer and CNN (convolutional neural network) to restore the normal light images. Specifically, the proposed enhancement network is designed into a U-shape structure with several functional fusion blocks. Each fusion block includes a transformer stem and a CNN stem, and those two stems collaborate to accurately extract the local and global features. In this way, the transformer stem is responsible for efficiently learning global semantic information and capturing long-term dependencies, while the CNN stem is good at learning local features and focusing on detailed features. Thus, the proposed enhancement network can accurately capture the comprehensive semantic information of low-light images, which significantly contribute to recover normal light images. The proposed method is compared with the current popular algorithms quantitatively and qualitatively. Subjectively, our method significantly improves the image brightness, suppresses the image noise, and maintains the texture details and color information. For objective metrics such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), image perceptual similarity (LPIPS), DeltaE, and NIQE, our method improves the optimal values by 1.73 dB, 0.05, 0.043, 0.7939, and 0.6906, respectively, compared with other methods. The experimental results show that our proposed method can effectively solve the problems of underexposure, noise interference, and color inconsistency in micro-optical images, and has certain application value.