Low-light Imaging Research Articles

Low-light images enhancement and denoising network based on unsupervised learning multi-stream feature modeling

Inspired by the generative adversarial networks EnlightenGAN, this paper proposes a novel low-light images Enhancement and Denoising model based on unsupervised learning Multi-Stream feature modeling (MSED). The model has two stages: generator network and discriminator network. Generator network includes global and local feature modeling network. First, Swin Transformer Block is innovatively introduced in the global feature modeling of generator network stage. It makes the interaction between the image and the convolutional kernel related to the image content. Its shift window mechanism can model the global feature dependency of the input image with less memory consumption, and extract the color, texture, and shape features of the image, thereby effectively suppressing noise and artifacts. Second, in the local feature modeling, a multi-scale image and feature fusion branch is added. It not only extracts reduced features from large-scale low-light images, but also extracts features from multiple downsampled low-light images, and then combines the two through attention and DSFF. By utilizing the complementary information of the reduced features and downsampled images, various underexposure/ overexposure phenomena caused by low-light images can be effectively avoided. In the discriminator network stage, the deep/shallow feature aggregation module is added to enhance the discrimination ability, and the inconsistencies are suppressed by learning the contradictory information of spatial filter, so that the shallow representation of information and the deep semantic information can guide each other. Thanks to the synergy of the above three innovative work, compared with many existing advanced low-light images enhancement models, MSED can achieve SOTA level performance on several public datasets. However, when dealing with low-light images with blurry content caused by rapid motion, MSED still cannot effectively restore their detailed information.

Low-Light Image Enhancement Method for Electric Power Operation Sites Considering Strong Light Suppression

Insufficient light, uneven light, backlighting, and other problems lead to poor visibility of the image of an electric power operation site. Most of the current methods directly enhance the low-light image while ignoring local strong light that may appear in the electric power operation site, resulting in overexposure and a poor enhancement effect. Aiming at the above problems, we propose a low-light image enhancement method for electric power operation sites by considering strong light suppression. Firstly, a sliding-window-based strong light judgment method was designed, which used a sliding window to segment the image, and a brightness judgment was performed based on the average value of the deviation and the average deviation of the subimages of the grayscale image from the strong light threshold. Then, a light effect decomposition method based on a layer decomposition network was used to decompose the light effect of RGB images with the presence of strong light and eliminate the light effect layer. Finally, a Zero-DCE (Zero-Reference Deep Curve Estimation) low-light enhancement network based on a kernel selection module was constructed to enhance the low-light images with reduced or no strong light interference. Comparison experiments using the electric power operation private dataset and the SICE (Single Image Contrast Enhancement) Part 2 public dataset showed that our proposed method outperformed the current state-of-the-art low-light enhancement methods in terms of both subjective visual effects and objective evaluation metrics, which effectively improves the image quality of electric power operation sites in low-light environments and provides excellent image bases for other computer vision tasks, such as the estimation of operators’ posture.

Global-and-local aware network for low-light image enhancement

Photos taken under nighttime or backlit conditions often suffer from complex and unpredictable degradation, such as low visibility, messy noise, and distorted color. Previous methods mainly focused on global brightness and contrast while ignoring structural and textural details, or they handled the fusion of features without adequately considering their intrinsic association, resulting in incomplete feature representations. To address this issue, we propose a global-and-local aware network (GLAN) by projecting the features into the frequency domain and incorporating them in a knowledge-sharing manner. This method effectively integrates the global modeling capability of the transformer and the local sensitivity of the convolutional neural network to represent structure and texture. First, the global branch, which is comprised of transformer blocks, performs feature extraction under the global receptive field, while the local branch constructs multi-scale features to provide local fine-grained details. Then, we design a novel adaptive multi-scale feature block (AMSFB) that deploys channel split operation to decrease the calculation amount. To better learn the channel and spatial correlations of intermediate features, we introduce a multi-scale channel attention module (MSCAM) and a pixel attention module (PAM) into the AMSFB. Finally, a frequency-aware interaction module (FAIM) is developed for bidirectional information supplementation, which builds feature descriptors simultaneously covering low-frequency and high-frequency information based on the discrete cosine transform (DCT). Through extensive quantitative and qualitative experiments, our method can achieve competitive results compared with over ten state-of-the-art image enhancement methods on eight benchmark datasets.

Non-Uniform-Illumination Image Enhancement Algorithm Based on Retinex Theory

To address the issues of fuzzy scene details, reduced definition, and poor visibility in images captured under non-uniform lighting conditions, this paper presents an algorithm for effectively enhancing such images. Firstly, an adaptive color balance method is employed to address the color differences in low-light images, ensuring a more uniform color distribution and yielding a low-light image with improved color consistency. Subsequently, the image obtained is transformed from the RGB space to the HSV space, wherein the multi-scale Gaussian function is utilized in conjunction with the Retinex theory to accurately extract the lighting components and reflection components. To further enhance the image quality, the lighting components are categorized into high-light areas and low-light areas based on their pixel mean values. The low-light areas undergo improvement through an enhanced adaptive gamma correction algorithm, while the high-light areas are enhanced using the Weber–Fechner law for optimal results. Then, each block area of the image is weighted and fused, leading to its conversion back to the RGB space. And a multi-scale detail enhancement algorithm is utilized to further enhance image details. Through comprehensive experiments comparing various methods based on subjective visual perception and objective quality metrics, the algorithm proposed in this paper convincingly demonstrates its ability to effectively enhance the brightness of non-uniformly illuminated areas. Moreover, the algorithm successfully retains details in high-light regions while minimizing the impact of non-uniform illumination on the overall image quality.

Single image relighting based on illumination field reconstruction.

Relighting a single low-light image is a crucial and challenging task. Previous works primarily focused on brightness enhancement but neglected the differences in light and shadow variations, which leads to unsatisfactory results. Herein, an illumination field reconstruction (IFR) algorithm is proposed to address this issue by leveraging physical mechanism guidance, physical-based supervision, and data-based modeling. Firstly, we derived the Illumination field modulation equation as a physical prior to guide the network design. Next, we constructed a physical-based dataset consisting of image sequences with diverse illumination levels as supervision. Finally, we proposed the IFR neural network (IFRNet) to model the relighting progress and reconstruct photorealistic images. Extensive experiments demonstrate the effectiveness of our method on both simulated and real-world datasets, showing its generalization ability in real-world scenarios, even training solely from simulation.

Low-light image enhancement based onRetinex-Net with color restoration.

Low-light images often suffer from a variety of degradation problems such as loss of detail, color distortions, and prominent noise. In this paper, the Retinex-Net model and loss function with color restoration are proposed to reduce color distortion in low-light image enhancement. The model trains the decom-net and color recovery-net to achieve decomposition of low-light images and color restoration of reflected images, respectively. First, a convolutional neural network and the designed loss functions are used in the decom-net to decompose the low-light image pair into an optimal reflection image and illumination image as the input of the network, and the reflection image after normal light decomposition is taken as the label. Then, an end-to-end color recovery network with a simplified model and time complexity is learned and combined with the color recovery loss function to obtain the correction reflection map with higher perception quality, and gamma correction is applied to the decomposed illumination image. Finally, the corrected reflection image and the illumination image are synthesized to get the enhanced image. The experimental results show that the proposed network model has lower brightness-order-error (LOE) and natural image quality evaluator (NIQE) values, and the average LOE and NIQE values of the low-light dataset images can be reduced to 942 and 6.42, respectively, which significantly improves image quality compared with other low-light enhancement methods. Generally, our proposed method can effectively improve image illuminance and restore color information in the end-to-end learning process of low-light images.

Design of a Large-Format Low-Light Imaging System Based on the RGB Filter Wheel

In order to capture true-color information of distant targets under extremely low light, a large-format low-light imaging system is designed based on an RGB filter wheel. By decomposing the system indicators, this study proposes a method for acquiring low-light true-color images using a large-aperture, low-distortion optical lens combined with an RGB filter wheel capable of multi-line sequential exposure. The optical field segmentation is achieved using a four-panel optical reflective prism, and the images from four high-sensitivity SCOMS detectors are stitched together to form a composite image. The working principle of the system is explained, and the low-light imaging capability is thoroughly evaluated. The dimensions and rotation speed of the filter wheel are then calculated in detail, ensuring accurate synchronization of the filter wheel’s speed and exposure time. The calculation method for the parameters of the four-panel reflective prism structure is investigated, mathematical expressions for the geometric parameters of the prism assembly are provided, and a prism assembly suitable for four-way spectral separation is designed. Based on the research and design results, a large-swath-width, low-light true-color imaging system is developed that is suitable for an environmental illuminance of 0.01 lux. The system achieves a ground pixel resolution of 0.5 m (at an altitude of 5 km) and an effective image resolution of 4 K × 4 K, and is capable of accurately reproducing target color information. Laboratory and field flight tests verified that the large-swath-width images obtained by the imaging system are clear, with high contrast and resolution. After image fusion and spectral registration, the color images exhibit full saturation and high fidelity, meeting the requirements of low-light true-color imaging under airborne conditions. The design methodology of this low-light imaging system can serve as a reference for the development of airborne low-light imaging equipment.

A Zero-Shot Low Light Image Enhancement Method Integrating Gating Mechanism.

Photographs taken under harsh ambient lighting can suffer from a number of image quality degradation phenomena due to insufficient exposure. These include reduced brightness, loss of transfer information, noise, and color distortion. In order to solve the above problems, researchers have proposed many deep learning-based methods to improve the illumination of images. However, most existing methods face the problem of difficulty in obtaining paired training data. In this context, a zero-reference image enhancement network for low light conditions is proposed in this paper. First, the improved Encoder-Decoder structure is used to extract image features to generate feature maps and generate the parameter matrix of the enhancement factor from the feature maps. Then, the enhancement curve is constructed using the parameter matrix. The image is iteratively enhanced using the enhancement curve and the enhancement parameters. Second, the unsupervised algorithm needs to design an image non-reference loss function in training. Four non-reference loss functions are introduced to train the parameter estimation network. Experiments on several datasets with only low-light images show that the proposed network has improved performance compared with other methods in NIQE, PIQE, and BRISQUE non-reference evaluation index, and ablation experiments are carried out for key parts, which proves the effectiveness of this method. At the same time, the performance data of the method on PC devices and mobile devices are investigated, and the experimental analysis is given. This proves the feasibility of the method in this paper in practical application.

Low-Light Image Contrast Enhancement with Adaptive Noise Attenuator for Augmented Vehicle Detection

The rapid progress in deep learning technologies has accelerated the use of object detection models, but most models do not operate satisfactorily in low-light environments. As a result, many studies have been conducted on image enhancement techniques aiming to make objects more visible by increasing contrast, but the process of image enhancement may negatively impact detection as it further strengthens unwanted noises due to indirect factors of light reflection such as overall low brightness, streetlamps, and neon signboards. Therefore, in this study, we propose a technique for improving the performance of object detection in low-light environments. The proposed technique inverts a low-light image to make it similar to a hazy image and then uses a haze removal algorithm based on entropy and fidelity to increase image contrast, clarifying the boundary between the object and the background. In the next step, we used the adaptive 2D Wiener filter (A2WF) to attenuate the noise accidentally strengthened during the image enhancement process and reinforced the boundary between the object and the background to increase detection performance. The test evaluation results showed that the proposed image enhancement scheme significantly increased image perception performance with the perception-based image quality evaluator being 12.73% lower than existing image enhancement techniques. In a comparison of vehicle detection performance, the proposed technique for enhancing nighttime images combined with the detection model proved its effectiveness by increasing the average precision by up to 18.63% against existing detection methods.

A Collaborative Learning-based Urban Low-light Small-target Face Image Enhancement Method

Face recognition is an essential technology in intelligent transportation and security within smart cities. Nevertheless, face images taken in nighttime urban environments often suffer from low brightness, small sizes, and low resolution, which pose significant challenges for accurate face feature recognition. To address this issue, we propose the Low-light Small-target Face Enhancement (LSFE) method, a collaborative learning-based image brightness enhancement approach specifically designed for small-target faces in low-light environments. LSFE employs a multilevel feature stratification module to acquire detailed face image features at different levels, revealing hidden facial image information within the dark. In addition, we design a network combining collaborative learning and self-attention mechanisms, which effectively captures long-distance pixel dependencies in low-brightness face images and enhances their brightness in a stepwise manner. The enhanced feature maps are then fused through a branch fusion module. Experimental results demonstrate that LSFE can more effectively enhance the luminance of small-target face images in low-light scenes while retaining more visual information, compared to other existing methods.

Discovering Hidden Offshore Lighting Structures with Multiyear Low-Light Imaging Satellite Data

The NASA/NOAA Visible Infrared Imaging Radiometer Suite (VIIRS) day/night band (DNB) has a capability to detect lighting present at the earth’s surface. This low-light imaging capability was built to satisfy a requirement from meteorologists to detect clouds at night in the visible based on reflected moonlight. On land, human settlements are the dominant source of DNB detected surface lighting. Lights are also detected offshore primarily arising from fishing boats using lights to attract catch. In 2015, the Earth Observation Group developed the VIIRS Boat Detection (VBD) product and today several fishery agencies use VBD data to monitor fishing activity and compliance with closures. Recently EOG compiled the full record VBD detections as a 15 arc second global grid. The record spans 2012-2021 in Asia and 2017-2021 elsewhere. Upon reviewing the multiyear accumulation of VBD detections we were surprised to find a diversity of previously unseen lighting features [1]. These structures are not evident in VBD from single nights but are only revealed by accumulating the detections across a year or more. The additional features include lit platforms, transit lanes, and vessel anchorages associated with ports and passage straits. Another surprising finding is geopolitical divisions in the density of cumulative vessel detections. In this paper we provide a look into one of the densest areas of VBD detections in the world, from the Yellow Sea, East China Sea and Sea of Japan (East Sea).

Multi-Resolution Edge-aware Lighting Enhancement Network

Images taken under poor lighting conditions generally present problems such as low brightness, noise, and color distortion, so dealing with low-light images is challenging. To solve this problem, current mainstream methods focus on designing special deep networks. For example, many methods are based on Retinex-based models. However, the solution of the Retinex model is an ill-conditioned problem in mathematics, so these methods are difficult to obtain the expected result. In this paper, following the new paradigm of embedding prior knowledge in deep learning, we design a simple yet surprisingly effective neural network, called Multi-Resolution Edge-aware Lighting Enhancement Network (MELENet). Since resolution and acutance are two of the key factors related to the visual quality of natural images, MELENet with a dual-branch structure aims to incorporate multi-resolution edge information into deep learning for guiding lighting enhancement. Additionally, by adopting special loss functions, the model suppresses noise and prevents color distortion while obtaining sound lighting enhancement effects. Experiments have verified that such a simple solution exhibits outstanding performance compared to existing mainstream lighting enhancement methods. We also directly apply the proposed model to image de-hazing and obtain state-of-the-art results.

CUI-Net: a correcting uneven illumination net for low-light image enhancement

Uneven lighting conditions often occur during real-life photography, such as images taken at night that may have both low-light dark areas and high-light overexposed areas. Traditional algorithms for enhancing low-light areas also increase the brightness of overexposed areas, affecting the overall visual effect of the image. Therefore, it is important to achieve differentiated enhancement of low-light and high-light areas. In this paper, we propose a network called correcting uneven illumination network (CUI-Net) with sparse attention transformer and convolutional neural network (CNN) to better extract low-light features by constraining high-light features. Specifically, CUI-Net consists of two main modules: a low-light enhancement module and an auxiliary module. The enhancement module is a hybrid network that combines the advantages of CNN and Transformer network, which can alleviate uneven lighting problems and enhance local details better. The auxiliary module is used to converge the enhancement results of multiple enhancement modules during the training phase, so that only one enhancement module is needed during the testing phase to speed up inference. Furthermore, zero-shot learning is used in this paper to adapt to complex uneven lighting environments without requiring paired or unpaired training data. Finally, to validate the effectiveness of the algorithm, we tested it on multiple datasets of different types, and the algorithm showed stable performance, demonstrating its good robustness. Additionally, by applying this algorithm to practical visual tasks such as object detection, face detection, and semantic segmentation, and comparing it with other state-of-the-art low-light image enhancement algorithms, we have demonstrated its practicality and advantages.

Cyclic Generative Attention-Adversarial Network for Low-Light Image Enhancement.

Images captured under complex conditions frequently have low quality, and image performance obtained under low-light conditions is poor and does not satisfy subsequent engineering processing. The goal of low-light image enhancement is to restore low-light images to normal illumination levels. Although many methods have emerged in this field, they are inadequate for dealing with noise, color deviation, and exposure issues. To address these issues, we present CGAAN, a new unsupervised generative adversarial network that combines a new attention module and a new normalization function based on cycle generative adversarial networks and employs a global-local discriminator trained with unpaired low-light and normal-light images and stylized region loss. Our attention generates feature maps via global and average pooling, and the weights of different feature maps are calculated by multiplying learnable parameters and feature maps in the appropriate order. These weights indicate the significance of corresponding features. Specifically, our attention is a feature map attention mechanism that improves the network's feature-extraction ability by distinguishing the normal light domain from the low-light domain to obtain an attention map to solve the color bias and exposure problems. The style region loss guides the network to more effectively eliminate the effects of noise. The new normalization function we present preserves more semantic information while normalizing the image, which can guide the model to recover more details and improve image quality even further. The experimental results demonstrate that the proposed method can produce good results that are useful for practical applications.

Low-light image enhancement by two-stream contrastive learning in both spatial and frequency domains

Low-light image enhancement by two-stream contrastive learning in both spatial and frequency domains

DEANet: Decomposition Enhancement and Adjustment Network for Low-Light Image Enhancement

Poor illumination greatly affects the quality of obtained images. In this paper, a novel convolutional neural network named DEANet is proposed on the basis of Retinex for low-light image enhancement. DEANet combines the frequency and content information of images and is divided into three subnetworks: decomposition, enhancement, and adjustment networks, which perform image decomposition; denoising, contrast enhancement, and detail preservation; and image adjustment and generation, respectively. The model is trained on the public LOL dataset, and the experimental results show that it outperforms the existing state-of-the-art methods regarding visual effects and image quality.

Illumination-Adaptive Unpaired Low-Light Enhancement

Supervised networks address the task of low-light enhancement using paired images. However, collecting a wide variety of low-light/clean paired images is tedious as the scene needs to remain static during imaging. In this paper, we propose an unsupervised low-light enhancement network using context-guided illumination-adaptive norm (CIN). Inspired by coarse to fine methods, we propose to address this task in two stages. In stage-I, a pixel amplifier module (PAM) is used to generate a coarse estimate with an overall improvement in visibility and aesthetic quality. Stage-II further enhances the saturated dark pixels and scene properties of the image using CIN. Different ablation studies show the importance of PAM and CIN in improving the visible quality of the image. Next, we propose a region-adaptive single input multiple output (SIMO) model that can generate multiple enhanced images from a single low-light image. The objective of SIMO is to let users choose the image of their liking from a pool of enhanced images. Human subjective analysis of SIMO results shows that the distribution of preferred images varies, endorsing the importance of SIMO-type models. Lastly, we propose a low-light road scene (LLRS) dataset having an unpaired collection of low-light and clean scenes. Unlike existing datasets, the clean and low-light scenes in LLRS are real and captured using fixed camera settings. Exhaustive comparisons on publicly available datasets, and the proposed dataset reveal that the results of our model outperform prior art quantitatively and qualitatively.

Low-Light Image Enhancement via Stage-Transformer-Guided Network

Images collected in low-light environments usually suffer from multiple, non-uniform distributed distortions, including local dark, dim light, backlit and so on. In this paper, we propose a Stage-Transformer-Guided Network (STGNet) that effectively handles region-specific distributions and enhance diverse low-light images. Specifically, our STGNet adopts a multi-stage way to progressively learn hierarchical features that benefit the robustness of our model. At each stage, we design an efficient transformer with horizontal and vertical attentions that jointly capture degradation distributions with different magnitudes and orientations. We also introduce learnable degradation queries to adaptively select task-specific features of degradations for enhancement. In addition, we design a histogram loss for enhancement and combine it with other loss functions, in order to exploit both global contrast and local details during network training. Benefiting from the above contributions, our STGNet achieves the state-of-the-art performances on both synthetic and real-world datasets.

Low light image enhancement algorithm based on improved multi-objective grey wolf optimization with detail feature enhancement

This article aims to enhance low-light images by proposing a novel algorithm called MoGDF. It departs from the traditional approach of using deep learning with low-light and normal images as training data. Instead, exposure images at five different exposure scales are generated using a proposed model called EIPM, which consists of an improved Gamma function. Then it introduces a feature extraction network called FMEM, which leverages an improved PFPN structure to enhance the fusion of multi-scale information and strengthen the connection between high-level and low-level semantic information. Finally, it optimizes the fusion weights using an improved multiple-objective grey wolf optimization algorithm with comprehensive improvements in the convergence factor updating strategy, adaptive inertia weight strategy, and individual position strategy. The experiments show that the MoGDF outperforms the current state-of-the-art algorithms in terms of color restoration and detail preservation, and has significant advantages in objective and subjective evaluation metrics, with improvements of 5.69%, 2.60%, 1.11%, and 6.67% in PSNR, SSIM, NIQE, and LPIPS, respectively.

A reflectance re-weighted Retinex model for non-uniform and low-light image enhancement

Image enhancement is a fundamental low-level task of significant importance that can directly affect high-level image processing tasks. Although various methods have been proposed to enhance images, the effectiveness of current methods deteriorates significantly under non-uniform lighting. Since the brightness may vary dramatically in different regions of real-world photos, current methods hardly achieve a good balance between enhancing low-light regions and retaining normal-light regions in the same image. Consequently, either the low-light regions are under-enhanced or the normal-light regions are over-enhanced, while at the same time, color distortion and artifacts are frequently found. To overcome this shortcoming, we propose a robust Retinex-based model with reflectance map re-weighting that can improve the brightness level of the low-light image and re-balance the brightness concurrently. We introduce an alternating scheme to solve our proposed model, in which the illumination map, reflectance map, and weighting map are updated iteratively. By utilizing the regularization terms, the noise is well-suppressed during the process. An initialization scheme for the weighting map is also proposed to make our model adaptable to a wide range of light conditions. To the best of our knowledge, we are the first to propose a variational model with an explicitly constructed re-weighting prior and the associated weighing map concept for the reflectance map. It can estimate the reflectance map, suppress noise, and re-balance the brightness simultaneously. A series of experimental results on a variety of popular datasets demonstrate the efficacy of our method and its superiority in enhancing real low-light images when compared to other state-of-the-art methods.