Low-light Enhancement Research Articles

Retraction Note: Sub-network modeling and integration for low-light enhancement of aerial images

Retraction Note: Sub-network modeling and integration for low-light enhancement of aerial images

A new two-stage low-light enhancement network with progressive attention fusion strategy

Low-light image enhancement is a very challenging subject in the field of computer vision. It has a large number of degradation problems such as accumulated noise, artifacts, and color distortion. Therefore, how to solve the degradation problems and obtain clear images with high visual quality has become an important issue. It can effectively improve the performance of high-level computer vision tasks. In this study, we propose a new two-stage low-light enhancement network with a progressive attention fusion strategy, and the two hallmarks of this method are the use of global feature fusion (GFF) and local detail restoration (LDR), which can enrich the global content of the image and restore local details. Experimental results on the LOL dataset show that the proposed model can achieve good enhancement effects. Moreover, on the benchmark dataset without reference images, the proposed model also obtains a better NIQE score, which outperforms most existing state-of-the-art methods in both quantitative and qualitative evaluations. All these verify the effectiveness and superiority of the proposed method.

Low-light enhancement network for nighttime highway visibility estimation in foggy weather

Low-light enhancement network for nighttime highway visibility estimation in foggy weather

Automatic assessment of concrete cracks in low-light, overexposed, and blurred images restored using a generative AI approach

Deep learning-based computer vision techniques have high efficiency in assessing concrete cracks from images, and the assessment can be automated using robots for higher efficiency. However, assessment accuracy is often compromised by low-quality images. This paper presents a Conditional Generative Adversarial Network (CGAN)-based approach to restore low-light, overexposed, and blurred images. The approach integrates attention mechanisms and residual learning and uses Wasserstein loss with gradient penalty. Crack assessment results show that the proposed approach outperforms state-of-the-art methods, regarding structural similarity (SSIM: 0.78 for deblurring, 0.95 for low-light enhancement, and 0.96 for overexposure correction) and peak signal-to-noise ratio (PSNR: 28.6 for deblurring, 31.4 for low-light enhancement, and 31.6 for overexposure correction). Restored images have been used to train a deep learning model for assessing concrete cracks. The Intersection over Union (IoU) and F1 score of crack segmentation are higher than 0.98 and 0.99, respectively, revealing high accuracy in crack assessment tasks.

New color channel driven physical lighting model for low-light image enhancement

Outdoor imaging systems, affected by low-light conditions, generally produce low-quality images with poor visibility. Low-quality images can directly influence high-level tasks such as surveillance and autonomous navigation systems. Enhancing the images captured under inadequate lighting conditions aims to generate higher visual quality in these images. However, current low-light enhancement methods may result in color unnaturalness, information loss, and strange artifacts. We propose a new color channel-driven physical lighting model (NCC-PLM) to respond to these issues to improve image quality. More concretely, we first apply a gamma correction to the input image according to its darkness degree, which is determined by its average intensity value. Then, we introduce a new color channel prior to estimate the environmental light (EL) and light scattering attenuation rate (LSAR). Finally, the enhanced image is obtained through the estimations and physical lighting model. Experimental results on various datasets demonstrate the proposed method's effectiveness and superiority over the compared methods both visually and qualitatively. Specifically, we enhance the visual quality of low-light images by revealing intricate details and maintaining color consistency, leading to a natural appearance.

Attention Retinex Network(A4R-Net) for face detection under low-light environment

The degradation of recognition rates in low-light environments is a critical issue in terms of security when using object and face recognition technologies in various locations. Existing low-light enhancement models have shown limitations in terms of computational cost and performance. However, this paper overcomes these limitations. The experimental results demonstrate that our model achieves the same performance as existing models with 13 times lower computational cost and a face detection performance of 82.2%.The structure of this paper is as follows: Introduction, which provides the background and explains the limitations of existing models. Proposed Method, which details the structure and working principles of A4R-Net. Experimental Results, which present the evaluation of low-light enhancement performance and the comparison of face detection using YOLOv4 [1]. Conclusion, which discusses the contributions of this research and future research directions.The source code and dataset is https://github.com/Obiru2698/obiru2698.github.io/

Unsupervised Illumination Adaptation for Low-Light Vision.

Insufficient lighting poses challenges to both human and machine visual analytics. While existing low-light enhancement methods prioritize human visual perception, they often neglect machine vision and high-level semantics. In this paper, we make pioneering efforts to build an illumination enhancement model for high-level vision. Drawing inspiration from camera response functions, our model could enhance images from the machine vision perspective despite being lightweight in architecture and simple in formulation. We also introduce two approaches that leverage knowledge from base enhancement curves and self-supervised pretext tasks to train for different downstream normal-to-low-light adaptation scenarios. Our proposed framework overcomes the limitations of existing algorithms without requiring access to labeled data in low-light conditions. It facilitates more effective illumination restoration and feature alignment, significantly improving the performance of downstream tasks in a plug-and-play manner. This research advances the field of low-light machine analytics and broadly applies to various high-level vision tasks, including classification, face detection, optical flow estimation, and video action recognition.

Unsupervised low-light image enhancement by data augmentation and contrastive learning

ABSTRACT Today, with the increasing demand for visual perception and high-level computational vision tasks, the field of low-light enhancement is rapidly developing. However, models trained on existing datasets often fail or suffer significant performance degradation in real-world low-light scenarios. This performance degradation is frequently due to the limitations of current databases, which typically contain small quantities of paired images of a single type. This article proposes an unsupervised model with a unique data augmentation technique that transforms a regular image database into a paired image database. By adjusting image parameters during training to change exposure, a regular image database can be converted into a paired one. The model restores low-exposure images by extracting lighting features through comparative learning. Evaluations of the LOL and DIV2K datasets demonstrate the proposed model's effectiveness, achieving notable results in low-light image enhancement. This method removes dataset restrictions, broadening the model's range of applications.

Real-time low-light video enhancement on smartphones

Real-time low-light video enhancement on smartphones remains an open challenge due to hardware constraints such as limited sensor size and processing power. While night mode cameras have been introduced in smartphones to acquire high-quality images in light-constrained environments, their usability is restricted to static scenes as the camera must remain stationary for an extended period to leverage long exposure times or burst imaging techniques. Concurrently, significant process has been made in low-light enhancement on images coming out from the camera’s image signal processor (ISP), particularly through neural networks. These methods do not improve the image capture process itself; instead, they function as post-processing techniques to enhance the perceptual brightness and quality of captured imagery for display to human viewers. However, most neural networks are computationally intensive, making their mobile deployment either impractical or requiring considerable engineering efforts. This paper introduces VLight, a novel single-parameter low-light enhancement algorithm that enables real-time video enhancement on smartphones, along with real-time adaptation to changing lighting conditions and user-friendly fine-tuning. Operating as a custom brightness-booster on digital images, VLight provides real-time and device-agnostic enhancement directly on users’ devices. Notably, it delivers real-time low-light enhancement at up to 67 frames per second (FPS) for 4K videos locally on the smartphone.

Low-light enhancement method with dual branch feature fusion and learnable regularized attention

Restricted by the lighting conditions, the images captured at night tend to suffer from color aberration, noise, and other unfavorable factors, making it difficult for subsequent vision-based applications. To solve this problem, we propose a two-stage size-controllable low-light enhancement method, named Dual Fusion Enhancement Net (DFEN). The whole algorithm is built on a double U-Net structure, implementing brightness adjustment and detail revision respectively. A dual branch feature fusion module is adopted to enhance its ability of feature extraction and aggregation. We also design a learnable regularized attention module to balance the enhancement effect on different regions. Besides, we introduce a cosine training strategy to smooth the transition of the training target from the brightness adjustment stage to the detail revision stage during the training process. The proposed DFEN is tested on several low-light datasets, and the experimental results demonstrate that the algorithm achieves superior enhancement results with the similar parameters. It is worth noting that the lightest DFEN model reaches 11 FPS for image size of 1224×1024 in an RTX 3090 GPU.Graphical

ST-Phys: Unsupervised Spatio-Temporal Contrastive Remote Physiological Measurement.

Remote photoplethysmography (rPPG) is a non-contact method that employs facial videos for measuring physiological parameters. Existing rPPG methods have achieved remarkable performance. However, the success mainly profits from supervised learning over massive labeled data. On the other hand, existing unsupervised rPPG methods fail to fully utilize spatio-temporal features and encounter challenges in low-light or noise environments. To address these problems, we propose an unsupervised contrast learning approach, ST-Phys. We incorporate a low-light enhancement module, a temporal dilated module, and a spatial enhanced module to better deal with long-term dependencies under the random low-light conditions. In addition, we design a circular margin loss, wherein rPPG signals originating from identical videos are attracted, while those from distinct videos are repelled. Our method is assessed on six openly accessible datasets, including RGB and NIR videos. Extensive experiments reveal the superior performance of our proposed ST-Phys over state-of-the-art unsupervised rPPG methods. Moreover, it offers advantages in parameter reduction and noise robustness.

A robust and real-time lane detection method in low-light scenarios to advanced driver assistance systems

Lane detection, which relies on front-view RGB cameras, is a crucial aspect of Advanced Driver Assistance Systems (ADAS), but its effectiveness is notably reduced in low-light conditions. This issue is exacerbated by the lack of specialized datasets and generalizable methods for such scenarios. To address this gap, we introduce NightLane, a comprehensive dataset tailored for low-light, multi-traffic lane detection. We adhere to stringent data annotation guidelines, ensuring reliable detection accuracy. Additionally, we propose the Fused Low-Light Enhancement Framework (FLLENet), which leverages modern detection networks and incorporates a low-light enhancement module and attention mechanisms. The enhancement module, based on zero-reference learning, improves image quality and channel richness, while the attention mechanisms effectively extract and utilize these features. Our extensive testing on NightLane and CULane datasets demonstrates superior performance in low-light lane detection, showcasing FLLENet’s robust generalizability and efficacy. Specifically, our approach achieves an F1 measure of 76.90 on CULane and 78.91 on NightLane, demonstrating its effectiveness against state-of-the-art methods. We also evaluate the real-time applicability of our framework on a low-power embedded lane detection system using NVIDIA Jetson AGX/Orin, achieving high accuracy and real-time performance. Our work offers a new approach and reference in the field of low-light lane detection, potentially aiding in the ongoing enhancement of ADAS (ADAS). Dateset are available at https://github.com/pengjingt/FLLENet.

Detailed-based dictionary learning for low-light image enhancement using camera response model for industrial applications

Images captured in low-light environments are severely degraded due to insufficient light, which causes the performance decline of both commercial and consumer devices. One of the major challenges lies in how to balance the image enhancement properties of light intensity, detail presentation, and colour integrity in low-light enhancement tasks. This study presents a novel image enhancement framework using a detailed-based dictionary learning and camera response model (CRM). It combines dictionary learning with edge-aware filter-based detail enhancement. It assumes each small detail patch could be sparsely characterised in the over-complete detail dictionary that was learned from many training detail patches using iterative ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1}$$\\end{document}-norm minimization. Dictionary learning will effectively address several enhancement concerns in the progression of detail enhancement if we remove the visibility limit of training detail patches in the enhanced detail patches. We apply illumination estimation schemes to the selected CRM and the subsequent exposure ratio maps, which recover a novel enhanced detail layer and generate a high-quality output with detailed visibility when there is a training set of higher-quality images. We estimate the exposure ratio of each pixel using illumination estimation techniques. The selected camera response model adjusts each pixel to the desired exposure based on the computed exposure ratio map. Extensive experimental analysis shows an advantage of the proposed method that it can obtain enhanced results with acceptable distortions. The proposed research article can be generalised to address numerous other similar problems, such as image enhancement for remote sensing or underwater applications, medical imaging, and foggy or dusty conditions.

Towards a Flexible Semantic Guided Model for Single Image Enhancement and Restoration.

Low-light image enhancement (LLIE) investigates how to improve the brightness of an image captured in illumination-insufficient environments. The majority of existing methods enhance low-light images in a global and uniform manner, without taking into account the semantic information of different regions. Consequently, a network may easily deviate from the original color of local regions. To address this issue, we propose a semantic-aware knowledge-guided framework (SKF) that can assist a low-light enhancement model in learning rich and diverse priors encapsulated in a semantic segmentation model. We concentrate on incorporating semantic knowledge from three key aspects: a semantic-aware embedding module that adaptively integrates semantic priors in feature representation space, a semantic-guided color histogram loss that preserves color consistency of various instances, and a semantic-guided adversarial loss that produces more natural textures by semantic priors. Our SKF is appealing in acting as a general framework in the LLIE task. We further present a refined framework SKF++ with two new techniques: (a) Extra convolutional branch for intra-class illumination and color recovery through extracting local information and (b) Equalization-based histogram transformation for contrast enhancement and high dynamic range adjustment. Extensive experiments on various benchmarks of LLIE task and other image processing tasks show that models equipped with the SKF/SKF++ significantly outperform the baselines and our SKF/SKF++ generalizes to different models and scenes well. Besides, the potential benefits of our method in face detection and semantic segmentation in low-light conditions are discussed. The code and pre-trained models have been publicly available at https://github.com/langmanbusi/Semantic-Aware-Low-Light-Image-Enhancement.

Joint Luminance Adjustment and Color Correction for Low-Light Image Enhancement Network

Most of the existing low-light enhancement research focuses on global illumination enhancement while ignoring the issues of brightness unevenness and color distortion. To address this dilemma, we propose a low-light image enhancement method that can achieve good performance in luminance adjustment and color correction simultaneously. Specifically, the Luminance Adjustment Module is designed to model the global luminance adjustment parameters while taking into account the relationship between global and local illumination features, in order to prevent overexposure or underexposure. Furthermore, we design a Color Correction Module based on the attention mechanism, which utilizes the attention mechanism to capture global color features and correct the color deviation in the illumination-enhanced image. Additionally, we design a color loss function based on a 14-dimensional statistical feature vector related to color, enabling further restoration of the image’s true color. We conduct empirical studies on multiple public low-light datasets, demonstrating that the proposed method outperforms other representative state-of-the-art models regarding illumination enhancement and color correction.

Multi-branch low-light enhancement algorithm based on spatial transformation

Multi-branch low-light enhancement algorithm based on spatial transformation

A Comprehensive Study of Object Tracking in Low-Light Environments.

Accurate object tracking in low-light environments is crucial, particularly in surveillance, ethology applications, and biometric recognition systems. However, achieving this is significantly challenging due to the poor quality of captured sequences. Factors such as noise, color imbalance, and low contrast contribute to these challenges. This paper presents a comprehensive study examining the impact of these distortions on automatic object trackers. Additionally, we propose a solution to enhance the tracking performance by integrating denoising and low-light enhancement methods into the transformer-based object tracking system. Experimental results show that the proposed tracker, trained with low-light synthetic datasets, outperforms both the vanilla MixFormer and Siam R-CNN.

SpaceLight: A Framework for Enhanced On-Orbit Navigation Imagery

In the domain of space rendezvous and docking, visual navigation plays a crucial role. However, practical applications frequently encounter issues with poor image quality. Factors such as lighting uncertainties, spacecraft motion, uneven illumination, and excessively dark environments collectively pose significant challenges, rendering recognition and measurement tasks during visual navigation nearly infeasible. The existing image enhancement methods, while visually appealing, compromise the authenticity of the original images. In the specific context of visual navigation, space image enhancement’s primary aim is the faithful restoration of the spacecraft’s mechanical structure with high-quality outcomes. To address these issues, our study introduces, for the first time, a dedicated unsupervised framework named SpaceLight for enhancing on-orbit navigation images. The framework integrates a spacecraft semantic parsing network, utilizing it to generate attention maps that pinpoint structural elements of spacecraft in poorly illuminated regions for subsequent enhancement. To more effectively recover fine structural details within these dark areas, we propose the definition of a global structure loss and the incorporation of a pre-enhancement module. The proposed SpaceLight framework adeptly restores structural details in extremely dark areas while distinguishing spacecraft structures from the deep-space background, demonstrating practical viability when applied to visual navigation. This paper is grounded in space on-orbit servicing engineering projects, aiming to address visual navigation practical issues. It pioneers the utilization of authentic on-orbit navigation images in the research, resulting in highly promising and unprecedented outcomes. Comprehensive experiments demonstrate SpaceLight’s superiority over state-of-the-art low-light enhancement algorithms, facilitating enhanced on-orbit navigation image quality. This advancement offers robust support for subsequent visual navigation.

Real-Time Attentive Dilated U-Net for Extremely Dark Image Enhancement

Images taken under low-light conditions suffer from poor visibility, color distortion, and graininess, all of which degrade the image quality and hamper the performance of downstream vision tasks, such as object detection and instance segmentation in the field of autonomous driving, making low-light enhancement an indispensable basic component of high-level visual tasks. Low-light enhancement aims to mitigate these issues, and has garnered extensive attention and research over several decades. The primary challenge in low-light image enhancement arises from the low signal-to-noise ratio caused by insufficient lighting. This challenge becomes even more pronounced in near-zero lux conditions, where noise overwhelms the available image information. Both traditional image signal processing pipeline and conventional low-light image enhancement methods struggle in such scenarios. Recently, deep neural networks have been used to address this challenge. These networks take unmodified RAW images as input and produce the enhanced sRGB images, forming a deep learning based image signal processing pipeline. However, most of these networks are computationally expensive and thus far from practical use. In this article, we propose a lightweight model called attentive dilated U-Net (ADU-Net) to tackle this issue. Our model incorporates several innovative designs, including an asymmetric U-shape architecture, dilated residual modules for feature extraction, and attentive fusion modules for feature fusion. The dilated residual modules provide strong representative capability, whereas the attentive fusion modules effectively leverage low-level texture information and high-level semantic information within the network. Both modules employ a lightweight design but offer significant performance gains. Extensive experiments demonstrate that our method is highly effective, achieving an excellent balance between image quality and computational complexity—that is, taking less than 4ms for a high-definition 4K image on a single GTX 1080Ti GPU and yet maintaining competitive visual quality. Furthermore, our method exhibits pleasing scalability and generalizability, highlighting its potential for widespread applicability.

Polarization-Based Two-Stage Image Dehazing in a Low-Light Environment

Fog, as a common weather condition, severely affects the visual quality of images. Polarization-based dehazing techniques can effectively produce clear results by utilizing the atmospheric polarization transmission model. However, current polarization-based dehazing methods are only suitable for scenes with strong illumination, such as daytime scenes, and cannot be applied to low-light scenes. Due to the insufficient illumination at night and the differences in polarization characteristics between it and sunlight, polarization images captured in a low-light environment can suffer from loss of polarization and intensity information. Therefore, this paper proposes a two-stage low-light image dehazing method based on polarization. We firstly construct a polarization-based low-light enhancement module to remove noise interference in polarization images and improve image brightness. Then, we design a low-light polarization dehazing module, which combines the polarization characteristics of the scene and objects to remove fog, thereby restoring the intensity and polarization information of the scene and improving image contrast. For network training, we generate a simulation dataset for low-light polarization dehazing. We also collect a low-light polarization hazy dataset to test the performance of our method. Experimental results indicate that our proposed method can achieve the best dehazing effect.