Low-light Scenes Research Articles

Polarization-Based Two-Stage Image Dehazing in a Low-Light Environment

Fog, as a common weather condition, severely affects the visual quality of images. Polarization-based dehazing techniques can effectively produce clear results by utilizing the atmospheric polarization transmission model. However, current polarization-based dehazing methods are only suitable for scenes with strong illumination, such as daytime scenes, and cannot be applied to low-light scenes. Due to the insufficient illumination at night and the differences in polarization characteristics between it and sunlight, polarization images captured in a low-light environment can suffer from loss of polarization and intensity information. Therefore, this paper proposes a two-stage low-light image dehazing method based on polarization. We firstly construct a polarization-based low-light enhancement module to remove noise interference in polarization images and improve image brightness. Then, we design a low-light polarization dehazing module, which combines the polarization characteristics of the scene and objects to remove fog, thereby restoring the intensity and polarization information of the scene and improving image contrast. For network training, we generate a simulation dataset for low-light polarization dehazing. We also collect a low-light polarization hazy dataset to test the performance of our method. Experimental results indicate that our proposed method can achieve the best dehazing effect.

Salient object detection in low-light RGB-T scene via spatial-frequency cues mining

Low-light conditions pose significant challenges to vision tasks, such as salient object detection (SOD), due to insufficient photons. Light-insensitive RGB-T SOD models mitigate the above problems to some extent, but they are limited in performance as they only focus on spatial feature fusion while ignoring the frequency discrepancy. To this end, we propose an RGB-T SOD model by mining spatial-frequency cues, called SFMNet, for low-light scenes. Our SFMNet consists of spatial-frequency feature exploration (SFFE) modules and spatial-frequency feature interaction (SFFI) modules. To be specific, the SFFE module aims to separate spatial-frequency features and adaptively extract high and low-frequency features. Moreover, the SFFI module integrates cross-modality and cross-domain information to capture effective feature representations. By deploying both modules in a top-down pathway, our method generates high-quality saliency predictions. Furthermore, we construct the first low-light RGB-T SOD dataset as a benchmark for evaluating performance. Extensive experiments demonstrate that our SFMNet can achieve higher accuracy than the existing models for low-light scenes.

An object planar grasping pose detection algorithm in low-light scenes

An object planar grasping pose detection algorithm in low-light scenes

Contrastive Learning for Low-Light Raw Denoising (Student Abstract)

Image/video denoising in low-light scenes is an extremely challenging problem due to limited photon count and high noise. In this paper, we propose a novel approach with contrastive learning to address this issue. Inspired by the success of contrastive learning used in some high-level computer vision tasks, we bring in this idea to the low-level denoising task. In order to achieve this goal, we introduce a new denoising contrastive regularization (DCR) to exploit the information of noisy images and clean images. In the feature space, DCR makes the denoised image closer to the clean image and far away from the noisy image. In addition, we build a new feature embedding network called Wnet, which is more effective to extract high-frequency information. We conduct the experiments on a real low-light dataset that captures still images taken on a moonless clear night in 0.6 millilux and videos under starlight (no moon present). The results show that our method can achieve a higher PSNR and better visual quality compared with existing methods.

LIIS: Low-light image instance segmentation

Image features in low-light scenes become hard to distinguish and full of noise, which makes the performance of current popular instance segmentation models drastically degraded. We propose a two-stage approach for instance segmentation of low-light images with enhancement followed by segmentation. Stage-I corresponds to the Low-Light Image Enhancement (LLIE) process. We propose a post-processing Detail Enhancement Denoising Module (DEDM) to suppress degradation effects caused by the enhancement in the preprocessing stage. Stage-II represents the segmentation process of enhanced images. We construct the W-BCNet instance segmentation network and design a Wavelet Feature Fusion Module (WFFM) in the feature extraction stage to preserve more fine-grained features. We achieve great segmentation results on LIS, detailed comparative experiments and ablation studies show the advantages and excellent generalization ability of our model.

EMNet: Edge-guided multi-level network for salient object detection in low-light images

Salient object detection (SOD) has achieved remarkable performance in well-lit scenes. However, when generalized to low-light scenes, the performance of SOD shows significant decrease owing to more challenging conditions such as weak brightness, low contrast, and poor signal-to-noise ratio. To address this issue, we propose a novel edge-guided and multi-level network (EMNet) for SOD in low light images, which learns robust multiscale region features by optimizing the boundaries of salient objects and employing a multi-stage cascaded strategy. To be more specific, the proposed Edge Feature Highlight (EFH) module can establish mapping relationships at different scales and fuse the outputs obtained from pairs of different branches for extracting accurate boundary information. Secondly, Multi-layer Feature Fusion (MFF) module is proposed for combining multi-scale deep features with salient edge cues, using stepwise fusion for effective integration of deep features. Finally, we employ a coarse-to-fine way for iterative prediction to generate high-quality saliency maps. We conducted comprehensive experiments on LLI dataset, and the results demonstrate that the proposed method achieves the best performance in terms of five evaluation metrics, where an average improvement of max Fβ, ωFβ, Em, Sm, and MAE outperforms state-of-the-art method by 7.20%, 12.06%, 4.62%, 5.00%, and 36.34%, respectively.

Portrait Semantic Segmentation Method Based on Dual Modal Information Complementarity

Semantic segmentation of human images is a research hotspot in the field of computer vision. At present, the semantic segmentation models based on U-net generally lack the ability to capture the spatial information of images. At the same time, semantic incompatibility exists because the feature maps of encoder and decoder are directly connected in the skip connection stage. In addition, in low light scenes such as at night, it is easy for false segmentation and segmentation accuracy to appear. To solve the above problems, a portrait semantic segmentation method based on dual-modal information complementarity is proposed. The encoder adopts a double branch structure, and uses a SK-ASSP module that can adaptively adjust the convolution weights of different receptor fields to extract features in RGB and gray image modes respectively, and carries out cross-modal information complementarity and feature fusion. A hybrid attention mechanism is used in the jump connection phase to capture both the channel and coordinate context information of the image. Experiments on human matting dataset show that the PA and MIoU coefficients of this algorithm model reach 96.58% and 94.48% respectively, which is better than U-net benchmark model and other mainstream semantic segmentation models.

MTIE-Net: Multi-technology fusion of low-light image enhancement network.

Images obtained in low-light scenes are often accompanied by problems such as low visibility, blurred details, and color distortion, enhancing them can effectively improve the visual effect and provide favorable conditions for advanced visual tasks. In this study, we propose a Multi-Technology Fusion of Low-light Image Enhancement Network (MTIE-Net) that modularizes the enhancement task. MTIE-Net consists of a residual dense decomposition network (RDD-Net) based on Retinex theory, an encoder-decoder denoising network (EDD-Net), and a parallel mixed attention-based self-calibrated illumination enhancement network (PCE-Net). The low-light image is first decomposed by RDD-Net into a lighting map and reflectance map; EDD-Net is used to process noise in the reflectance map; Finally, the lighting map is fused with the denoised reflectance map as an input to PCE-Net, using the Fourier transform for illumination enhancement and detail recovery in the frequency domain. Numerous experimental results show that MTIE-Net outperforms the comparison methods in terms of image visual quality enhancement improvement, denoising, and detail recovery. The application in nighttime face detection also fully demonstrates its promise as a pre-processing means in practical applications.

FOLD: Low-Level Image Enhancement for Low-Light Object Detection Based on FPGA MPSoC

Object detection has a wide range of applications as the most fundamental and challenging task in computer vision. However, the image quality problems such as low brightness, low contrast, and high noise in low-light scenes cause significant degradation of object detection performance. To address this, this paper focuses on object detection algorithms in low-light scenarios, carries out exploration and research from the aspects of low-light image enhancement and object detection, and proposes low-level image enhancement for low-light object detection based on the FPGA MPSoC method. On the one hand, the low-light dataset is expanded and the YOLOv3 object detection model is trained based on the low-order image enhancement technique, which improves the detection performance of the model in low-light scenarios; on the other hand, the model is deployed on the MPSoC board to achieve an edge object detection system, which improves the detection efficiency. Finally, validation experiments are conducted on the publicly available low-light object detection dataset and the ZU3EG-AXU3EGB MPSoC board, and the results show that the method in this paper can effectively improve the detection accuracy and efficiency.

Disentangling Modality and Posture Factors: Memory-Attention and Orthogonal Decomposition for Visible-Infrared Person Re-Identification.

Striving to match the person identities between visible (VIS) and near-infrared (NIR) images, VIS-NIR reidentification (Re-ID) has attracted increasing attention due to its wide applications in low-light scenes. However, owing to the modality and pose discrepancies exhibited in heterogeneous images, the extracted representations inevitably comprise various modality and posture factors, impacting the matching of cross-modality person identity. To solve the problem, we propose a disentangling modality and posture factors (DMPFs) model to disentangle modality and posture factors by fusing the information of features memory and pedestrian skeleton. Specifically, the DMPF comprises three modules: three-stream features extraction network (TFENet), modality factor disentanglement (MFD), and posture factor disentanglement (PFD). First, aiming to provide memory and skeleton information for modality and posture factors disentanglement, the TFENet is designed as a three-stream network to extract VIS-NIR image features and skeleton features. Second, to eliminate modality discrepancy across different batches, we maintain memory queues of previous batch features through the momentum updating mechanism and propose MFD to integrate features in the whole training set by memory-attention layers. These layers explore intramodality and intermodality relationships between features from the current batch and memory queues under the optimization of the optimal transport (OT) method, which encourages the heterogeneous features with the same identity to present higher similarity. Third, to decouple the posture factors from representations, we introduce the PFD module to learn posture-unrelated features with the assistance of the skeleton features. Besides, we perform subspace orthogonal decomposition on both image and skeleton features to separate the posture-related and identity-related information. The posture-related features are adopted to disentangle the posture factors from representations by a designed posture-features consistency (PfC) loss, while the identity-related features are concatenated to obtain more discriminative identity representations. The effectiveness of DMPF is validated through comprehensive experiments on two VIS-NIR pedestrian Re-ID datasets.

LRE-Net: A Vision-Based real-time rail extraction method under Metro tunnel low-light scene

Vision-based methods for detecting the dynamic clearance intrusion of trains in underground tunnels encounter challenges in extracting rails from low-contrast images caused by low-light scenes. This study introduces a solution for Rail Extraction in the Low-light scenes of underground tunnels (LRE-Net). Exposure features were integrated into the backbone to enhance the capability of the model in capturing global features in low-light scenes. Additionally, this study proxies the track extraction as a row-grid-based classification formulation. Two auxiliary tasks were introduced as regularization terms for generic feature learning during the training phase to address the potential overfitting resulting from the sparse nature of the track-extraction task. Experiments were conducted on a dataset consisting of 7,643 images, demonstrating that LRE-Net achieved the highest extraction accuracy of 93.94% at 88 frames per second (FPS). Furthermore, the impact of noise in low-light images on track extraction was analyzed, and strategies were proposed to enhance accuracy.

A Collaborative Learning-based Urban Low-light Small-target Face Image Enhancement Method

Face recognition is an essential technology in intelligent transportation and security within smart cities. Nevertheless, face images taken in nighttime urban environments often suffer from low brightness, small sizes, and low resolution, which pose significant challenges for accurate face feature recognition. To address this issue, we propose the Low-light Small-target Face Enhancement (LSFE) method, a collaborative learning-based image brightness enhancement approach specifically designed for small-target faces in low-light environments. LSFE employs a multilevel feature stratification module to acquire detailed face image features at different levels, revealing hidden facial image information within the dark. In addition, we design a network combining collaborative learning and self-attention mechanisms, which effectively captures long-distance pixel dependencies in low-brightness face images and enhances their brightness in a stepwise manner. The enhanced feature maps are then fused through a branch fusion module. Experimental results demonstrate that LSFE can more effectively enhance the luminance of small-target face images in low-light scenes while retaining more visual information, compared to other existing methods.

Illumination-Adaptive Unpaired Low-Light Enhancement

Supervised networks address the task of low-light enhancement using paired images. However, collecting a wide variety of low-light/clean paired images is tedious as the scene needs to remain static during imaging. In this paper, we propose an unsupervised low-light enhancement network using context-guided illumination-adaptive norm (CIN). Inspired by coarse to fine methods, we propose to address this task in two stages. In stage-I, a pixel amplifier module (PAM) is used to generate a coarse estimate with an overall improvement in visibility and aesthetic quality. Stage-II further enhances the saturated dark pixels and scene properties of the image using CIN. Different ablation studies show the importance of PAM and CIN in improving the visible quality of the image. Next, we propose a region-adaptive single input multiple output (SIMO) model that can generate multiple enhanced images from a single low-light image. The objective of SIMO is to let users choose the image of their liking from a pool of enhanced images. Human subjective analysis of SIMO results shows that the distribution of preferred images varies, endorsing the importance of SIMO-type models. Lastly, we propose a low-light road scene (LLRS) dataset having an unpaired collection of low-light and clean scenes. Unlike existing datasets, the clean and low-light scenes in LLRS are real and captured using fixed camera settings. Exhaustive comparisons on publicly available datasets, and the proposed dataset reveal that the results of our model outperform prior art quantitatively and qualitatively.

SCFusion: Infrared and Visible Fusion Based on Salient Compensation.

The aim of infrared and visible image fusion is to integrate the complementary information of the two modalities for high-quality fused images. However, many deep learning fusion algorithms have not considered the characteristics of infrared images in low-light scenes, leading to the problems of weak texture details, low contrast of infrared targets and poor visual perception in the existing methods. Therefore, in this paper, we propose a salient compensation-based fusion method that makes sufficient use of the characteristics of infrared and visible images to generate high-quality fused images under low-light conditions. First, we design a multi-scale edge gradient module (MEGB) in the texture mainstream to adequately extract the texture information of the dual input of infrared and visible images; on the other hand, the salient tributary is pre-trained by salient loss to obtain the saliency map based on the salient dense residual module (SRDB) to extract salient features, which is supplemented in the process of overall network training. We propose the spatial bias module (SBM) to fuse global information with local information. Finally, extensive comparison experiments with existing methods show that our method has significant advantages in describing target features and global scenes, the effectiveness of the proposed module is demonstrated by ablation experiments. In addition, we also verify the facilitation of this paper's method for high-level vision on a semantic segmentation task.

Zero-referenced low-light image enhancement with adaptive filter network

The current strive toward efficient intelligent visual systems suffers from challenges in the task of low-light image enhancement. To improve image perception, the low-light scenes in different illumination conditions must be properly focused. However, typical CNN-based methods use the same set of parameters for all images, which limits the capability for handling complex scenes. Meanwhile, the existing deep models integrate the low-level and high-level features by simply adding or concatenating operations, lacking unique designs for the low-light image enhancement task. To address the above challenges, we propose a zero-referenced adaptive filter network (ZAFN) for low-light image enhancement. Specifically, the adaptive filters are generated by the integration of high-level contents from multiple partial scenes. The iterative enlightening process is then conducted using the low-level features that are dynamically modulated with the adaptive filters. To alleviate the requirement of paired training data and enable zero-referenced learning, we propose a color enhancement loss, a global consistency loss, and a self-regularized denoising loss for high-quality results. Our ZAFN model, which has a light model size and low computational cost, outperforms other state-of-the-art zero-referenced methods on four popular datasets.

Study on a Low-Illumination Enhancement Method for Online Monitoring Images Considering Multiple-Exposure Image Sequence Fusion

In order to improve the problem of low image quality caused by insufficient illumination, a low-light image enhancement method with robustness is proposed, which can effectively handle extremely dark images while achieving good results for scenes with insufficient local illumination. First, we expose the images to different degrees to form a multi-exposure image sequence; then, we introduce global-based luminance weights and contrast-based gradient weights to fuse the multi-exposure image sequence; finally, we use a bootstrap filter to suppress the noise that may occur during the image processing. We employ pertinent assessment criteria, such as the Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), the Average Gradient (AG), and the Figure Definition (FD), to assess how well the method enhances. Experimental results show that PSNR (31.32) and SSIM (0.74) are the highest in pretty dark scenes compared to most conventional algorithms such as MF, BIMEF, LECARM, etc. Similarly, in processing uneven illumination such as “moonlit night” images, the AG (10.21) and the FD (14.54) are at maximum. In addition, other evaluation metrics such as Shannon (SH) are optimal in the above scenarios. In addition, we apply the algorithm in this paper to the online monitoring images of electric power equipment, which can improve the image lightness while recovering the detail information. The algorithm has strong robustness in extremely dark images and natural low-light images, and the enhanced images have minimal distortion and best appearance in different low-light scenes.

Global guidance-based integration network for salient object detection in low-light images

Most of current salient object detection (SOD) methods focus on well-lit scenes, and their performance drops when generalized into low-light scenes due to limitations such as blurred boundaries and low contrast. To solve this problem, we propose a global guidance-based integration network (G2INet) customized for low-light SOD. First, we propose a Global Information Flow (GIF) to extract comprehensive global information, for guiding the fusion of multi-level features. To facilitate information integration, we design a Multi-level features Cross Integration (MCI) module, which progressively fuses low-level details, high-level semantics, and global information by interweaving. Furthermore, a U-shaped Attention Refinement (UAR) module is proposed to further refine edges and details for accurate saliency predictions. In terms of five metrics, extensive experimental results demonstrate that our method outperforms the existing twelve state-of-the-art models.

Smoke Image Segmentation Algorithm Suitable for Low-Light Scenes

The real-time monitoring and analysis system based on video images has been implemented to detect fire accidents on site. While most segmentation methods can accurately segment smoke areas in bright and clear images, it becomes challenging to obtain high performance due to the low brightness and contrast of low-light smoke images. An image enhancement model cascaded with a semantic segmentation model was proposed to enhance the segmentation effect of low-light smoke images. The modified Cycle-Consistent Generative Adversarial Network (CycleGAN) was used to enhance the low-light images, making smoke features apparent and improving the detection ability of the subsequent segmentation model. The smoke segmentation model was based on Transformers and HRNet, where semantic features at different scales were fused in a dense form. The addition of attention modules of spatial dimension and channel dimension to the feature extraction units established the relationship mappings between pixels and features in the two-dimensional spatial directions, which improved the segmentation ability. Through the Foreground Feature Localization Module (FFLM), the discrimination between foreground and background features was increased, and the ability of the model to distinguish the thinner positions of smoke edges was improved. The enhanced segmentation method achieved a segmentation accuracy of 91.68% on the self-built dataset with synthetic low-light images and an overall detection time of 120.1 ms. This method can successfully meet the fire detection demands in low-light environments at night and lay a foundation for expanding the all-weather application of initial fire detection technology based on image analysis.

Combining Low-Light Scene Enhancement for Fast and Accurate Lane Detection

Lane detection is a crucial task in the field of autonomous driving, as it enables vehicles to safely navigate on the road by interpreting the high-level semantics of traffic signs. Unfortunately, lane detection is a challenging problem due to factors such as low-light conditions, occlusions, and lane line blurring. These factors increase the perplexity and indeterminacy of the lane features, making them hard to distinguish and segment. To tackle these challenges, we propose a method called low-light enhancement fast lane detection (LLFLD) that integrates the automatic low-light scene enhancement network (ALLE) with the lane detection network to improve lane detection performance under low-light conditions. Specifically, we first utilize the ALLE network to enhance the input image’s brightness and contrast while reducing excessive noise and color distortion. Then, we introduce symmetric feature flipping module (SFFM) and channel fusion self-attention mechanism (CFSAT) to the model, which refine the low-level features and utilize more abundant global contextual information, respectively. Moreover, we devise a novel structural loss function that leverages the inherent prior geometric constraints of lanes to optimize the detection results. We evaluate our method on the CULane dataset, a public benchmark for lane detection in various lighting conditions. Our experiments show that our approach surpasses other state of the arts in both daytime and nighttime settings, especially in low-light scenarios.

UID2021: An Underwater Image Dataset for Evaluation of No-Reference Quality Assessment Metrics

Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing. However, the development of underwater image quality assessment (UIQA) is limited for the lack of publicly available underwater image datasets with human subjective scores and reliable objective UIQA metrics. To address this issue, we establish a large-scale underwater image dataset, dubbed UID2021, for evaluating no-reference (NR) UIQA metrics. The constructed dataset contains 60 multiply degraded underwater images collected from various sources, covering six common underwater scenes (i.e., bluish scene, blue-green scene, greenish scene, hazy scene, low-light scene, and turbid scene), and their corresponding 900 quality improved versions are generated by employing 15 state-of-the-art underwater image enhancement and restoration algorithms. Mean opinion scores with 52 observers for each image of UID2021 are also obtained by using the pairwise comparison sorting method. Both in-air and underwater-specific NR IQA algorithms are tested on our constructed dataset to fairly compare their performance and analyze their strengths and weaknesses. Our proposed UID2021 dataset enables ones to evaluate NR UIQA algorithms comprehensively and paves the way for further research on UIQA. The dataset is available at https://github.com/Hou-Guojia/UID2021 .