Low Illumination Environment Research Articles

Mapping with Monocular Camera Sensor under Adversarial Illumination for Intelligent Vehicles.

High-precision maps are widely applied in intelligent-driving vehicles for localization and planning tasks. The vision sensor, especially monocular cameras, has become favoured in mapping approaches due to its high flexibility and low cost. However, monocular visual mapping suffers from great performance degradation in adversarial illumination environments such as on low-light roads or in underground spaces. To address this issue, in this paper, we first introduce an unsupervised learning approach to improve keypoint detection and description on monocular camera images. By emphasizing the consistency between feature points in the learning loss, visual features in dim environment can be better extracted. Second, to suppress the scale drift in monocular visual mapping, a robust loop-closure detection scheme is presented, which integrates both feature-point verification and multi-grained image similarity measurements. With experiments on public benchmarks, our keypoint detection approach is proven robust against varied illumination. With scenario tests including both underground and on-road driving, we demonstrate that our approach is able to reduce the scale drift in reconstructing the scene and achieve a mapping accuracy gain of up to 0.14 m in textureless or low-illumination environments.

Research on target detection method of underwater robot in low illumination environment

Research on target detection method of underwater robot in low illumination environment

A Robust Learned Feature-Based Visual Odometry System for UAV Pose Estimation in Challenging Indoor Environments

Unmanned Aerial Vehicles (UAVs) are becoming popular nowadays due to their versatility and flexibility for indoor applications, such as the autonomous visual inspection for the inner surface of a pressure vessel. Nevertheless, robust and reliable position estimation is critical for completing these tasks. Visual Odometry (VO) and Visual Simultaneous Localisation and Mapping (VSLAM) allow the UAV to estimate its position in unknown environments. However, traditional feature-based VO/VSLAM systems struggle to deal with complex scenes such as low illumination and textureless environments. Replacing the traditional features with deep learning-based features provides the advantage of handling the challenging environment, but the efficiency is ignored. In this work, an efficient VO system based on a novel lightweight feature extraction network for UAV onboard platforms has been developed. The Deformable Convolution (DFConv) is utilised to improve the feature extraction capability. Owing to the limited onboard computing capability, the Depth-wise Separable Convolution (DWConv) is adopted to calculate the offsets for the deformable convolution and construct the backbone network to improve the feature extraction efficiency. Experiments on public datasets indicate that the efficiency of the VO system is improved by 30.03% while preserving the accuracy on embedded platforms with the feature points and descriptors detected by the proposed Convolutional Neural Network (CNN). Moreover, the proposed VO system is verified through UAV flying tests in a real-world scenario. The results prove that the proposed VO system is able to handle the challenging environments where both the latest traditional and deep learning feature-based VO/VSLAM systems fail, and it is feasible for UAV self-localisation and autonomous navigation in the confined, low illumination and textureless indoor environment.

Image Enhancement and Brightness Equalization Algorithms in Low Illumination Environment Based on Multiple Frame Sequences

Image Enhancement and Brightness Equalization Algorithms in Low Illumination Environment Based on Multiple Frame Sequences

Tracking System for a Coal Mine Drilling Robot for Low-Illumination Environments

In recent years, discriminative correlation filters (DCF) based trackers have been widely used in mobile robots due to their efficiency. However, underground coal mines are typically a low illumination environment, and tracking in this environment is a challenging problem that has not been adequately addressed in the literature. Thus, this paper proposes a Low-illumination Long-term Correlation Tracker (LLCT) and designs a visual tracking system for coal mine drilling robots. A low-illumination tracking framework combining image enhancement strategies and long-time tracking is proposed. A long-term memory correlation filter tracker with an interval update strategy is utilized. In addition, a local area illumination detection method is proposed to prevent the failure of the enhancement algorithm due to local over-exposure. A convenient image enhancement method is proposed to boost efficiency. Extensive experiments on popular object tracking benchmark datasets demonstrate that the proposed tracker significantly outperforms the baseline trackers, achieving high real-time performance. The tracker’s performance is verified on an underground drilling robot in a coal mine. The results of the field experiment demonstrate that the performance of the novel tracking framework is better than that of state-of-the-art trackers in low-illumination environments.

A snapshot near-infrared hyperspectral demosaicing method with convolutional neural networks in low illumination environment

Snapshot near-infrared hyperspectral imaging technology based on multispectral color filter array has been widely utilized in quantitative and visualized analysis. Hyperspectral demosaicing method is a key part to increase spatial resolution for the snapshot hyperspectral imaging. However, the existing demosaicing methods will be disturbed by noise in case of the low illumination. In this paper, a snapshot hyperspectral demosaicing method is proposed to reconstruct raw mosaic images into full-resolution hyperspectral images. The demosaicing process is integrated with Convolutional Neural Network (CNN) to extract features automatically, guaranteeing a good performance and strong robustness without parameter adjustment. We experimentally validate the merits of the method by 5 × 5 raw mosaic images. The Peak Signal-to-Noise Ratio (PSNR) and Spectral Angle Mapper (SAM) values of reconstructed hyperspectral images can be improved by the proposed method, which has been successfully used in commercial hyperspectral cameras.

Improved Ship Object Detection in Low-Illumination Environments Using RetinaMFANet

Video-based ship object detection has long been a popular research issue that has received attention in the water transportation industry. However, in low-illumination environments, such as at night or in fog, the water environment has a complex variety of light sources, video surveillance images are often accompanied by noise, and information on the details of objects in images is worsened. These problems cause high rates of false detection and missed detection when performing object detection for ships in low-illumination environments. Thus, this paper takes the detection of ship objects in low-illumination environments at night as the research object. The technical difficulties faced by object detection algorithms in low-illumination environments are analyzed, and a dataset of ship images is constructed by collecting images of ships (in the Nanjing section of Yangtze River in China) in low-illumination environments. In view of the outstanding performance of the RetinaNet model in general object detection, a new multiscale feature fusion network structure for a feature extraction module is proposed based on the same network architecture, in such a way that the extraction of more potential feature information from low-illumination images can be realized. In line with the feature detection network, the regression and classification detection network for anchor boxes is improved by means of the attention mechanism, guiding the network structure in the detection of object features. Moreover, the design and optimization of the augmentation of multiple random images and prior bounding boxes in the training process are also carried out. Finally, on the basis of experimental validation analysis, the optimized detection model was able to improve ship detection accuracy by 3.7% with a limited decrease in FPS (frames per second), and has better results in application.

Low Illumination Soybean Plant Reconstruction and Trait Perception

Agricultural equipment works poorly under low illumination such as nighttime, and there is more noise in soybean plant images collected under light constraints, and the reconstructed soybean plant model cannot fully and accurately represent its growth condition. In this paper, we propose a low-illumination soybean plant reconstruction and trait perception method. Our method is based on low-illumination enhancement, using the image enhancement algorithm EnlightenGAN to adjust soybean plant images in low-illumination environments to improve the performance of the scale-invariant feature transform (SIFT) algorithm for soybean plant feature detection and matching and using the motion recovery structure (SFM) algorithm to generate the sparse point cloud of soybean plants, and the point cloud of the soybean plants is densified by the face slice-based multi-view stereo (PMVS) algorithm. We demonstrate that the reconstructed soybean plants are close to the growth conditions of real soybean plants by image enhancement in challenging low-illumination environments, expanding the application of three-dimensional reconstruction techniques for soybean plant trait perception, and our approach is aimed toward achieving the accurate perception of current crop growth conditions by agricultural equipment under low illumination.

A Novel Deep Neural Network Method for HAR-Based Team Training Using Body-Worn Inertial Sensors.

Human activity recognition (HAR) became a challenging issue in recent years. In this paper, we propose a novel approach to tackle indistinguishable activity recognition based on human wearable sensors. Generally speaking, vision-based solutions struggle with low illumination environments and partial occlusion problems. In contrast, wearable inertial sensors can tackle this problem and avoid revealing personal privacy. We address the issue by building a multistage deep neural network framework that interprets accelerometer, gyroscope, and magnetometer data that provide useful information of human activities. Initially, the stage of variational autoencoders (VAE) can extract the crucial information from raw data of inertial measurement units (IMUs). Furthermore, the stage of generative adversarial networks (GANs) can generate more realistic human activities. Finally, the transfer learning method is applied to enhance the performance of the target domain, which builds a robust and effective model to recognize human activities.

Pose Measurement Technology of Roadheader Body based on Fusion of Visual and SINS*

In order to solve the problem of pose measurement of roadheader body in low illumination and high dust environment in coal mine, a pose measurement technology of roadheader body based on visual and strapdown inertial navigation system (SINS) is proposed. By extracting the point and line features and using the monocular vision positioning principle based on the gate three lines, the spatial pose of the roadheader body measured by vision is obtained according to the spatial matrix transformation. At the same time, the spatial pose of the roadheader body measured by SINS is calculated by gyroscope and accelerometer. Then the least square method is used to align the two-sensor data, and the registration data is fused based on the extended Kalman filter fusion algorithm to obtain the final fusion roadheader pose data. The experimental results show that the measurement error of the roadheader body position in the x axis direction of the roadway coordinate system is less than ± 20 mm, the error of the y axis direction is less than ±35 mm, the error of the z axis direction is less than ± 10 mm, the yaw angle error is between-0.4° to 0.7°, the roll angle error is between -0.2° and 0.6°, and the pitch angle error is between -0.02° and 0.5°. This method can be used to measure the pose of the roadheader stably and accurately in the complex environment of high dust, miscellaneous light and low illumination.

Application of YOLOv4 Algorithm for Foreign Object Detection on a Belt Conveyor in a Low-Illumination Environment

The most common failures of belt conveyors are runout, coal piles and longitudinal tears. The detection methods for longitudinal tearing are currently not particularly effective. A key study area for minimizing longitudinal belt tears with the advancement of machine learning is how to use machine vision technology to detect foreign items on the belt. In this study, the real-time detection of foreign items on belt conveyors is accomplished using a machine vision method. Firstly, the KinD++ low-light image enhancement algorithm is used to improve the quality of the captured low-quality images through feature processing. Then, the GridMask method partially masks the foreign objects in the training images, thus extending the data set. Finally, the YOLOv4 algorithm with optimized anchor boxes is combined to achieve efficient detection of foreign objects in belt conveyors, and the method is verified as effective.

A More Effective Zero-DCE Variant: Zero-DCE Tiny

The purpose of Low Illumination Image Enhancement (LLIE) is to improve the perception or interpretability of images taken in low illumination environments. This work inherits the work of Zero-Reference Deep Curve Estimation (Zero-DCE) and proposes a more effective image enhancement model, Zero-DCE Tiny. First, the new model introduces the Cross Stage Partial Network (CSPNet) into the original U-net structure, divides basic feature maps into two parts, and then recombines it through the structure of cross-phase connection to achieve a richer gradient combination with less computation. Second, we replace all the deep separable convolutions except the last layer with Ghost modules, which makes the network lighter. Finally, we introduce the channel consistency loss into the non-reference loss, which further strengthens the constraint on the pixel distribution of the enhanced image and the original image. Experiments show that compared with Zero-DCE++, the network proposed in this work is more lightweight and surpasses the Zero-DCE++ method in some important image enhancement evaluation indexes.

Lighting the darkness in the sea: A deep learning model for underwater image enhancement

Currently, optical imaging cameras are widely used on underwater vehicles to obtain images and support numerous marine exploration tasks. Many underwater image enhancement algorithms have been proposed in the past few years to suppress backscattering noise and improve the signal-to-noise ratio of underwater images. However, these algorithms are mainly focused on underwater image enhancement tasks in a bright environment. Thus, it is still unclear how these algorithms would perform on images acquired in an underwater scene with low illumination. Images obtained in a dark underwater scene usually include more noise and have very low visual quality, which may easily lead to artifacts during the process of enhancement. To bridge this gap, we thoroughly study the existing underwater image enhancement methods and low illumination image enhancement methods based on deep learning and propose a new underwater image enhancement network to solve the problem of serious degradation of underwater image quality in a low illumination environment. Due to the lack of ready-made datasets for training, we also propose the first dataset for low-light underwater image enhancement to train our model. Our method can be implemented to skillfully and simultaneously address low-light degradation and scattering degradation in low-light underwater images. Experimental results also show that our method is robust against different illumination levels, which greatly expands the applicable scenarios of our method. Compared with previous underwater image enhancement methods and low-light image enhancement methods, outstanding performance is achieved using our method in various low-light underwater scenes.

Weighted combination and singular spectrum analysis based remote photoplethysmography pulse extraction in low-light environments.

Camera-based vital signs monitoring in recent years has attracted more and more researchers and the results are promising. However, a few research works focus on heart rate extraction under extremely low illumination environments. In this paper, we propose a novel framework for remote heart rate estimation under low-light conditions. This method uses singular spectrum analysis (SSA) to decompose the filtered signal into several reconstructed components. A spectral masking algorithm is utilized to refine the preliminary candidate components on the basis of a reference heart rate. The contributive components are fused into the final pulse signal. To evaluate the performance of our framework in low-light conditions, the proposed approach is tested on a large-scale multi-illumination HR dataset (named MIHR). The test results verify that the proposed method has stronger robustness to low illumination than state-of-the-art methods, effectively improving the signal-to-noise ratio and heart rate estimation precision. We further perform experiments on the PUlse RatE detection (PURE) dataset which is recorded under normal light conditions to demonstrate the generalization of our method. The experiment results show that our method can stably detect pulse rate and achieve comparative results. The proposed method pioneers a new solution to the remote heart rate estimation in low-light conditions.

Dynamic distribution and correlation analysis of the angle kappa in myopia patients undergoing femtosecond-assisted laser in situ keratomileusis.

Purpose:To explore the offset distribution of pupillary centres, the offset between the pupil centre and the coaxially sighted corneal light reflex (P-Dist) and their correlation in femtosecond laser combined with excimer laser in situ keratomileusis.Methods:Randomly selected 194 patients (398 eyes) who underwent femtosecond-assisted laser in situ keratomileusis with preoperative use of WaveLight Allegro Topolyzer Corneal Topography (WaveLight Laser Technologies AG, Erlangen, Germany) to measure the pupil size and centre position. The P-Dist of the patients was recorded by the X and Y axis eyeball tracking adjustment program of the WaveLight Eagle Vision EX500 excimer laser system.Results:The P-Dist was 0.214 ± 0.092 mm in the right eyes and 0.228 ± 0.105 mm in the left eyes (P = .041). Under scotopic conditions, the pupil centre of left eye X-axis was −0.046 ± 0.091 mm, the right eye was −0.152 ± 0.084 mm, with significant differences (P = .015), and the Y-axis direction showed no significant changes (P = .062). The white to white was positively correlated with changes of pupil diameter (scotopic pupil diameter-photopic pupil diameter) (r = 0.270, P < .001). The equivalent spherical mirror and measured centroid shift were negatively correlated (r = −0.214, P = .002).Conclusion:The angle kappa of the right eye is smaller than that of the left eye and from scotopic to photopic condition, the pupil centroid shift of both eyes to the nasal inferior side. If the cornea is too large, the low illumination environment should be maintained during the operation to improve the efficiency of pupil matching.

Research on low illumination coal gangue image enhancement based on improved Retinex algorithm

ABSTRACT Aiming at the problems of coal and gangue image recognition in low illumination or high dust concentration environment, an adaptive enhancement method based on improved Retinex algorithm, is proposed: H-GF-MSR algorithm. The algorithm is processed in HSV color space, uses guided filter for multi-scale Retinex algorithm in brightness component, and carries out adaptive saturation stretching in saturation component. Then the images are processed by histogram equalization, and the two algorithms are fused to improve the accuracy. Compared with SSR algorithm, MSRCR algorithm and other classical algorithms, this algorithm is improved in suppressing the generation of halo and image edge enhancement. The experimental results show that in terms of image evaluation indexes such as information entropy, image mean, image average gradient and peak signal-to-noise ratio, compared with SSR algorithm, the algorithm improves by 5.4%, 41.1%, 33.2% and 0.4% respectively; compared with MSRCR algorithm, the average improvement is 2.9%, 32.4%, 31.7% and 0.7% respectively. Through the enhancement experiments of coal gangue image in the illumination range of 10 lux ~ 200 lux, the algorithm has better enhancement effect. It can effectively suppress the bright light and restore the original details of the object.

A Survey on Face and Body Based Human Recognition Robust to Image Blurring and Low Illumination

Many studies have been actively conducted on human recognition in indoor and outdoor environments. This is because human recognition methods in such environments are closely related to everyday life situations. Besides, these methods can be applied for finding missing children and identifying criminals. Methods for human recognition in indoor and outdoor environments can be classified into three categories: face-, body-, and gait-based methods. There are various factors that hinder indoor and outdoor human recognition, for example, blurring of captured images, cutoff in images due to the camera angle, and poor recognition in images acquired in low-illumination environments. Previous studies conducted to solve these problems focused on facial recognition only. This is because the face is typically assumed to contain more important information for human recognition than the body. However, when a human face captured by a distant camera is small, or even impossible to identify with the naked eye, the body’s information can help with recognition. For this reason, this survey paper reviews both face- and body-based human recognition methods. In previous surveys, recognition on low-resolution images were reviewed. However, survey papers on blurred images are not comprehensive. Therefore, in this paper, we review studies on blurred image restoration in detail by classifying them based on whether deep learning was used and whether the human face and body were combined. Although previous survey papers on recognition covered low-illumination environments as well, they excluded deep learning methods. Therefore, in this survey, we also include details on deep-learning-based low-illumination image recognition methods. We aim to help researchers who will study related fields in the future.

Photon-limited imaging through scattering medium based on speckle coding

Photon-limited imaging a target hidden behind a scattering medium is a hot technology at present, and it has important applications in many fields. A basic problem is that the target information is annihilated by noise from the speckle captured under weak illumination conditions, which makes it difficult to reconstruct the target. In order to solve this problem, we propose a highly sensitive approach to image through scattering media based on encoding speckles. Under a light level of 0.1 Lux, the target passing through the scattering medium is encoded multiple times by the Gaussian matrix, which can better extract the effective information in speckles. Theoretically, increasing the number of encoding times can improve the ability of information extraction, but also cause time consuming. But our method only needs to encode 39.1% of the full sampling to recover the original target, which effectively improves the encoding efficiency. In addition, Our algorithm has perfect noise resistance, when artificial noise is added to the data , the average peak signal to noise ratio (PSNR) of the algorithm can get 18.16 dB, and the average structural similarity index measure (SSIM) reaches more than 0.65. We also experimentally verify the effects of the algorithm with different network parameters on target reconstruction. The strong reconstruction ability of the proposed method in a low illumination environment can provide a basis for practical application of the proposed method.

Clarity method of fog and dust image in fully mechanized mining face

At present, the abnormal state of equipment and surrounding rocks in the fully mechanized mining face is mainly detected by visual methods. However, the vision sensor works in a low-light environment and it is affected by factors such as water fog and dust, which lead to blurred images. The defogging algorithm of image based on boundary constraint and context regularization has a good effect on image restoration in the daily environment, but the recovery quality is poor in low illumination environment. Therefore, a method based on boundary constraint and nonlinear context regularization is proposed. The model of fog and dust image is established, and the transmittance function is roughly estimated by boundary constraint method. Then, the nonlinear context regularization method based on logarithmic transformation is used to estimate and optimize the scene transmission model to improve the brightness of the image, and the low illumination fog and dust image is restored by the optimized transmittance function. The logarithmic transformation multiple is selected according to the peak value of image brightness. In order to highlight the effectiveness of our method, the widely used and improved Dark Channel Prior or other methods are used for comparison. The experiment results indicate that our method can effectively remove fog and dust and improve the brightness of the image of the fully mechanized face. It is of great significance to ensure safe production and safety of workers and equipment in coal mine.

Fast Matching Method of UAV Aerial Photography Enhanced Low Illumination Image

Aiming at the problems of insufficient image contrast in three-dimensional reconstruction of UAV in low illumination environment and the unstable iteration times of the RANSAC algorithm in the feature matching process, real-time matching method of UAV aerial image is proposed. First, a new image enhancement algorithm is applied to the image to enhance its quality and visibility. Second, the enhanced fast algorithm in ORB extracts the feature points from the preprocessed image, and cross-matching performs rough matching. Finally, the PROSAC algorithm solves the homography matrix by selecting the highest quality interior points from the extracted feature points. To improve the matching accuracy, some exterior points that do not conform to the geometric characteristics of the image are removed based on the homography matrix and the set mismatch threshold. The results show that the improved ORB algorithm is applied to the low illumination environment of UAV aerial photography, the image matching accuracy in 3D reconstruction is improved, and the correct matching rate tends to 97.24~99.39%. The relevant research findings and conclusions provide a fast and effective method for UAV image matching in different low illumination environments.