Nighttime Images Research Articles

Tracker Meets Night: A Transformer Enhancer for UAV Tracking

Most previous progress in object tracking is realized in daytime scenes with favorable illumination. State-of-the-arts can hardly carry on their superiority at night so far, thereby considerably blocking the broadening of visual tracking-related unmanned aerial vehicle (UAV) applications. To realize reliable UAV tracking at night, a spatial-channel Transformer-based low-light enhancer (namely SCT), which is trained in a novel task-inspired manner, is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, the novel spatial-channel attention module is proposed to model global information while preserving local context. In the enhancement process, SCT denoises and illuminates nighttime images simultaneously through a robust non-linear curve projection. Moreover, to provide a comprehensive evaluation, we construct a challenging nighttime tracking benchmark, namely DarkTrack2021, which contains 110 challenging sequences with over 100 K frames in total. Evaluations on both the public UAVDark135 benchmark and the newly constructed DarkTrack2021 benchmark show that the task-inspired design enables SCT with significant performance gains for nighttime UAV tracking compared with other top-ranked low-light enhancers. Real-world tests on a typical UAV platform further verify the practicability of the proposed approach. The DarkTrack2021 benchmark and the code of the proposed approach are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/vision4robotics/SCT</uri> .

Visibility classification and influencing-factors analysis of airport: A deep learning approach

This paper discusses the visibility classification and influencing-factor analysis of an airport, which were carried out through a data-driven deep learning approach and multiple nonlinear regression analysis. First, a deep integrated model for airport visibility classification was developed, and the relationship between runway visual range (RVR) and surface meteorological elements for different reasons was analyzed. To extract more low-level features from nighttime images, the original images were transformed into pseudo-color images, and the image pairs were input into the classification model. The proposed deep integrated model combined two popular convolutional neural network (CNN) models: VGG-16 and Xception. The results showed that the accuracy of the proposed model was 87.64%, with the F1-score being 88.58%; it achieved the best performance among other well-known experimental CNNs. Afterward, the relationships between RVR and meteorological optical range (MOR), source light intensity (I), and background light (BL) were analyzed, and the results showed that RVR increased with increasing I under the same MOR and BL conditions; however, it decreased with increasing log (BL) values under the same I and MOR conditions. RVR increased as MOR increased when other conditions remained unchanged. Subsequently, the multivariate regression analysis was conducted to fit the relationship between MOR and surface meteorological elements, and the results showed that the cubic curve was the best-fit curve with R-square values of 0.937 and 0.860 in December 2019 and March 2020, respectively. Finally, combined with the analysis of the factors influencing the RVR, a significant negative correlation was observed with relative humidity, along with a significant positive correlation with temperature and a positive correlation with average wind speed in the winter of 2019 and spring of 2020. However, the pressure factor showed the opposite correlation, possibly because the arrival of cold air in winter increases the ground pressure, which is conducive for the dissipation of pollutants. Meanwhile, the increase in temperature and decrease in pressure in spring increases the chance of disturbances in the atmosphere, thereby increasing the likelihood of adverse visibility factors. ◆ Pseudo-color images were found beneficial for improving the accuracy of visibility identification. ◆ An integrated model combined VGG-16 and Xception achieved the best performance compared to that of other popular models. ◆ The relationship between RVR and MOR, I, and BL was found. ◆ The multivariate regression analysis between MOR and surface meteorological elements in different seasons was conducted. ◆ There is a correlational relationship between RVR and surface meteorological elements.

Joint dehazing and denoising for single nighttime image via multi-scale decomposition

Joint dehazing and denoising for single nighttime image via multi-scale decomposition

Non-Uniform Illumination and Low Light Image Enhancement Techniques: An Exhaustive Study

Abstract: Different kinds of Images captured are an important medium to represent meaningful information. It can be problematic for artificial intelligence, computer vision techniques and detection algorithms to extract valuable information from those images with poor lighting. In this paper, a study of low illumination-based low-light night image enhancement techniques are presented which work on reflectance, degradation, unsatisfactory lightings, noise, limited range visibility, low contrast, color variations, illumination, color distortion, and quality is reduced. Improving the images in low light conditions is a prerequisite in many fields, such as surveillance systems, road safety and inland waterway transport, object tracking, scientific research, the detection system, the counting system and the navigation system. Low-illumination or night image enhancement algorithms can advance the visual quality of low-light images and these images can be used in many practical application’s artificial intelligence and computer vision techniques. The methods used for enhancement of low illumination must perform, preserving details, contrast improvement, color correction, noise reduction, image enhancement, restoration, etc. Keywords: Image Enhancement, Low Illumination, Reflectance, Low Contrast, Low Light Images, Night Time Images, Low Visibility Images.

An Effective Framework for Enhancement of Hazed and Low-Illuminated Images

Abstract: Haze removal is important for computer photography and computer vision applications. However, most of the existing methods for removing theha- ziness are designed for daytime images and may not always work well at hazy night images. Unlike image conditions during the sunny day, images captured in winter night conditions can suffer from irregular lighting due to artificial light sources with varying colors and non-uniform illumination, which show low brightness, contrast and color distortion. In this paper, we propose a new frame- work for presenting night-time hazy imaging, which works on haze removal and low-illumination correction algorithm taking into consideration both the non-uni- form illumination of artificial light sources and the effects of dispersion and at- tenuation of fog. Therefore, firstly, we will give a hazy low-illuminated image having low light as input and then apply a technique to clarify the visibility of the input image. Then, apply the contrast enhancement and after that apply the LIME technique and finally, apply the white balance technique and we will get our improved output image. The experimental results show that the proposed algorithm can achieve an illumination balance, results without haziness and good color cor- rection capacity. Keywords: Image Enhancement, Low Illumination, Reflectance, Low Contrast, Low Light Images, Nighttime Images, Low Visibility Images, Nighttime Haze Removal.

Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization

Visual localization is a crucial component in the application of mobile robot and autonomous driving. Image retrieval is an efficient and effective technique in image-based localization methods. Due to the drastic variability of environmental conditions, e.g. illumination, seasonal and weather changes, retrieval-based visual localization is severely affected and becomes a challenging problem. In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner. The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models without and with Grad-SAM loss. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. The strong generalization ability of our approach is verified on RobotCar dataset using models pre-trained on urban part of CMU-Seasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision, especially under the challenging environments with illumination variance, vegetation and night-time images. The code and pretrained models are available on https://github.com/HanjiangHu/DISAM.

Real-time image-based air quality estimation by deep learning neural networks

Air quality profoundly impacts public health and environmental equity. Efficient and inexpensive air quality monitoring instruments could be greatly beneficial for human health and air pollution control. This study proposes an image-based deep learning model (CNN−RC) that integrates a convolutional neural network (CNN) and a regression classifier (RC) to estimate air quality at areas of interest through feature extraction from photos and feature classification into air quality levels. The models were trained and tested on datasets with different combinations of the current image, the baseline image, and HSV (hue, saturation, value) statistics for increasing model reliability and estimation accuracy. A total of 3549 hourly air quality datasets (including photos, PM2.5, PM10, and the air quality index (AQI)) collected at the Linyuan air quality monitoring station of Kaohsiung City in Taiwan constituted the case study. The main breakthrough of this study is to timely produce an accurate image-based estimation of several pollutants simultaneously by using only one single deep learning model. The test results show that estimation accuracy in terms of R2 for PM2.5, PM10, and AQI based on daytime (nighttime) images reaches 76% (83%), 84% (84%), and 76% (74%), respectively, which demonstrates the great capability of our method. The proposed model offers a promising solution for rapid and reliable multi-pollutant estimation and classification based solely on captured images. This readily scalable measurement approach could address major gaps between air quality data acquired from expensive instruments worldwide.

RST Analysis of Anomalous TIR Sequences in Relation with Earthquakes Occurred in Turkey in the Period 2004–2015

The paper provides, for the first time, a long-term (&gt;10 years) analysis of anomalous transients in Earth’s emitted radiation over Turkey and neighbouring regions. The RST (Robust Satellite Techniques) approach is used to identify Significant Sequences of Thermal Anomalies (SSTAs) over about 12 years (May 2004 to October 2015) of night-time MSG-SEVIRI satellite images. The correlation analysis is performed with earthquakes with M ≥ 4, which occurred in the investigated period/region within a pre-defined space-time volume around SSTA occurrences. It confirms, also for Turkey, the possibility to qualify SSTAs among the candidate parameters of a multi-parametric system for time-Dependent Assessment of Seismic Hazard (t-DASH). After analysing about 4000 images (about 400 million of single satellite records), just 155 SSTAs (about 4 every 100 images) were isolated; 115 (74% out of the total) resulted in earthquake-related (false-positive rate 26%). Results of the error diagram confirms a non-casual correlation between RST-based SSTAs and earthquake occurrences, with probability gain values up to 2.2 in comparison with the random guess. The analysis, separately performed on Turkish areas characterized by different faults and earthquakes densities, demonstrates the SSTA correlation with a dynamic seismicity more than with static tectonic settings.

A Variation-Based Nighttime Image Dehazing Flow With a Physically Valid Illumination Estimator and a Luminance-Guided Coloring Model

A Variation-Based Nighttime Image Dehazing Flow With a Physically Valid Illumination Estimator and a Luminance-Guided Coloring Model

Nighttime image dehazing with a new light segmentation method and a linear image depth estimation model

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed

Variational Single Nighttime Image Haze Removal With a Gray Haze-Line Prior.

Influenced by glowing effects, nighttime haze removal is a challenging ill-posed task. Existing nighttime dehazing methods usually result in glowing artifacts, color shifts, overexposure, and noise amplification. Thus, through statistical and theoretical analyses, we propose a simple and effective gray haze-line prior (GHLP) to identify accurate hazy feature areas. This prior demonstrates that haze is concentrated on the haze line in the RGB color space and can be accurately projected into the gray component in the Y channel of the YUV color space. Based on this prior, we establish a new unified nighttime haze removal framework and then decompose a nighttime hazy image into color and gray components in the YUV color space. Glowing color correction and haze removal are two important consecutive steps in the nighttime dehazing process. The glowing color correction method is designed to separately remove glow in the color component and enhance illumination in the gray component. After obtaining a refined nighttime hazy image, we propose a new structure-aware variational framework to simultaneously estimate the inverted scene radiance and the transmission in the gray component. This approach can not only recover the high-quality nighttime scene radiance but also preserve the significant structural information and intrinsic color of the scene. Quantitative and qualitative comparisons validate the excellent effectiveness of the proposed nighttime dehazing method against previous state-of-the-art methods. In addition, the proposed approach can be extended to achieve image enhancement for inclement weather scenes, such as sandstorm scenes and extreme daytime hazy scenes.

Coloring Panchromatic Nighttime Satellite Images: Comparing the Performance of Several Machine Learning Methods

Artificial light-at-night (ALAN), emitted from the ground and visible from space, marks human presence on Earth. Since the launch of the Suomi National Polar Partnership satellite with the Visible Infrared Imaging Radiometer Suite Day/Night Band (VIIRS/DNB) onboard, global nighttime images have significantly improved; however, they remained panchromatic. Although multispectral images are also available, they are either commercial or free of charge, but sporadic. In this paper, we use several machine learning techniques, such as linear, kernel, random forest regressions, and elastic map approach, to transform panchromatic VIIRS/DBN into Red Green Blue (RGB) images. To validate the proposed approach, we analyze RGB images for eight urban areas worldwide. We link RGB values, obtained from ISS photographs, to panchromatic ALAN intensities, their pixel-wise differences, and several land-use type proxies. Each dataset is used for model training, while other datasets are used for the model validation. The analysis shows that model-estimated RGB images demonstrate a high degree of correspondence with the original RGB images from the ISS database. Yet, estimates, based on linear, kernel and random forest regressions, provide better correlations, contrast similarity and lower WMSEs levels, while RGB images, generated using elastic map approach, provide higher consistency of predictions.

“Pyramid Deep dehazing”: An unsupervised single image dehazing method using deep image prior

Single image dehazing remove haze from degraded images and recover clean scenes. Prior based Methods can achieve great results on some haze images, but their performance is limited by the handcraft prior itself. Recently, many learning-based approaches has been proposed. Most of these modules rely on matching clean and haze images for training. However, such kind of real world data can be hard to get. Also the domain shift between training and testing data may affect the results. Some unsupervised methods have been proved to work on haze scenes but they still rely on handcraft priors to guide the training. In this paper, we proposed an Unsupervised Single Image Dehazing method using internal learning based on the optical model of haze and other haze-like degradation images. A Pyramid Deep Image strategy is used to gradually generate clean background. The entire training doesn’t need any extra data or handcraft prior, and only needs the testing image itself. The proposed method is able to deal with different kinds of haze images including other haze-like degradation (like nighttime images and underwater images).

Cross-Sensor Nighttime Lights Image Calibration for DMSP/OLS and SNPP/VIIRS with Residual U-Net

Remote sensing of nighttime lights (NTL) is widely used in socio-economic studies of economic growth, urbanization, stability of power grid, environmental light pollution, pandemics and military conflicts. Currently, NTL data are collected with two sensors: (1) Operational Line-scan System (OLS) onboard the satellites from the Defense Meteorology Satellite Program (DMSP) and (2) Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the Suomi NPP (SNPP) and NOAA-20 satellites from the Joint Polar Satellite System (JPSS). However, the nighttime images acquired by these two sensors are incompatible in spatial resolution and dynamic range. To address this problem, we propose a method for the cross-sensor calibration with residual U-net convolutional neural network (CNN). The CNN produces DMSP-like NTL composites from the VIIRS annual NTL composites. The pixel radiances predicted from VIIRS are highly correlated with NTL observed with OLS (0.96 &lt; R2 &lt; 0.99). The method can be used to extend long-term series of annual NTL after the end of DMSP mission or to cross-calibrate same year NTL from different satellites to study diurnal variations.

Extending the DMSP Nighttime Lights Time Series beyond 2013

Data collected by the Defense Meteorological Satellite Program’s Operational Linescan System (DMSP-OLS) sensors have been archived and processed by the Earth Observation Group (EOG) at the National Oceanic and Atmospheric Administration (NOAA) to make global maps of nighttime images since 1994. Over the years, the EOG has developed automatic algorithms to make Stable Lights composites from the OLS visible band data by removing the transient lights from fires and fishing boats. The ephemeral lights are removed based on their high brightness and short duration. However, the six original satellites collecting DMSP data gradually shifted from day/night orbit to dawn/dusk orbit, which is to an earlier overpass time. At the beginning of 2014, the F18 satellite was no longer collecting usable nighttime data, and the focus had shifted to processing global nighttime images from Visible Infrared Imaging Radiometer Suite (VIIRS) Day/Night Band (DNB) data. Nevertheless, it was soon discovered that the F15 and F16 satellites had started collecting pre-dawn nighttime data from 2012 onwards. Therefore, the established algorithms of the previous years were extended to process OLS data from 2013 onwards. Moreover, the existence of nighttime data from three overpass times for the year 2013–DMSP satellites F18 and F15 from early evening and pre-dawn, respectively, and the VIIRS from after midnight, made it possible to intercalibrate the images of three different overpass times and study the diurnal pattern of nighttime lights.

Urban sprawl and its stress on the risk of extreme hydrological events (EHEs) in the Kelani River basin, Sri Lanka

Land modification and urban sprawl cause incremental changes in hydrological processes due to the results of novel or hybrid ecosystems. This study mainly investigated the unforeseen urban expansion in the lower Kelani River basin, Sri Lanka, and its impact on the increment of hydrological extremes. Remote sensing data, including night-time light images (NOAA/AVHRR) and Landsat (TM/ETM+/OLI) data of different wavelengths, were analysed in this study. Land use and land cover data of the river basin were obtained from 1995. Shannon's entropy was used to demarcate urban sprawl in the river basin over nearly two decades. A spatial regression model was built to identify the correlation between increments of hydrological extremes and urban sprawl. This study revealed that the Kelani River basin has experienced a high urban sprawl rate over the past 23 years and that the total urban land area has increased by 130%. The flood risk analysis revealed that the flood frequency has also dramatically increased due to urban sprawl, and nearly 20 minor flood events have been recorded over the last two decades. Most of the urban areas situated in the lower river basin have invaded into the area with a higher flood risk, and the total flood damage has increased over the study period. The built-up land extent increased from 24.07 km2 to 56.39 km2. In 2018, the flood plain and the lower basin were mostly occupied by human settlements. Therefore, it is essential to improve current policies and mitigation plans to minimize the negative impact of rapid urban sprawl in the study area.

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

A Nighttime Vehicle Detection Method with Attentive GAN for Accurate Classification and Regression.

Vehicle detection plays a vital role in the design of Automatic Driving System (ADS), which has achieved remarkable improvements in recent years. However, vehicle detection in night scenes still has considerable challenges for the reason that the vehicle features are not obvious and are easily affected by complex road lighting or lights from vehicles. In this paper, a high-accuracy vehicle detection algorithm is proposed to detect vehicles in night scenes. Firstly, an improved Generative Adversarial Network (GAN), named Attentive GAN, is used to enhance the vehicle features of nighttime images. Then, with the purpose of achieving a higher detection accuracy, a multiple local regression is employed in the regression branch, which predicts multiple bounding box offsets. An improved Region of Interest (RoI) pooling method is used to get distinguishing features in a classification branch based on Faster Region-based Convolutional Neural Network (R-CNN). Cross entropy loss is introduced to improve the accuracy of classification branch. The proposed method is examined with the proposed dataset, which is composed of the selected nighttime images from BDD-100k dataset (Berkeley Diverse Driving Database, including 100,000 images). Compared with a series of state-of-the-art detectors, the experiments demonstrate that the proposed algorithm can effectively contribute to vehicle detection accuracy in nighttime.

A New Approach to Identify On-Ground Lamp Types from Night-Time ISS Images

Artificial night-time light (NTL), emitted by various on-ground human activities, has become intensive in many regions worldwide. Its adverse effects on human and ecosystem health crucially depend on the light spectrum, making the remote discrimination between different lamp types a highly important task. However, such studies remain extremely limited, and none of them exploit freely available satellite imagery. In the present analysis, the possibility to remotely assess the relative contribution of different lamp types into outdoor lighting is tested. For this sake, we match two data sources: (i) the radiometrically calibrated RGB image provided by the ISS (coarse spectral resolution data), and (ii) a set of in situ measurements with detailed spectral signatures conducted by ourselves (fine spectral resolution data). First, we analyze the fine spectral resolution data: using spectral signatures of standard lamp types from the LICA UCM library as endmembers, we perform an unmixing analysis upon NTL in situ measurements; by this, we obtain the estimates for relative contributions of the standard lamp types in each examined in situ measurement. Afterward, we focus on the coarse spectral resolution data: by using various types of statistical models, we predict the estimated relative contributions of each lamp type via RGB characteristics of spatially corresponding pixels of the ISS image. The built models predict sufficiently well (with R2 reaching ~0.87) the contributions of two standard lamp types: high-pressure sodium (HPS) and metal-halide (MH) lamps, the most widespread lamp types in the study area (Haifa, Israel). The restored map for HPS allocation demonstrates high concordance with the network of municipal roads, while that for MH shows notable coincidence with the industrial facilities and the airport.

Traffic Flow Parameters Collection under Variable Illumination Based on Data Fusion

Accurate traffic flow parameters are the supporting data for analyzing traffic flow characteristics. Vehicle detection using traffic surveillance pictures is a typical method for gathering traffic flow characteristics in urban traffic scenes. In complicated lighting conditions at night, however, neither classical nor deep-learning-based image processing algorithms can provide adequate detection results. This study proposes a fusion technique combining millimeter-wave radar data with image data to compensate for the lack of image-based vehicle detection under complicated lighting to complete all-day parameters collection. The proposed method is based on an object detector named CenterNet. Taking this network as the cornerstone, we fused millimeter-wave radar data into it to improve the robustness of vehicle detection and reduce the time-consuming postcalculation of traffic flow parameters collection. We collected a new dataset to train the proposed method, which consists of 1000 natural daytime images and 1000 simulated nighttime images with a total of 23094 vehicles counted, where the simulated nighttime images are generated by a style translator named CycleGAN to reduce labeling workload. Another four datasets of 2400 images containing 20161 vehicles were collected to test the proposed method. The experimental results show that the method proposed has good adaptability and robustness at natural daytime and nighttime scenes.