Scene Brightness Research Articles

Enabling the Assimilation of CrIS Shortwave Infrared Observations in Global NWP at NOAA. Part I: Background and Methods

Abstract Radiance observations from Earth-observing satellites have a significant positive impact on numerical weather prediction (NWP) forecasts, but some spectral regions are not fully exploited. Observations from hyperspectral infrared (IR) sounders in the longwave region (650-1100 cm−1), for instance, are routinely assimilated in many NWP models, but observations in the shortwave region (2155-2550 cm−1) are not. Each of these regions provides information on the temperature structure of the atmosphere, but the shortwave IR (SWIR) region is considered challenging to assimilate due to noise equivalent delta temperature (NEDT) that is highly variable depending on scene brightness temperature and to phenomena that are difficult to model, like non-Local Thermodynamic Equilibrium (NLTE) and solar reflectance. With recent advances in small-satellite technology, SWIR temperature sounders may provide an agile and cost-effective complement to the current constellation of IR sounders. Therefore, a better understanding of the use and impact of SWIR observations in data assimilation for NWP is warranted. In part one of this study, as presented here, the amount of unique information (as determined by Empirical Orthogonal Decomposition (EOD)) made available to a data assimilation system by Cross-track Infrared Sounder (CrIS) SWIR observations is reviewed, recent advancements to the Community Radiative Transfer Model (CRTM) for the simulation of CrIS shortwave radiances are tested, and enhancements to NOAA’s Global Data Assimilation System (GDAS) for the assimilation of CrIS SWIR observations are implemented and evaluated. Part two of this study, which seeks to assess the value of assimilating shortwave IR observations in global NWP, is also introduced.

DBI-Attack: Dynamic Bi-Level Integrated Attack for Intensive Multi-Scale UAV Object Detection

Benefiting from the robust feature representation capability of convolutional neural networks (CNNs), the object detection technology of intelligent high-altitude UAV remote sensing has been developed rapidly. In this field, the adversarial examples (AEs) pose serious security risks and vulnerabilities to deep learning-based systems. Due to the limitation of object size, image degradation, and scene brightness, adding adversarial disturbances to small and dense objects is extremely challenging. To study the threat of AE for UAV object detection, a dynamic bi-level integrated attack (DBI-Attack) is proposed for intensive multi-scale UAV object detection. Firstly, we use the dynamic iterative attack (DIA) method to generate perturbation on the classification level by improving the momentum iterative fast gradient sign method (MIM). Secondly, the bi-level adversarial attack method (BAAM) is constructed to add global perturbation on the decision level for completing the white-box attack. Finally, the integrated black-box attack method (IBAM) is combined to realize the black-box mislabeling and fabrication attacks. We experiment on the real drone traffic vehicle detection datasets to better evaluate the attack effectiveness. The experimental results show that the proposed method can achieve mislabeling and fabrication attacks on the UAV object detectors in black-box conditions. Furthermore, the adversarial training is applied to improve the model robustness. This work aims to call more attention to the adversarial and defensive aspects of UAV target detection models.

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

Improving pedestrians’ navigation safety at night by enhancing legibility of foreground and background information on the display

Traffic intersections are dangerous areas for pedestrians at night, primarily when people use their mobile phones while walking; pedestrians’ line of sight switches between their phones and the environment, which causes inattentional blindness during emergencies. The salience of the foreground and background information is different by the lightness conditions; therefore, identifying the lightness level of the foreground and background is critical. In this study, the brightness of urban traffic intersection scenes in the nighttime was analyzed, and improved color tones of the navigation interface for enhancing information recognition were identified. Moreover, identify the balance of visual attention by adjusting lightness and color treatment between foreground and background information. A glance legibility experiment indicated that a warm color tone with contrast and gamma correction under a lighting value of 65 in the LAB color mode is the optimal recognition combination. The results of this study indicated that low gamma under high lighting effectively reduced glare-related distractions and resulted in the lowest error rate in background information recognition. This paper proposes increasing the visual attention of urban pedestrians during nighttime navigation through the balancing of the lighting of the foreground interface and the color treatment of the background. Navigating at nighttime can be safer when their attention toward the environment increases and decreases attention distraction.

Assessment of spatial brightness for a visual field in interior spaces based on indirect corneal illuminance.

Quantitative evaluation of spatial brightness has been difficult, mainly due to the lack of a metric that is both highly related to subjective evaluation and convenient to measure in the field. This work investigated the applicability of using indirect corneal illuminance to evaluate spatial brightness for a visual field in interior spaces. Three lighting scenes with different patterns of lighting distribution, which all delivered indirect light to the subjects, were compared against each other in pairs for spatial brightness. The corresponding indirect corneal illuminance required for each test scene to match the spatial brightness of the reference scene with a fixed corneal illuminance was obtained. The results showed that our proposed metric had a high correlation with subjective evaluation of spatial brightness even under very different patterns of lighting distribution. Furthermore, the proposed metric was compared with the prior metrics of MRSE and Lav,B40 in spatial brightness evaluation, and the former showed the best correlation with subjective judgments. Since the spatial brightness assessment for various visual fields together compose people's overall impression of an illuminated space, the proposed metric of indirect corneal illuminance, which combines both accuracy and convenience in measurement, could serve as a preferred metric for spatial brightness evaluation.

Image Reconstruction of Synthetic Aperture Radiometer by Transformer

In passive microwave remote sensing of the earth, compared with real aperture radiometer, synthetic aperture radiometer is a very powerful instrument with many advantages. However, the system design is more complex than the real aperture radiometer, and the hardware ideality is often not guaranteed. The non-ideal characteristics of the system hardware will bring a variety of errors to the system, which will cause the Fourier transform relationship between the visibility function and the brightness temperature image to no longer be established, thus reducing the quality of microwave brightness temperature image reconstruction by traditional methods. In this paper, a new microwave brightness temperature image reconstruction method for synthetic aperture radiometer by Transformer is proposed. This method uses a specially designed Transformer structure to extract the spectrum features in the visibility function. This method learns the mapping relationship between the visibility function and the original scene brightness temperature image through the supervised learning method, and learns as much as possible the spectrum information contained in the original scene brightness temperature image. Moreover, when there are missing baselines, this method will supplement the missing observation frequency information, so as to obtain better reconstructed image quality. With the above advantages, this method can suppress the Gibbs oscillation, and greatly reduce the side lobe. Compared with the existing reconstruction methods, whether missing baselines or not, the proposed image reconstruction method by Transformer has advantages in image quality. We verify the performance of this brightness temperature image reconstruction method through simulation and experiment.

Pupillary Light Reflex Correction for Robust Pupillometry in Virtual Reality

Virtual reality (VR) headsets with an integrated eye tracker enable the measurement of pupil size fluctuations correlated with cognition during a VR experience. We present a method to correct for the light-induced pupil size changes, otherwise masking the more subtle cognitively-driven effects, such as cognitive load and emotional state. We explore multiple calibration sequences to find individual mapping functions relating the luminance to pupil dilation that can be employed in real-time during a VR experience. The resulting mapping functions are evaluated in a VR-based n-back task and in free exploration of a six-degrees-of-freedom VR scene. Our results show estimating luminance from a weighted average of the fixation area and the background yields the best performance. Calibration sequence composed of either solid gray or realistic scene brightness levels shown for 6 s in a pseudo-random order proved most robust.

Passive sensing by Sentinel-1 SAR: Methods and applications

In the TOPSAR acquisition mode, Sentinel-1 sensors acquire some data in passive mode, with a periodicity of approximately one second. These data provide valuable information about scene brightness and Radio Frequency pollution. This paper reviews the method to extract these passively sensed data and identify Radio Frequency Interferences (RFI). Two sets of measurements are generated by back-projecting passively sensed data on the ground, generating Earth Brightness Temperature (BT) and RFI equivalent temperature. A proper calibration has been implemented thanks to AMSR-2 data. The paper shows the result of one year of data collection from Sentinel 1 A and B regarding RFI and BT. We discuss the stability of the systems and the accuracy of the measurements by cross-comparing results from the two sensors, and we show how this information can be exploited for precise SAR data denoising and RFI mitigation.

Characterization of Brightness Temperature Biases at Channels 13 and 14 for FY-3C MWHS-2

The second generation Microwave Humidity Sounder (MWHS-2) onboard Fengyun (FY)-3C has a data quality comparable to that of counterpart microwave moisture sounders. However, channels 13 and 14 have large biases that are negatively correlated with the instrument temperature and prevent the operational assimilation of the data of these two channels. To better understand the biases of channels 13 and 14, the correlation of the observation minus simulation data (O-B bias) of different channels with the instrument temperature and other factors is investigated for FY-3C MWHS-2 in this paper. A sensitivity analysis using a gradient boosting decision tree indicates that the instrument temperature and scan position are the two dominant factors, with total bias-contribution scores exceeding 0.8 for both channels 13 and 14. This conclusion is verified through further analysis of the bias distributions for the scan position, scene brightness temperature and ascending/descending orbits. Using 12-week data recorded in 2016, the correlations of the O-B bias with the scan position and instrument temperature are specifically investigated and a correction algorithm is formulated, with which data for 2016 and 2017 are corrected. The corrected data in both channels have smaller, more stable biases, and are less affected by the instrument temperatures and scan position. The bias contributions associated with these two factors should thus be further studied.

Glyoxal tropospheric column retrievals from TROPOMI – multi-satellite intercomparison and ground-based validation

Abstract. We present the first global glyoxal (CHOCHO) tropospheric column product derived from the TROPOspheric Monitoring Instrument (TROPOMI) on board the Sentinel-5 Precursor satellite. Atmospheric glyoxal results from the oxidation of other non-methane volatile organic compounds (NMVOCs) and from direct emissions caused by combustion processes. Therefore, this product is a useful indicator of VOC emissions. It is generated with an improved version of the BIRA-IASB scientific retrieval algorithm relying on the differential optical absorption spectroscopy (DOAS) approach. Among the algorithmic updates, the DOAS fit now includes corrections to mitigate the impact of spectral misfits caused by scene brightness inhomogeneity and strong NO2 absorption. The product comes along with a full error characterization, which allows for providing random and systematic error estimates for every observation. Systematic errors are typically in the range of 1 ×1014–3 ×1014 molec. cm−2 (∼30 %–70 % in emission regimes) and originate mostly from a priori data uncertainties and spectral interferences with other absorbing species. The latter may be at the origin, at least partly, of an enhanced glyoxal signal over equatorial oceans, and further investigation is needed to mitigate them. Random errors are large (>6×1014 molec. cm−2) but can be reduced by averaging observations in space and/or time. Benefiting from a high signal-to-noise ratio and a large number of small-size observations, TROPOMI provides glyoxal tropospheric column fields with an unprecedented level of detail. Using the same retrieval algorithmic baseline, glyoxal column data sets are also generated from the Ozone Monitoring Instrument (OMI) on Aura and from the Global Ozone Monitoring Experiment-2 (GOME-2) on board Metop-A and Metop-B. Those four data sets are intercompared over large-scale regions worldwide and show a high level of consistency. The satellite glyoxal columns are also compared to glyoxal columns retrieved from ground-based Multi-AXis DOAS (MAX-DOAS) instruments at nine stations in Asia and Europe. In general, the satellite and MAX-DOAS instruments provide consistent glyoxal columns both in terms of absolute values and variability. Correlation coefficients between TROPOMI and MAX-DOAS glyoxal columns range between 0.61 and 0.87. The correlation is only poorer at one mid-latitude station, where satellite data appear to be biased low during wintertime. The mean absolute glyoxal columns from satellite and MAX-DOAS generally agree well for low/moderate columns with differences of less than 1×1014 molec. cm−2. A larger bias is identified at two sites where the MAX-DOAS columns are very large. Despite this systematic bias, the consistency of the satellite and MAX-DOAS glyoxal seasonal variability is high.

Regional Atmospheric Light Optimisation Algorithm for Heterogeneous Image Dehazing

Under foggy and other severe weather conditions, image acquisition equipment is not effective. It often produces an image with low contrast and low scene brightness, which is difficult to use in other image-based applications. The dark channel prior dehazing algorithm will cause the brightness of the image to decrease and sometimes introduce halos in the sky area. To solve this problem, we proposed a region similarity optimisation algorithm based on a dark channel prior. First, a vector comprising RGB layer dark channel value was obtained as the original atmospheric ambient light, and then, the proposed regional similarity linear function was used to adjust the atmospheric ambient light matrix. Next, the transmittance of different colour channels was derived and the multichannel soft matting algorithm was employed to produce more effective transmittance. Finally, the atmospheric ambient light and transmittance were substituted into the atmospheric scattering model to calculate clean images. Experimental results show that the proposed algorithm outperformed the existing mainstream dehazing algorithms in terms of both visual judgement and quality analysis with nonhomogeneous haze datasets. The algorithm not only improves the image details but also improves the brightness and saturation of the dehazing result; therefore, the proposed algorithm is effective in the restoration of the hazy image.

A switched view of Retinex: Deep self-regularized low-light image enhancement

Self-regularized low-light image enhancement does not require any normal-light image in training, thereby freeing from the chains of paired or unpaired training data that are time-consuming to obtain. However, existing methods suffer color deviation and fail to generalize to various lighting conditions. This paper presents a novel self-regularized method based on Retinex, which, inspired by HSV, preserves all colors (Hue, Saturation) and only integrates Retinex theory into brightness (Value). Besides, we design a novel random brightness disturbance approach to generate another abnormal brightness of the same scene. It is combined with the original form of brightness to estimate the same reflectance, which is achieved by a CNN. The reflectance, which is assumed irrelevant to any illumination according to the Retinex theory, is treated as the enhanced brightness. Our method is efficient as a low-light image is decoupled into two subspaces, i.e., color and brightness, for better preservation and enhancement. Extensive experiments demonstrate that our method outperforms multiple state-of-the-art algorithms qualitatively and quantitatively and adapts to more lighting conditions. Our code is available at https://github.com/Github-LHT/A-Switched-View-of-Retinex-Deep-Self-Regularized-Low-Light-Image-Enhancement.

Evaluation of graphic data corruptions impact on artificial intelligence applications

The article is devoted to the problem of the reliability of applications based on artificial intelligence. The authors made an attempt to evaluate the impact of the graphic data distortion at the input of a convolutional neural network on the result of image classification. The experiment is based on the fault injection method. A series of independent tests were carried out for such distortions as Gaussian noise, salt and pepper, speckle and Poisson noise, as well as median blur, motion blur, scene brightness changing, rotation, rain, and snow. The results showed that Gaussian noise was the least critical distortion; environmental conditions (rain, snow, brightness) and image rotation up to 20 degrees are less critical than focus losing and motion blur, while the most critical distortion is speckle noise. It was verified that preprocessing the input data of the neural network improves the accuracy of image recognition.

Segmentation of Illuminated Areas of Light Using CNN and Large-Scale RGB+D Dataset for Augmented and Mixed Reality Systems

This work is devoted to the problem of restoring realistic rendering for augmented and mixed reality systems. Finding the light sources and restoring the correct distribution of scene brightness is one of the key parameters that allows to solve the problem of correct interaction between the virtual and real worlds. With the advent of such datasets as, "LARGE-SCALE RGB + D," it became possible to train neural networks to recognize the depth map of images, which is a key requirement for working with the environment in real time. Additionally, in this work, convolutional neural networks were trained on the synthesized dataset with realistic lighting. The results of the proposed methods are presented, the accuracy of restoring the position of the light sources is estimated, and the visual difference between the image of the scene with the original light sources and the same scene. The speed allows it to be used in real-time AR/VR systems.

The Infrared Absolute Radiance Interferometer (ARI) for CLARREO

The Absolute Radiance Interferometer (ARI) is an infrared spectrometer designed to serve as an on-orbit radiometric reference with the ultra-high accuracy (better than 0.1 K 3‑σ or k = 3 brightness temperature at scene brightness temperature) needed to optimize measurement of the long-term changes of Earth’s atmosphere and surface. If flown in an orbit that frequently crosses sun-synchronous orbits, ARI could be used to inter-calibrate the international fleet of infrared (IR) hyperspectral sounders to similar measurement accuracy, thereby establishing an observing system capable of achieving sampling biases on high-information-content spectral radiance products that are also < 0.1 K 3‑σ. It has been shown that such a climate observing system with <0.1 K 2‑σ overall accuracy would make it possible to realize times to detect subtle trends of temperature and water vapor distributions that closely match those of an ideal system, given the limit set by the natural variability of the atmosphere. This paper presents the ARI sensor's overall design, the new technologies developed to allow on-orbit verification and test of its accuracy, and the laboratory results that demonstrate its capability. In addition, we describe the techniques and uncertainty estimates for transferring ARI accuracy to operational sounders, providing economical global coverage. Societal challenges posed by climate change suggest that a Pathfinder ARI should be deployed as soon as possible.

Can We See with Melanopsin?

A small fraction of mammalian retinal ganglion cells are directly photoreceptive thanks to their expression of the photopigment melanopsin. These intrinsically photosensitive retinal ganglion cells (ipRGCs) have well-established roles in a variety of reflex responses to changes in ambient light intensity, including circadian photoentrainment. In this article, we review the growing evidence, obtained primarily from laboratory mice and humans, that the ability to sense light via melanopsin is also an important component of perceptual and form vision. Melanopsin photoreception has low temporal resolution, making it fundamentally biased toward detecting changes in ambient light and coarse patterns rather than fine details. Nevertheless, melanopsin can indirectly impact high-acuity vision by driving aspects of light adaptation ranging from pupil constriction to changes in visual circuit performance. Melanopsin also contributes directly to perceptions of brightness, and recent data suggest that this influences the appearance not only of overall scene brightness, but also of low-frequency patterns.

Influence of Background Spectral Distribution on Perceptions of Discomfort Glare

<div class="section abstract"><div class="htmlview paragraph">The advent of light-emitting diode (LED) technology for automotive lighting allows flexibility of the spectral distribution of forward headlighting systems, while meeting current requirements for “white” illumination. As vehicle headlights have become whiter (with more short-wavelength light output) over the past several decades, their potential impacts on visual discomfort for oncoming and preceding drivers have been hotly debated. It is known that a greater proportion of short-wavelength energy increases discomfort glare, and that increasing the background light level (e.g., through roadway lighting) will decrease perceptions of discomfort. More recently it has been demonstrated that the visual system exhibits enhanced short-wavelength sensitivity for perceptions of scene brightness. As a result, roads illuminated by light sources with higher correlated color temperatures (CCTs) will be judged as appearing to be brighter than those illuminated to the same light level by sources with lower CCTs. The present laboratory study was conducted to identify whether the increased scene brightness of a road illuminated with greater short-wavelength light helps to mitigate discomfort glare more than the same scene illuminated to the same light level, but with less short-wavelength light. The results indicate that the spectral distribution of the background plays little role in the degree to which it lessens discomfort glare. The implications of these results for vehicle and road lighting practices are discussed.</div></div>

Optimization of Adaptive Lighting Technology Based on Varying-Frequency Sinusoidal Grating Projection

In order to enhance the image quality of multi-reflective 3D scenes during machine vision analysis, an active self-adaptive illumination system to solve the overexposure and underexposure problems is introduced. Based on projector-camera system, adaptive illumination was achieved by modulating the projection brightness at each pixel to be proportional to the pixel brightness of the target scenes, which should firstly establish the accurate pixel correspondence between the projector and camera. In this paper, the performance of adaptive illumination is improved by optimizing the correspondence stage compared with previous research. Different grating patterns with different frequency components are projected and compared to acquire the most effective correspondence, which improves the accuracy of corresponding depth map and the experiment results can achieve higher lighting accuracy reducing the number of iterations as well as the computing time. The image quality indexes, image contrast and number of saturated or dark pixels, are combined to analyze and judge the optimized lighting result. The results demonstrate that the developed technology in this research can achieve a rate of 20 frames per second by using a standard commercial CCD (charge coupled device) camera, where it can run in real time and becomes a viable solution for the industry.

Rhodopsin and melanopsin contributions to human brightness estimation.

We examined the contributions of rhodopsin and melanopsin to human brightness estimation under dim lighting. Absolute brightness magnitudes were estimated for full-field, rhodopsin-, or melanopsin-equated narrowband lights (${\lambda _{\rm max}}:\;{462}$λmax:462, 499, 525 nm). Our data show that in scotopic illumination ($ - {5.1}$-5.1 to $ - {3.9}\;{\log}\;\unicode{x00B5} {\rm Watts}\cdot{\rm cm}^{ - 2}$-3.9logµWatts⋅cm-2), the perceptual brightness estimates of rhodopic irradiance-equated conditions are independent of their corresponding melanopic irradiance, whereas brightness estimates with melanopic irradiance-equated conditions increase with increasing rhodopic irradiance. In mesopic illumination ($ - {3.4}$-3.4 to $ - {1.9}\;{\log}\;\unicode{x00B5} {\rm Watts}\cdot{\rm cm}^{ - 2}$-1.9logµWatts⋅cm-2), the brightness estimates with both lighting conditions increase with increasing rhodopic or melanopic irradiances. Rhodopsin activation therefore entirely signals scotopic brightness perception and plateaus in mesopic illumination where intrinsic melanopsin contributions become first evident. We infer that all photoreceptor signals are transmitted to higher visual centers for representing scene brightness in scotopic and mesopic illumination through both conventional and melanopsin ganglion cell pathways.

Correcting Misleading Image Quality Measurements

We discuss several common image quality measurements that are often misinterpreted, so that bad images are falsely interpreted as good, and we describe how to obtain valid measurements. Sharpness, which is measured by MTF (Modulation Transfer Function) curves, is frequently summarized by MTF50 (the spatial frequency where MTF falls to half its low frequency value) But because MTF50 strongly rewards excessive sharpening, we recommend other summary metrics, especially MTF50P (the spatial frequency where MTF falls to half its peak value), that provide a more stable indication of system performance. Camera dynamic range (DR), defined as the range of exposure (scene brightness) where the image has good contrast and Signalto- Noise Ratio (SNR), Is usually measured with grayscale step charts. We have recently seen several cases where flare light radiating out from bright areas of the image fogs dense patches, causing unreasonably high DR measurements. This situation is difficult to handle with linear test charts, where the flare light is aligned with the patches, but can be handled well in charts with circular patch patterns, where the patch where pixel level ceases to decrease defines the upper DR limit.