Extreme Illumination Conditions Research Articles

Mineralogical Characterization of the Lunar South Polar Region: 1. The Artemis Exploration Zone

AbstractThe lunar south pole is a region of focused scientific and exploration interest, with several crewed and robotic missions to this region planned within the next decade. Understanding the mineralogy of the region is essential to inform landing site characterization and selection and provides the key context for interpreting samples and in situ observations. At high latitudes, extreme illumination conditions (high phase angles) can negatively impact the data quality of orbital instruments. This is especially true for passive near‐infrared spectrometers such as the Moon Mineralogy Mapper (M3) and the Kaguya Spectral Profiler, which measure the spectral properties of the surface using reflected sunlight. Using Moon Mineralogy Mapper data, we observed that the south polar region is associated with a detectable mafic signature consistent with the presence of pyroxenes. The strongest mafic signatures are associated with the South Pole—Aitken Basin, suggesting that impact melt and basin ejecta from the lower crust and upper mantle are present within this region. This observation is validated in several ways: (a) comparisons between M3 data acquired during different mission phases, (b) comparisons between multiple spectral parameters sensitive to the presence of mafic minerals, (c) comparisons between the north and south lunar polar regions, and (d) comparisons with publicly available Kaguya polar mineralogy maps and Lunar Prospector elemental abundances. We also investigate the nature of an anomalous high‐albedo region within 2–3° of the south pole observed in Lunar Orbiter Laser Altimeter reflectance data exhibiting a spatially conflicting apparent FeO abundance pattern between several data sets.

VERI-D: A new dataset and method for multi-camera vehicle re-identification of damaged cars under varying lighting conditions

Vehicle re-identification (V-ReID) is a critical task that aims to match the same vehicle across images from different camera viewpoints. The previous studies have leveraged attribute clues, such as color, model, and license plate, to enhance the V-ReID performance. However, these methods often lack effective interaction between the global–local features and the final V-ReID objective. Moreover, they do not address the challenging issues in real-world scenarios, such as high viewpoint variations, extreme illumination conditions, and car appearance changes (e.g., due to damage or wrong driving). We propose a novel framework to tackle these problems and advance the research in V-ReID, which can handle various types of car appearance changes and achieve robust V-ReID under varying lighting conditions. Our main contributions are as follows: (i) we propose a new Re-ID architecture named global–local self-attention network, which integrates local information into the feature learning process and enhances the feature representation for V-ReID and (ii) we introduce a novel damaged vehicle Re-ID dataset called VERI-D, which is the first publicly available dataset that focuses on this challenging yet practical scenario. The dataset contains both natural and synthetic images of damaged vehicles captured from multiple camera viewpoints and under different lighting conditions. (iii) We conduct extensive experiments on the VERI-D dataset and demonstrate the effectiveness of our approach in addressing the challenges associated with damaged vehicle re-identification. We also compare our method to several state-of-the-art V-ReID methods and show its superiority.

Neuromorphic imaging and classification with graph learning

Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. They can capture dynamic scenes with little motion blur and more details in extreme illumination conditions. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. While several event representations and processing methods have been developed to address such an issue, they are typically driven by a large number of events, leading to substantial overheads in runtime and memory. In this paper, we propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification. Extensive experiments show that our approach leads to better results and excels at the challenging realistic situations where only a small number of events and limited computational resources are available, paving the way for neuromorphic applications embedded into mobile facilities.

Influence of the threshold settings of photon counting detectors on the observed speckle contrast in coherent scattering experiments

Speckle-based experiments, in particular X-ray Photon Correlation Spectroscopy (XPCS), are among the ones that benefit most from the development of next generation light sources. The key quantity that determines whether or not it will be possible to perform an XPCS experiment is the speckle contrast β, which measures the visibility of the speckle pattern. The speckle contrast is influenced by geometrical conditions and light-source properties [1], and recently it has been noticed that artifacts in photon-counting detectors strongly interfere with the determination of the real value of β when the number of detected photons per frame is extremely low [2]. Here we report how the threshold setting of a photon counting detector, an EIGER X 4M, affects the detected contrast even at non extreme illumination conditions (0.01-0.05 photons/pixel/second). We found that by increasing the threshold value, not only leads to the expected drop in detected intensity, but also to a significant increase in the value of β. Additionally we will present an in-operando situation in which the higher threshold helps critically to increase the signal-to-noise ratio of the measured scattering pattern resulting in a five-fold increment of the speckle contrast.

PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

Infrared and visible image fusion aims to synthesize a single fused image containing salient targets and abundant texture details even under extreme illumination conditions. However, existing image fusion algorithms fail to take the illumination factor into account in the modeling process. In this paper, we propose a progressive image fusion network based on illumination-aware, termed as PIAFusion, which adaptively maintains the intensity distribution of salient targets and preserves texture information in the background. Specifically, we design an illumination-aware sub-network to estimate the illumination distribution and calculate the illumination probability. Moreover, we utilize the illumination probability to construct an illumination-aware loss to guide the training of the fusion network. The cross-modality differential aware fusion module and halfway fusion strategy completely integrate common and complementary information under the constraint of illumination-aware loss. In addition, a new benchmark dataset for infrared and visible image fusion, i.e., Multi-Spectral Road Scenarios (available at https://github.com/Linfeng-Tang/MSRS), is released to support network training and comprehensive evaluation. Extensive experiments demonstrate the superiority of our method over state-of-the-art alternatives in terms of target maintenance and texture preservation. Particularly, our progressive fusion framework could round-the-clock integrate meaningful information from source images according to illumination conditions. Furthermore, the application to semantic segmentation demonstrates the potential of our PIAFusion for high-level vision tasks. Our codes will be available at https://github.com/Linfeng-Tang/PIAFusion.

Practical Face Reconstruction via Differentiable Ray Tracing

AbstractWe present a differentiable ray‐tracing based novel face reconstruction approach where scene attributes – 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, and scene illumination – are estimated from unconstrained monocular images. The proposed method models scene illumination via a novel, parameterized virtual light stage, which in‐conjunction with differentiable ray‐tracing, introduces a coarse‐to‐fine optimization formulation for face reconstruction. Our method can not only handle unconstrained illumination and self‐shadows conditions, but also estimates diffuse and specular albedos. To estimate the face attributes consistently and with practical semantics, a two‐stage optimization strategy systematically uses a subset of parametric attributes, where subsequent attribute estimations factor those previously estimated. For example, self‐shadows estimated during the first stage, later prevent its baking into the personalized diffuse and specular albedos in the second stage. We show the efficacy of our approach in several real‐world scenarios, where face attributes can be estimated even under extreme illumination conditions. Ablation studies, analyses and comparisons against several recent state‐of‐the‐art methods show improved accuracy and versatility of our approach. With consistent face attributes reconstruction, our method leads to several style – illumination, albedo, self‐shadow – edit and transfer applications, as discussed in the paper.

Face illumination processing via dense feature maps and multiple receptive fields

Recently, illumination processing of facial image based on generative adversarial networks has made favourable progress. However, the image quality is not so satisfactory and the recognition accuracy is low when the face image under extreme illumination conditions. For these reasons, an elaborately-designed architecture based on convolutional neural network and generative adversarial networks for processing face illumination is presented. A novel dense feature maps loss that computes loss by using the varisized feature maps extracted from different convolutional layers of pre-trained feature network is put forward. Moreover, multiple-receptive-fields-based generator that uses multiple encoders during encoding phase is also proposed, and these encoders have the same structure with different kernel size. A variety of experimental results demonstrate that the method is superior to the state-of-the-art methods under various illumination challenges. Code will be available soon at https://github.com/ling20cn/IP-GAN

Evaluation of Event-Based Corner Detectors

Bio-inspired Event-Based (EB) cameras are a promising new technology that outperforms standard frame-based cameras in extreme lighted and fast moving scenes. Already, a number of EB corner detection techniques have been developed; however, the performance of these EB corner detectors has only been evaluated based on a few author-selected criteria rather than on a unified common basis, as proposed here. Moreover, their experimental conditions are mainly limited to less interesting operational regions of the EB camera (on which frame-based cameras can also operate), and some of the criteria, by definition, could not distinguish if the detector had any systematic bias. In this paper, we evaluate five of the seven existing EB corner detectors on a public dataset including extreme illumination conditions that have not been investigated before. Moreover, this evaluation is the first of its kind in terms of analysing not only such a high number of detectors, but also applying a unified procedure for all. Contrary to previous assessments, we employed both the intensity and trajectory information within the public dataset rather than only one of them. We show that a rigorous comparison among EB detectors can be performed without tedious manual labelling and even with challenging acquisition conditions. This study thus proposes the first standard unified EB corner evaluation procedure, which will enable better understanding of the underlying mechanisms of EB cameras and can therefore lead to more efficient EB corner detection techniques.

Learning to Reconstruct HDR Images from Events, with Applications to Depth and Flow Prediction

Event cameras have numerous advantages over traditional cameras, such as low latency, high temporal resolution, and high dynamic range (HDR). We initially investigate the potential of creating intensity images/videos from an adjustable portion of the event data stream via event-based conditional generative adversarial networks (cGANs). Using the proposed framework, we further show the versatility of our method in directly handling similar supervised tasks, such as optical flow and depth prediction. Stacks of space-time coordinates of events are used as the inputs while the proposed framework is trained to predict either the intensity images, optical flows, or depth outputs according to the target task. We further demonstrate the unique capability of our approach in generating HDR images even under extreme illumination conditions, creating non-blurred images under rapid motion, and generating very high frame rate videos up to the temporal resolution of event cameras. The proposed framework is evaluated using a publicly available real-world dataset and a synthetic dataset we prepared by utilizing an event camera simulator.

A High-Performance Face Illumination Processing Method via Multi-Stage Feature Maps.

In recent years, Generative Adversarial Networks (GANs)-based illumination processing of facial images has made favorable achievements. However, some GANs-based illumination-processing methods only pay attention to the image quality and neglect the recognition accuracy, whereas others only crop partial face area and ignore the challenges to synthesize photographic face, background and hair when the original face image is under extreme illumination (Image under extreme illumination (extreme illumination conditions) means that we cannot see the texture and structure information clearly and most pixel values tend to 0 or 255.). Moreover, the recognition accuracy is low when the faces are under extreme illumination conditions. For these reasons, we present an elaborately designed architecture based on convolutional neural network and GANs for processing the illumination of facial image. We use ResBlock at the down-sampling stage in our encoder and adopt skip connections in our generator. This special design together with our loss can enhance the ability to preserve identity and generate high-quality images. Moreover, we use different convolutional layers of a pre-trained feature network to extract varisized feature maps, and then use these feature maps to compute loss, which is named multi-stage feature maps (MSFM) loss. For the sake of fairly evaluating our method against state-of-the-art models, we use four metrics to estimate the performance of illumination-processing algorithms. A variety of experimental data indicate that our method is superior to the previous models under various illumination challenges in illumination processing. We conduct qualitative and quantitative experiments on two datasets, and the experimental data indicate that our scheme obviously surpasses the state-of-the-art algorithms in image quality and identification accuracy.

A Comprehensive Experimental and Reproducible Study on Selfie Biometrics in Multistream and Heterogeneous Settings

This contribution presents a new database to address current challenges in face recognition. It contains face video sequences of 75 individuals acquired either through a laptop webcam or when mimicking the front-facing camera of a smartphone. Sequences have been acquired with a device allowing to record visual, near-infrared, and depth data at the same time. Recordings have been made across three sessions with different, challenging illumination conditions and variations in pose. Together with the database, several experimental protocols are provided and correspond to real world scenarios, when a mismatch in conditions between enrollment and probe images occurs. A comprehensive set of baseline experiments using publicly available baseline algorithms show that extreme illumination conditions and pose variations are remaining issues. However, the usage of different data domains—and their fusion—allows to mitigate such variation. Finally, experiments on heterogeneous face recognition are also presented using a state-of-the-art model based on deep neural networks, and showed better performance. When applied to other tasks, this model proved to surpass all existing baselines as well. The data, as well as the code to reproduce all experiments are made publicly available to help foster research in selfie biometrics using latest imaging devices.

Advancing retrievals of surface reflectance and vegetation indices over forest ecosystems by combining imaging spectroscopy, digital object models, and 3D canopy modelling

Imaging spectroscopy based methods offer unique capabilities for retrieving narrow-band vegetation indices which can be empirically related to functional traits of plants. However, in areas with complex topography, illumination effects affect the retrieval of such indices from high spatial resolution airborne or satellite data. Irradiance components at the pixel level are determined by atmospheric composition, as well as instantaneous illumination-surface-sensor geometries. An accurate pixel-wise description of direct and diffuse irradiance components is necessary to perform atmospheric corrections, finally resulting in improved surface reflectances and hence products. We assess three atmospheric correction strategies, differing in their approaches to simulate instantaneous as well as pixel-wise abundances of diffuse and direct irradiance. We use physically-based approaches in combination with either digital elevation models (DEM), fine resolution digital object models (DOM), or 3D modelling output from the Discrete Anisotropic Radiative Transfer (DART) model. The such obtained top-of-canopy reflectances at the Laegern test-site in Switzerland, are used to assess retrieval improvement for a set of indices (Normalized Difference Vegetation Index (NDVI), Photochemical Reflectance Index (PRI), as well as chlorophyll and carotenoid indices). We demonstrate that both, the DOM and the DART based approach, improve the retrievals for flat cast-shadows by ≤71% compared to using a DEM. In dense forest areas, improvements are less significant. Remaining key issues are related to overestimating surface reflectance under extreme illumination conditions.

Perceptual Lightness Modeling for High-Dynamic-Range Imaging

The human visual system (HVS) non-linearly processes light from the real world, allowing us to perceive detail over a wide range of illumination. Although models that describe this non-linearity are constructed based on psycho-visual experiments, they generally apply to a limited range of illumination and therefore may not fully explain the behavior of the HVS under more extreme illumination conditions. We propose a modified experimental protocol for measuring visual responses to emissive stimuli that do not require participant training, nor requiring the exclusion of non-expert participants. Furthermore, the protocol can be applied to stimuli covering an extended luminance range. Based on the outcome of our experiment, we propose a new model describing lightness response over an extended luminance range. The model can be integrated with existing color appearance models or perceptual color spaces. To demonstrate the effectiveness of our model in high dynamic range applications, we evaluate its suitability for dynamic range expansion relative to existing solutions.

Obstacle Detection System for Small UAVs using ADS-B and Thermal Imaging

Thermal Infrared (TIR) imaging is a promising technology which can provide enhanced capabilities to current vision-based Sense-and-Avoid (SAA) systems. It allows operation under extreme illumination conditions, such as direct sun exposure and during nighttime. This paper presents a lightweight obstacle detection system for small UAVs that integrates a TIR camera and an Automatic Dependent Surveillance Broadcast (ADS-B) receiver. Algorithms for the detection of flying obstacles in TIR images were developed and TIR images were experimentally compared with synchronized RGB images for validation. Matching between aircraft detected in TIR images and those reported by an ADS-B receiver was performed to obtain distance information to the visually detected aircraft. We experimentally proved that our system is able to enhance individual ADS-B and TIR detection capabilities by detecting aircraft under challenging illumination conditions at real-time frame rates while providing distance estimations to visual detections.

Using an improved SIFT algorithm and fuzzy closed-loop control strategy for object recognition in cluttered scenes.

Partial occlusions, large pose variations, and extreme ambient illumination conditions generally cause the performance degradation of object recognition systems. Therefore, this paper presents a novel approach for fast and robust object recognition in cluttered scenes based on an improved scale invariant feature transform (SIFT) algorithm and a fuzzy closed-loop control method. First, a fast SIFT algorithm is proposed by classifying SIFT features into several clusters based on several attributes computed from the sub-orientation histogram (SOH), in the feature matching phase only features that share nearly the same corresponding attributes are compared. Second, a feature matching step is performed following a prioritized order based on the scale factor, which is calculated between the object image and the target object image, guaranteeing robust feature matching. Finally, a fuzzy closed-loop control strategy is applied to increase the accuracy of the object recognition and is essential for autonomous object manipulation process. Compared to the original SIFT algorithm for object recognition, the result of the proposed method shows that the number of SIFT features extracted from an object has a significant increase, and the computing speed of the object recognition processes increases by more than 40%. The experimental results confirmed that the proposed method performs effectively and accurately in cluttered scenes.

Automatic Storm Damage Detection in Forests Using High‑Altitude Photogrammetric Imagery

Climate change has increased the occurrence of heavy storms that cause damage to forests. After a storm, it is necessary to obtain knowledge about the injured trees quickly in order to detect and aid in collecting the fallen trees and estimate the total damage. The objective in this study was to develop an automatic method for storm damage detection based on comparisons of digital surface models (DSMs), where the after-storm DSM was derived by automatic image matching using high-altitude photogrammetric imagery. This DSM was compared to a before-storm DSM, which was computed using national airborne laser scanning (ALS) data. The developed method was tested using imagery collected in extreme illumination conditions after winter storms on 8 January 2012 in Finland. The image matching yielded a high-quality surface model of the forest areas, which were mainly coniferous and mixed forests. The entire set of major damage forest test areas was correctly classified using the method. Our results showed that airborne, high-altitude photogrammetry is a promising tool for automating the detection of forest storm damage. With modern photogrammetric cameras, large areas can be collected efficiently, and the imagery also provides visual, stereoscopic support for various forest storm damage management tasks. Developing methods that work in different seasons are becoming more important, due to the increase in the number of natural disasters.

Daily Light–Dark Cycles Influence Hypoxia‐Inducible Factor 1 and Heat Shock Protein Levels in the Pacemakers of Crayfish

It has been proposed that at moderate concentrations, reactive oxygen species (ROS) play an important role as regulatory mediators in signaling processes. Many of the ROS-mediated responses, such as those mediated by the glutathione (GSH) system, protect cells against oxidative stress and reestablish "redox homeostasis." The response to stress is controlled by specific transcription factors, such as hypoxia-inducible factor 1 (HIF-1), which interacts with chaperones such as heat shock proteins (HSP) that stabilize intracellular components. We have reported that the oxidation produced by extreme light cycles was coupled to cellular fluctuations in the production of free radicals and ROS in crayfish. Herein, we investigated whether ROS resulting from light-dark cycles (20:4 LD) with a long photoperiod activated HIF-1α in the putative pacemakers of crayfish. Our results indicate that both the light resulting from equatorial and extreme daily light cycles and the constant darkness-induced HIF-1α and heat shock protein 70 (HSP70) appeared to regulate each other. The interaction between these proteins and the ability of crayfish to shift from the oxidative to glycolytic pathways, thereby synchronizing to extreme illumination conditions and maintaining a rhythmic predictive relationship with the environment, suggest HIF-1 as a key factor in these rhythmic metabolic interactions.

A robust lane recognition technique for vision-based navigation with a multiple clue-based filtration algorithm

This paper proposes a novel multiple clue-based filtration algorithm (MCFA), which is developed to detect lane markings on roads using camera vision images for autonomous mobile robot navigation. The main goal of the algorithm is the robust estimation of the relative position and angle of the lane in the image by using multiple clues based on different characteristics of the lane. In particular, robustness against environmental changes is enhanced greatly since a dynamic model of the lane, besides static features of the lane such as color, intensity, etc., is incorporated for reliable estimation. The efficiency of the algorithm is verified through mobile robot experiments under various extreme illumination conditions in outdoor environments. The increased robustness performance enables reliable closed-loop control of a mobile robot that operates in a variety of navigation-related missions.