Scene Illumination Research Articles

FHLight: A novel method of indoor scene illumination estimation using improved loss function

In augmented reality tasks, especially in indoor scenes, achieving illumination consistency between virtual objects and real environments is a critical challenge. Currently, mainstream methods are illumination parameters regression and illumination map generation. Among these two categories of methods, few works can effectively recover both high-frequency and low-frequency illumination information within indoor scenes. In this work, we argue that effective restoration of low-frequency illumination information forms the foundation for capturing high-frequency illumination details. In this way, we propose a novel illumination estimation method called FHLight. Technically, we use a low-frequency spherical harmonic irradiance map (LFSHIM) restored by the low-frequency illumination regression network (LFIRN) as prior information to guide the high-frequency illumination generator (HFIG) to restore the illumination map. Furthermore, we suggest an improved loss function to optimize the network training procedure, ensuring that the model accurately restores both low-frequency and high-frequency illumination information within the scene. We compare FHLight with several competitive methods, and the results demonstrate significant improvements in metrics such as RMSE, si-RMSE, and Angular error. In addition, visual experiments further confirm that FHLight is capable of generating scene illumination maps with genuine frequencies, effectively resolving the illumination consistency issue between virtual objects and real scenes. The code is available at https://github.com/WA-tyro/FHLight.git.

ChromaFlash: Snapshot Hyperspectral Imaging Using Rolling Shutter Cameras

Hyperspectral imaging captures scene information across narrow, contiguous bands of the electromagnetic spectrum. Despite its proven utility in industrial and biomedical applications, its ubiquity has been limited by bulky form factors, slow capture times, and prohibitive costs. In this work, we propose a generalized approach to snapshot hyperspectral imaging that only requires a standard rolling shutter camera and wavelength-adjustable lighting. The crux of this approach entails using the rolling shutter as a spatiotemporal mask, varying incoming light quicker than the camera's frame rate in order for the captured image to contain rows of pixels illuminated at different wavelengths. An image reconstruction pipeline then converts this coded image into a complete hyperspectral image using sparse optimization. We demonstrate the feasibility of this approach by deploying a low-cost system called ChromaFlash, which uses a smartphone's camera for image acquisition and a series of LEDs to change the scene's illumination. We evaluated ChromaFlash through simulations on two public hyperspectral datasets and assessed its spatial and spectral accuracy across various system parameters. We also tested the real-world performance of our prototype by capturing diverse scenes under varied ambient lighting conditions. In both experiments, ChromaFlash outperformed state-of-the-art techniques that use deep learning to convert RGB images into hyperspectral ones, achieving snapshot performance not demonstrated by prior attempts at accessible hyperspectral imaging.

LightFormer: Light-Oriented Global Neural Rendering in Dynamic Scene

The generation of global illumination in real time has been a long-standing challenge in the graphics community, particularly in dynamic scenes with complex illumination. Recent neural rendering techniques have shown great promise by utilizing neural networks to represent the illumination of scenes and then decoding the final radiance. However, incorporating object parameters into the representation may limit their effectiveness in handling fully dynamic scenes. This work presents a neural rendering approach, dubbed LightFormer , that can generate realistic global illumination for fully dynamic scenes, including dynamic lighting, materials, cameras, and animated objects, in real time. Inspired by classic many-lights methods, the proposed approach focuses on the neural representation of light sources in the scene rather than the entire scene, leading to the overall better generalizability. The neural prediction is achieved by leveraging the virtual point lights and shading clues for each light. Specifically, two stages are explored. In the light encoding stage, each light generates a set of virtual point lights in the scene, which are then encoded into an implicit neural light representation, along with screen-space shading clues like visibility. In the light gathering stage, a pixel-light attention mechanism composites all light representations for each shading point. Given the geometry and material representation, in tandem with the composed light representations of all lights, a lightweight neural network predicts the final radiance. Experimental results demonstrate that the proposed LightFormer can yield reasonable and realistic global illumination in fully dynamic scenes with real-time performance.

The Effect of Varying the Light Spectrum of a Scene on the Localisation of Photogrammetric Features

In modern digital photogrammetry, an image is usually registered via a digital matrix with an array of colour filters. From the registration of the image until feature points are detected on the image, the image is subjected to a series of calculations, i.e., demosaicing and conversion to greyscale, among others. These algorithms respond differently to the varying light spectrum of the scene, which consequently results in the feature location changing. In this study, the effect of scene illumination on the localisation of a feature in an image is presented. The demosaicing and greyscale conversion algorithms that produce the largest and smallest deviation of the feature from the reference point were assessed. Twelve different illumination settings from polychromatic light to monochromatic light were developed and performed, and five different demosaicing algorithms and five different methods of converting a colour image to greyscale were analysed. A total of 300 different cases were examined. As the study shows, the lowest deviation in the polychromatic light domain was achieved for light with a colour temperature of 5600 K and 5000 K, while in the monochromatic light domain, it was achieved for light with a green colour. Demosaicing methods have a significant effect on the localisation of a feature, and so the smallest feature deviation was achieved for smooth hue-type demosaicing, while for greyscale conversion, it was achieved for the mean type. Demosaicing and greyscale conversion methods for monochrome light had no effect. The article discusses the problem and concludes with recommendations and suggestions in the area of illuminating the scene with artificial light and the application of the algorithms, in order to achieve the highest accuracy using photogrammetric methods.

Concrete crack pixel-level segmentation: a comparison of scene illumination angle of incidence

Previous research has demonstrated how angled and directional lighting can enhance the detection of concrete cracks in low-light environments and outperform diffused lighting alternatives. This paper investigates the effect of different angles of incidence of directional lighting for concrete crack pixel-level segmentation. Five directional lighting datasets of cracked concrete slabs were captured, each using an angle of incidence of 10, 20, 30, 40, and 50 degrees, respectively. A directional lighting crack segmentation algorithm was applied to each lighting angle dataset. Algorithm output comparisons with ground truths revealed that the directional lighting method performed best on the 50-degree lighting dataset, obtaining a recall, precision, and F1 score of 68%, 81%, and 74%, respectively. However, qualitative analysis of the segmentation outputs on a sub-image scale revealed that towards the edges of the images, the segmentation performance of 30-degree lighting was significantly better, with results closely matching those of the ground truth. This research highlights that the lighting angle of incidence can increase the performance of directional lighting concrete crack segmentation depending on defect position. The results from this work have the potential to improve low-light environment concrete crack detection and monitoring.

Dual-Adaptive Heterojunction Synaptic Transistors for Efficient Machine Vision in Harsh Lighting Conditions.

Photoadaptive synaptic devices enable in-sensor processing of complex illumination scenes, while second-order adaptive synaptic plasticity improves learning efficiency by modifying the learning rate in a given environment. The integration of above adaptations in one phototransistor device will provide opportunities for developing high-efficient machine vision system. Here, a dually adaptable organic heterojunction transistor as a working unit in the system, which facilitates precise contrast enhancement and improves convergence rate under harsh lighting conditions, is reported. The photoadaptive threshold sliding originates from the bidirectional photoconductivity caused by the light intensity-dependent photogating effect. Metaplasticity is successfully implemented owing to the combination of ambipolar behavior and charge trapping effect. By utilizing the transistor array in a machine vision system, the details and edges can be highlighted in the 0.4% low-contrast images, and a high recognition accuracy of 93.8% with a significantly promoted convergence rate by about 5 times are also achieved. These results open a strategy to fully implement metaplasticity in optoelectronic devices and suggest their vision processing applications in complex lighting scenes.

New Era in Cultural Heritage Preservation: Cooperative Aerial Autonomy for Fast Digitalization of Difficult-to-Access Interiors of Historical Monuments

Digital documentation of large interiors of historical buildings is an exhausting task since most of the areas of interest are beyond typical human reach. We advocate the use of autonomous teams of multi-rotor Unmanned Aerial Vehicles (UAVs) to speed up the documentation process by several orders of magnitude while allowing for a repeatable, accurate, and condition-independent solution capable of precise collision-free operation at great heights. The proposed multi-robot approach allows for performing tasks requiring dynamic scene illumination in large-scale real-world scenarios, a process previously applicable only in small-scale laboratory-like conditions. Extensive experimental analyses range from single-UAV imaging to specialized lighting techniques requiring accurate coordination of multiple UAVs. The system's robustness is demonstrated in more than two hundred autonomous flights in fifteen historical monuments requiring superior safety while lacking access to external localization. This unique experimental campaign, cooperated with restorers and conservators, brought numerous lessons transferable to other safety-critical robotic missions in documentation and inspection tasks.

Event-based asynchronous HDR imaging by temporal incident light modulation

Dynamic range (DR) is a pivotal characteristic of imaging systems. Current frame-based cameras struggle to achieve high dynamic range imaging due to the conflict between globally uniform exposure and spatially variant scene illumination. In this paper, we propose AsynHDR, a pixel-asynchronous HDR imaging system, based on key insights into the challenges in HDR imaging and the unique event-generating mechanism of dynamic vision sensors (DVS). Our proposed AsynHDR system integrates the DVS with a set of LCD panels. The LCD panels modulate the irradiance incident upon the DVS by altering their transparency, thereby triggering the pixel-independent event streams. The HDR image is subsequently decoded from the event streams through our temporal-weighted algorithm. Experiments under the standard test platform and several challenging scenes have verified the feasibility of the system in HDR imaging tasks.

Bath Culture: Bath Custom-Tradition in Azerbaijani National Films

Bath culture emerging with the cleansing power of water is a civilizational experience coming down from past to present. The conducted archaeological and ethnographic studies have proved that the mankind have used baths since ancient times. The bath, which was highly appreciated by all nations, has been an integral part of their cultures and lifestyles. Although this concept presented as a place is a variable to be constructed and transformed, it denotes a process determining human relations and cultural manifestations in the social structure. The spread of baths is also connected with the features of migration factors transferring the mankind’s experience, habits and lifestyle to new settlements. 
 The paper studies the emergence and development history of baths, the physiological and hygienic aspects of the effect of baths on the human body. The main role of films in the human life and the skillful illumination of bath scenes in various styles in Azerbaijani films (“Qanun namina”, “If Not That One, Then This One”, “I Want To Get Married”, “The Stab in the Back”) are noted in the article, as well.

Enhancing Computational Efficiency in Event-Based Optical Camera Communication Using N-Pulse Modulation

Event cameras are bio-inspired devices that have revolutionized the acquisition of visual information by mimicking the neural architecture of the eye. These cameras respond asynchronously to changes in scene illumination at the pixel level, providing high-precision time information with low latency, typically in the order of microseconds. In this work, we experimentally evaluate an optical camera communication (OCC) link using an LED-based transmitter and an event camera as the receiver. We propose n-pulse modulation to encode data, adapting the system to the specific characteristics and operational principles of event cameras. The proposed scheme significantly reduces the demodulation complexity compared to other alternatives found in the literature. Furthermore, a set of experiments considering different camera bias sensitivities, encoding duty cycles, and LED radiant fluxes were carried out. The results showed that the BER performance was strongly dependent on the received optical power and the bias sensitivity. In addition, duty cycles between 0.3 and 0.7 at a 200 Hz transmission frequency presented the best performance, with a BER below 1.25×10−4, which is under the forward error correction (FEC) limit. This work showcases the cutting-edge capabilities of event-camera-based OCC technology and contributes to the ongoing revolution in optical wireless communication (OWC).

Computational ghost imaging with adaptive intensity illumination for scenes featuring specular surfaces

Imaging a target scene with specular surfaces is a daunting challenge for both direct imaging and indirect computational imaging techniques. The intense specular reflection component during the measurement severely degrades the quality of the reconstructed image, resulting in a substantial loss of scene information. To address this issue, we propose a computational ghost imaging (CGI) method with adaptive intensity illumination. Capitalizing on the encoded imaging feature of CGI, this method enables effective imaging of target scenes with specular surfaces through two series of measurements, eliminating the necessity for additional optical components. Based on the position and intensity information of pixels in the specular regions from the first series of measurements, our method modulates the illumination patterns to weaken the intensity of the specular region in the second series of measurements. Simulation and experimental results demonstrate that the utilization of these modulated illumination patterns for target scene measurement effectively mitigates interference from the specular surface during imaging. Consequently, the reconstructed image is capable of presenting more detailed information about the target scene other than the specular regions. Our work introduces a novel approach for imaging target scenes with specular surfaces and broadens the scope of applications for CGI in reality.

UAV Target Tracking Algorithm Based on Illumination Adaptation and Future Awareness in Low Illumination Scenes

Aiming at problems such as tracking failure caused by illumination changes often encountered during unmanned aerial vehicle (UAV) tracking, a target tracking algorithm with illumination adaptive and future-aware correlation filters is proposed based on the background-aware correlation filters (BACF) algorithm, which realizes reliable UAV tracking tasks at night. First, the dark scene is recognized, and an efficient image enhancement module is used to enhance the brightness of low illumination images. Then a future-aware module is constructed to train the tracking model using the contextual information of the target in the next frame for better robustness. Finally, the model updating stage involves adaptive filter updating and adaptive learning rate updating to enhance target tracking precision. The results of the comparison experiments with the state-of-the-art algorithms on the UAVDark135, UAVDT, and DTB70 datasets show that the algorithm in this paper outperforms the state-of-the-art tracking methods and has better tracking performance under light changes and fast motion (FM) scenarios. The tracking speed on a single Central processing unit (CPU) reaches 49 FPS, which satisfies the real-time requirement of UAV tracking.

Research on YOLOv5 Vehicle Detection and Positioning System Based on Binocular Vision

Vehicle detection and location is one of the key sensing tasks of automatic driving systems. Traditional detection methods are easily affected by illumination, occlusion and scale changes in complex scenes, which limits the accuracy and robustness of detection. In order to solve these problems, this paper proposes a vehicle detection and location method for YOLOv5(You Only Look Once version 5) based on binocular vision. Binocular vision uses two cameras to obtain images from different angles at the same time. By calculating the difference between the two images, more accurate depth information can be obtained. The YOLOv5 algorithm is improved by adding the CBAM attention mechanism and replacing the loss function to improve target detection. Combining these two techniques can achieve accurate detection and localization of vehicles in 3D space. The method utilizes the depth information of binocular images and the improved YOLOv5 target detection algorithm to achieve accurate detection and localization of vehicles in front. Experimental results show that the method has high accuracy and robustness for vehicle detection and localization tasks.

Two-Stage Traffic Sign Classification System

Traffic Sign Recognition (TSR) has received widespread attention due to increased traffic accidents caused by a failure to recognize road traffic signs. Most Traffic Sign Recognition remains challenging due to the illumination of natural scenes, and the diversity in traffic signs [31]. To solve the problem, we proposed a two-stage classification system to overcome the above-mentioned challenges and classify the traffic sign boards efficiently. During the first phase, colour based histogram equalizations applied to eliminates the illumination factors and then traffic sign boards are classified into many shapes, which includes circular, triangular, hexagonal, and diamond traffic signs boards based on the modified HOG feature descriptors and various machine learning classifiers are compared [12]. In the second phase, circular and triangular traffic signs are further categorized into specific subclasses using circular and triangular signs trained Convolutional Neural Network (CNN) respectively to overcome intra-class variation among the similar sign board. Experimentation was conducted on German Traffic Sign Recognition Benchmark dataset (GTSRB) [30] and the results show 99.96% top-1 accuracy during the first stage of classification. In the second stage, there is accuracy of 99.40% for circular traffic signs and 97.50% for triangular traffic signs.

Numerical model of floating oil on seawater, in situ thermal structure, thickness, and remote sensing

Thermal infrared (TIR) remote sensing provides oil slick remote sensing for oil spill response. Herein, we derive oil thickness, h, from thermal contrast, ΔT, using a numerical, three-media heat-transfer model that solved the convection–diffusion equation, verified against laboratory measurements of high spatial-resolution thermal profiles for a heavy Texas Raw Crude and a light Colorado Crude, τ = 2.0 and 0.25 mm, respectively. τ is the oil attenuation length and ΔT is the temperature difference between the oil slick and adjacent oil-free water.The model studied processes and factors underlying ΔT, that arise from differing insolation absorption and the resultant heat flows and radiative emissions. The model found that the thermal profile decreased linearly through optically thin oil with and without illumination and through thick oil without illumination. For illuminated, optically-thick, oil under illumination, an internal temperature peak (greater than the air and water temperature) develops in the oil. Insolation absorption was confirmed to drive this temperature peak and thus depends on the oil τ. The model reproduced an expected asymptotic relationship of ΔT to h in the thick h limit and scales with τ. Sensitivity studies found ΔT linearly sensitive to insolation. In contrast, the air–water temperature gradient is insensitive to h for large τ and is strongly and non-linearly dependent on h for small τ,This sensitivity underlies the critical role of τ in TIR oil slick remote sensing.The model provides an investigative oil spill tool that extends existing measurements to different ambient conditions, scene illumination, and crude oils.

Neural Global Illumination: Interactive Indirect Illumination Prediction under Dynamic Area Lights.

We propose neural global illumination, a novel method for fast rendering full global illumination in static scenes with dynamic viewpoint and area lighting. The key idea of our method is to utilize a deep rendering network to model the complex mapping from each shading point to global illumination. To efficiently learn the mapping, we propose a neural-network-friendly input representation including attributes of each shading point, viewpoint information, and a combinational lighting representation that enables high-quality fitting with a compact neural network. To synthesize high-frequency global illumination effects, we transform the low-dimension input to higher-dimension space by positional encoding and model the rendering network as a deep fully-connected network. Besides, we feed a screen-space neural buffer to our rendering network to share global information between objects in the screen-space to each shading point. We have demonstrated our neural global illumination method in rendering a wide variety of scenes exhibiting complex and all-frequency global illumination effects such as multiple-bounce glossy interreflection, color bleeding, and caustics.

Computer Graphics in Lighting Design: Course Work Results of Light and Engineering Students

3D graphics computer programs have long gone beyond interactive modelling and design tools. The programs allow real-time modelling of illumination in 3D scenes, which gives a unique opportunity to use them in education. Currently, the authors are trying to use computer graphics programs in teaching lighting design to students of engineering and humanities. The methodology developed by us has been evaluated in leading universities in Moscow. Based on the methodology, we managed to form a full-fledged course on teaching lighting design, which we conducted for students of Light and Engineering sub-department of the 3rd year at the NRU “MPEI”. This article gives a summary of the course, a description of the methodology, and presents the most striking works of students. Using examples of 3D modelling of illumination, the authors demonstrate how and because of which there are changes in engineering thinking towards creativity. Students move away from classical light and engineering and begin to take the first steps in the direction of lighting design. Conclusions are drawn about the expediency of introducing and further improving the methodology when studying the direction of lighting design in both engineering and humanities universities.

SIA: RGB-T salient object detection network with salient-illumination awareness

RGB-thermal salient object detection (RGB-T SOD) has rapidly developed and achieved excellent detection results. Unfortunately, the significant impact of illumination on salient object detection has not yet been adequately addressed, which results in mediocre detection performance of current methods in variable illumination scenes. To overcome the influence of illumination in variable illumination scenes, we present an RGB-T salient object detection network with salient-illumination awareness. Firstly, we propose a salient-illumination awareness estimator (SIAE) to evaluate the illumination condition of the RGB-T images. To make the generated illumination representation more comprehensively measure the quality of the images affected by illumination, we use saliency supervision and illumination semantic complement module (ISCM) to add salient awareness and semantic information to the illumination representation, respectively. Then, under the guidance of illumination representation, we use an illumination perception fusion module (IPFM) to complete the fusion of visible light and thermal infrared modality and predict salient objects. Next, to verify the rationality of our design idea and the validity of our proposed method, we construct a variable illumination dataset VI-RGBT3500 for experimental verification. We conduct many experiments on the VI-RGBT3500 dataset as well as existing datasets. The experimental results show that our method can achieve excellent detection results in variable illumination scenes. Moreover, excellent detection results can also be achieved in normal illumination scenes. The dataset and code are available at: https://github.com/VDT-2048/SIA.

Calibration of brightness of virtual reality light sources based on user perception in the real environment

Accurately conveying brightness information and improving the reducibility of the perceived visual scene are important aspects of the efforts to promote the application of virtual reality (VR) in light environment research. In this study, brightness perception threshold (BPT) experiments with controlled, repeatable conditions were conducted in equivalent real and VR environments using the minimum change method. The experiments were conducted with the luminance, shape, size, and position of the light-emitting surfaces as variables, using 20 distinct surface forms. After 28 participants finished the assessment, the experimental data were analyzed using SPSS. The analysis shows that the initial luminance, shape, and size all have significant influence on the BPT in both real and VR environments, and the influence rules are consistent. The effect of the positions of the light-emitting surfaces on the BPT varies depending on the shape. Based on the BPT data, the Number of Perceptions (NOP) was proposed as an intermediate variable, and 15 brightness calibration curves for VR and real environment light sources were established. This study uses the consistency of visual brightness to establish the correspondence between the VR presentations’ brightness and the physical illumination of real scenes, which has implications for improving the accuracy of VR applications to light environment research and design.

NEnv: Neural Environment Maps for Global Illumination

AbstractEnvironment maps are commonly used to represent and compute far‐field illumination in virtual scenes. However, they are expensive to evaluate and sample from, limiting their applicability to real‐time rendering. Previous methods have focused on compression through spherical‐domain approximations, or on learning priors for natural, day‐light illumination. These hinder both accuracy and generality, and do not provide the probability information required for importance‐sampling Monte Carlo integration. We propose NEnv, a deep‐learning fully‐differentiable method, capable of compressing and learning to sample from a single environment map. NEnv is composed of two different neural networks: A normalizing flow, able to map samples from uniform distributions to the probability density of the illumination, also providing their corresponding probabilities; and an implicit neural representation which compresses the environment map into an efficient differentiable function. The computation time of environment samples with NEnv is two orders of magnitude less than with traditional methods. NEnv makes no assumptions regarding the content (i.e. natural illumination), thus achieving higher generality than previous learning‐based approaches. We share our implementation and a diverse dataset of trained neural environment maps, which can be easily integrated into existing rendering engines.