Low Light Conditions Research Articles

Joint learning of RGBW color filter arrays and demosaicking

RGBW color filter arrays (CFAs) have gained widespread attention for their superior performance in low-light conditions. Most existing demosaicking methods are tailored for specific RGBW CFAs or involve manual joint design with CFAs. The close relationship between sampling and reconstruction means that restricting the search space through predefined CFAs and demosaicking methods severely limits the ability to achieve optimal performance. In this paper, we propose a new approach for joint learning of RGBW CFA and demosaicking. This approach can simultaneously learn optimal CFAs and demosaicking methods of any size, while also being capable of reconstructing mosaicked images of any size. We use a surrogate function and arbitrary CFA interpolation to ensure end-to-end learning of the RGBW CFA. We also propose a dual-branch fusion reconstruction network that utilizes the W channel to guide the reconstruction of R, G, and B channels, reducing color errors while preserving more image details. Extensive experiments demonstrate the superiority of our proposed method. Our code is available at: https://github.com/fql0528/learncfa.

Evolution of Opsin Genes in Caddisflies (Insecta: Trichoptera).

Insects have evolved complex and diverse visual systems in which light-sensing protein molecules called "opsins" couple with a chromophore to form photopigments. Insect photopigments group into three major gene families based on wavelength sensitivity: long wavelength (LW), short wavelength (SW), and ultraviolet wavelength (UV). In this study, we identified 123 opsin sequences from whole-genome assemblies across 25 caddisfly species (Insecta: Trichoptera). We discovered the LW opsins have the most diversity across species and form two separate clades in the opsin gene tree. Conversely, we observed a loss of the SW opsin in half of the trichopteran species in this study, which might be associated with the fact that caddisflies are active during low-light conditions. Lastly, we found a single copy of the UV opsin in all the species in this study, with one exception: Athripsodes cinereus has two copies of the UV opsin and resides within a clade of caddisflies with colorful wing patterns.

Enhancing single-pixel imaging reconstruction using hybrid transformer network with adaptive feature refinement

Single-pixel imaging (SPI) is a novel imaging technique that applies to acquiring spatial information under low light, high absorption, and backscattering conditions. The existing reconstruction techniques, such as pattern analysis and signal-recovery algorithms, are inefficient due to their iterative behaviors and substantial computational requirements. In this paper, we address these issues by proposing a hybrid convolutional-transformer network for efficient and accurate SPI reconstruction. The proposed model has a universal pre-reconstruction layer that can reconstruct the single-pixel measurements collected using any SPI method. Moreover, we utilize the hierarchical encoder-decoder network in U-Net architectures and employ the proposed CONText AggregatIon NEtwoRk (Container) as the adaptive feature refinement module to adaptively leverage the significance of globally and locally enhanced features in SPI reconstruction. As such, we can improve the conventional SPI methods in terms of reconstruction speed and accuracy. Extensive experiments show that the proposed model achieve a significant performance improvement as compared to traditional SPI methods digitally and experimentally while increasing the reconstruction frame rates by threefold. Moreover, the proposed model also outperforms state-of-the-art deep learning models in performing single-pixel imaging reconstruction.

Functional Design Analysis of Two Current Extended-Depth-of-Focus Intraocular Lenses.

To evaluate the differences between two extended depth-of-focus intraocular lenses, the Alcon IQ Vivity and the Bausch & Lomb LuxSmart and to compare them with a simple monofocal lens, the Alcon IQ, using a simulation-based approach. A mathematical lens model was created for each lens type based on a measured surface geometry. The lens model was then used in a raytracer to calculate a refractive power map of the lens and a ray propagation image for the focal zone. The simulations confirm the enhanced depth of focus of these two lenses. There are apparent differences between the models. For the Vivity, more light is directed into the far focus in low light conditions, whereas the LuxSmart behaves more pupil independent and prioritizes intermediate vision. The simulation-based approach was effective in evaluating and comparing the design aspects of these lenses. It can be positioned as a valuable third tool for lens characterization, complementing in vivo studies and in vitro measurements. With this approach not only focusing on the resulting optical performance, but the underlying functional mechanisms, it paves the way forward for a better adaptation to the individual needs and preferences of patients.

Sweet Immunity in Action: Unlocking Stem Reserves to Improve Yield and Quality. A Potential Key Role for Jasmonic Acid.

Common agronomic practices such as stem topping, side branch removal, and girdling can induce wound priming, mediated by jasmonic acid (JA). Low light conditions during greenhouse tomato production make the leaves more sensitive to the application of exogenous sugar, which is perceived as a "danger" in accordance with the concept of "Sweet Immunity". Consequently, source-sink balances are altered, leading to the remobilization of stem starch reserves and enabling the redirection of more carbon toward developing fruits, thereby increasing tomato yield and fruit quality. Similarities are drawn with the mobilization of fructans following defoliation of fodder grasses (wounding) and the remobilization of fructan and starch reserves under terminal drought and heat stress in wheat and rice (microwounding, cellular leakage). A central role for JA signaling is evident in all of these processes, closely intertwining with sugar signaling pathways. Therefore, JA signaling, associated with wounding and sugar priming events, offers numerous opportunities to alter source-sink balances across a broader spectrum of agricultural and horticultural crops, for instance, through the exogenous application of JA and fructans or a combination. This may entail reconfiguring and reversing phloem connections, potentially leading to an enhanced yield and product quality. Such processes may also disengage the growth-defense trade-off in plants.

Photorespiratory glycine contributes to photosynthetic induction during low to high light transition

Leaves experience near-constant light fluctuations daily. Past studies have identified many limiting factors of slow photosynthetic induction when leaves transition from low light to high light. However, the contribution of photorespiration in influencing photosynthesis during transient light conditions is largely unknown. This study employs dynamic measurements of gas exchange and metabolic responses to examine the contribution of photorespiration in constraining net rates of carbon assimilation during light induction. This work indicates that photorespiratory glycine accumulation during the early light induction contributes 5–7% to the additional carbon fixed relative to the low light conditions. Mutants with large glycine pools under photorespiratory conditions (5-formyl THF cycloligase and hydroxypyruvate reductase 1) showed a transient spike in net CO2 assimilation during light induction, with glycine buildup accounting for 22–36% of the extra carbon assimilated. Interestingly, levels of many C3 cycle intermediates remained relatively constant in both mutants and wild-type throughout the light induction period where glycine accumulated, indicating that recycling of carbon into the C3 cycle via photorespiration is not needed to maintain C3 cycle activity under transient conditions. Furthermore, our data show that oxygen transient experiments can be used as a proxy to identify the photorespiratory component of light-induced photosynthetic changes.

Regulatory mechanism of strigolactone in tall fescue to low-light stress

Strigolactone (SL), a plant hormone derived from carotenoids, has been recognized for its pivotal role in regulating plant growth. Nevertheless, the influence of SL on tall fescue (Festuca arundinacea) under low-light conditions remains unclear. This study aimed to investigate the impact of SL on various aspects of tall fescue, including its morphological characteristics, photosynthesis, levels of antioxidant and concentrations of SL, under low light intensity (LI). The findings showed that GR24, an artificial analog of SL, positively influenced several parameters of tall fescue under LI. In particular, it enhanced the morphological features such as plant height, leaf width, and biomass, while reducing the number of tillers. Furthermore, it improved the efficiency of photosynthetic by enhancing chlorophyll fluorescence and the gas exchange parameters, mitigating cell damage and improving the contents of antioxidants by increasing the levels of antioxidant enzymes and non-enzymatic antioxidant compounds. Moreover, treatment with SL led to elevated concentrations of this hormone and the levels of gene expression in related pathways. Owing to the immaturity of the genetic transformation system in tall fescue, partial validation through transgenic and mutant materials was obtained using Arabidopsis (Arabidopsis thaliana). These findings demonstrate that SL alleviates the physiological indicators of tall fescue under LI stress and enhances its tolerance to shade. Additionally, it suggests that SL may regulate the shade tolerance of tall fescue through the involvement of FaD14.

The Chrysosplenium sinicum genome provides insights into adaptive evolution of shade plants

Chrysosplenium sinicum, a traditional Tibetan medicinal plant, can successfully thrive in low-light environments for long periods of time. To investigate the adaptive evolution of shade plants in low-light environments, we generated a chromosome-scale genome assembly (~320 Mb) for C. sinicum by combining PacBio sequencing and Hi-C technologies. Based on our results, gene families related to photosynthesis and cell respiration greatly expanded and evolved in C. sinicum genome due to intracellular DNA transfer from organelle genome to nuclear genome. Under positive selective pressure, adaptive evolution of light-harvesting complex II (LHCII) component protein CsLhcb1s resulted in the expansion of threonine residues at the phosphorylation site of STN7 kinase, potentially establishing a crucial genomic foundation for enhancing C. sinicum’s adaptability in low-light environments. Through transcriptome and metabolome analysis, we identified chrysosplenol and chrysosplenoside as predominant flavonoid metabolites of C. sinicum and predicted their synthesis pathways. In addition, analysis of alternative splicing (AS) revealed that AS events help regulate state transition and flavonoid biosynthesis. The present study provides new insights into the genomes of shade plants exposed to low-light conditions and adaptive evolution of these genomes; in addition, the results improve our current knowledge on the biosynthetic and regulatory processes of chrysosplenol and chrysosplenoside.

Illumination Intelligent Adaptation and Analysis Framework: A comprehensive solution for enhancing nighttime driving fatigue monitoring.

Nighttime driving presents a critical challenge to road safety due to insufficient lighting and increased risk of driver fatigue. Existing methods for monitoring driver fatigue, mainly focusing on behavioral analysis and biometric monitoring, face significant challenges under low-light conditions. Their effectiveness, especially in dynamic lighting environments, is limited by their dependency on specific environmental conditions and active driver participation, leading to reduced accuracy and practicality in real-world scenarios. This study introduces a novel 'Illumination Intelligent Adaptation and Analysis Framework (IIAAF)', aimed at addressing these limitations and enhancing the accuracy and practicality of driver fatigue monitoring under nighttime low-light conditions. The IIAAF framework employs a multidimensional technology integration, including comprehensive body posture analysis and facial fatigue feature detection, per-pixel dynamic illumination adjustment technology, and a light variation feature learning system based on Convolutional Neural Networks (CNN) and time-series analysis. Through this integrated approach, the framework is capable of accurately capturing subtle fatigue signals in nighttime driving environments and adapting in real-time to rapid changes in lighting conditions. Experimental results on two independent datasets indicate that the IIAAF framework significantly improves the accuracy of fatigue detection under nighttime low-light conditions. This breakthrough not only enhances the effectiveness of driving assistance systems but also provides reliable scientific support for reducing the risk of accidents caused by fatigued driving. These research findings have significant theoretical and practical implications for advancing intelligent driving assistance technology and improving nighttime road safety.

Microalgal production and nutrient recovery under mixotrophic mode using cheese whey permeate

Mixotrophic microalgal solutions are efficient nutrient recovery methods, with potential to prolong the cultivation seasons in temperate climates. To improve operation sustainability, the study used landfill leachate for nitrogen source and whey permeate for phosphorus and organic carbon. A non-axenic polyculture, dominated by green algae, was cultivated in mixotrophic mode on glucose or whey permeate compared to a photoautotrophic control in outdoor pilot-scaled raceway ponds during Nordic spring and autumn. The whey permeate treatment had the highest algal growth rate and productivity (0.48 d−1, 183.8 mg L−1 d−1), nutrient removal (total nitrogen: 21.71 mg L−1 d−1, total phosphorus: 3.05 mg L−1 d−1) and recovery rate (carbon: 85.19 mg L−1 d−1, nitrogen: 17.01 mg L−1 d−1, phosphorus: 2.58 mg L−1 d−1). When grown in whey permeate, algal cultures demonstrated consistent productivity and biochemical composition in high (spring) and low light conditions (autumn), suggesting the feasibility of year-round production in Nordic conditions.

Optimizing Camera Exposure Time for Automotive Applications.

Camera-based object detection is integral to advanced driver assistance systems (ADAS) and autonomous vehicle research, and RGB cameras remain indispensable for their spatial resolution and color information. This study investigates exposure time optimization for such cameras, considering image quality in dynamic ADAS scenarios. Exposure time, the period during which the camera sensor is exposed to light, directly influences the amount of information captured. In dynamic scenarios, such as those encountered in typical driving scenarios, optimizing exposure time becomes challenging due to the inherent trade-off between Signal-to-Noise Ratio (SNR) and motion blur, i.e., extending exposure time to maximize information capture increases SNR, but also increases the risk of motion blur and overexposure, particularly in low-light conditions where objects may not be fully illuminated. The study introduces a comprehensive methodology for exposure time optimization under various lighting conditions, examining its impact on image quality and computer vision performance. Traditional image quality metrics show a poor correlation with computer vision performance, highlighting the need for newer metrics that demonstrate improved correlation. The research presented in this paper offers guidance into the enhancement of single-exposure camera-based systems for automotive applications. By addressing the balance between exposure time, image quality, and computer vision performance, the findings provide a road map for optimizing camera settings for ADAS and autonomous driving technologies, contributing to safety and performance advancements in the automotive landscape.

A bronchoscopic navigation method based on neural radiation fields.

We introduce a novel approach for bronchoscopic navigation that leverages neural radiance fields (NeRF) to passively locate the endoscope solely from bronchoscopic images. This approach aims to overcome the limitations and challenges of current bronchoscopic navigation tools that rely on external infrastructures or require active adjustment of the bronchoscope. To address the challenges, we leverage NeRF for bronchoscopic navigation, enabling passive endoscope localization from bronchoscopic images. We develop a two-stage pipeline: offline training using preoperative data and online passive pose estimation during surgery. To enhance performance, we employ Anderson acceleration and incorporate semantic appearance transfer to deal with the sim-to-real gap between training and inference stages. We assessed the viability of our approach by conducting tests on virtual bronchscopic images and a physical phantom against the SLAM-based methods. The average rotation error in our virtual dataset is about 3.18 and the translation error is around 4.95 mm. On the physical phantom test, the average rotation and translation error are approximately 5.14 and 13.12 mm. Our NeRF-based bronchoscopic navigation method eliminates reliance on external infrastructures and active adjustments, offering promising advancements in bronchoscopic navigation. Experimental validation on simulation and real-world phantom models demonstrates its efficacy in addressing challenges like low texture and challenging lighting conditions.

MPE-DETR: A multiscale pyramid enhancement network for object detection in low-light images

Object detection has broad applications in areas such as autonomous driving, security surveillance, and deep-sea exploration. However, the performance of current detection algorithms significantly decreases due to the loss of detail, increased noise, and color distortion in images under low-light or nighttime conditions. To address this problem, we propose a plug-and-play multiscale pyramid enhancement network (MPENet), which elegantly cascades with RT-DETR to establish an end-to-end framework for low-light object detection, named MPE-DETR. First, MPENet utilizes Gaussian blur to decompose images into Gaussian pyramids and Laplacian pyramids at different resolutions. Specifically, we designed a high-frequency texture enhancement (HTE) module to capture the edge and texture information of images, and a low-frequency noise smoothing (LNS) module to better understand the overall structure of images and capture global-scale features. Additionally, by concatenating the output features of the HTE and LNS modules along the channel dimension, feature fusion across different scales is realized. We conducted experiments on the ExDark and ExDark + LOD datasets, which are designed for low-light object detection. The results indicate that the proposed method achieved an improvement of 2.1% in mAP@.5 compared to that of existing SOTA models on the ExDark dataset, and demonstrated strong generalizability and robustness on the ExDark + LOD dataset. The code and results are available at https://github.com/PZDJL/MPENet.

Deciphering Optimal Radar Ensemble for Advancing Sleep Posture Prediction through Multiview Convolutional Neural Network (MVCNN) Approach Using Spatial Radio Echo Map (SREM).

Assessing sleep posture, a critical component in sleep tests, is crucial for understanding an individual's sleep quality and identifying potential sleep disorders. However, monitoring sleep posture has traditionally posed significant challenges due to factors such as low light conditions and obstructions like blankets. The use of radar technolsogy could be a potential solution. The objective of this study is to identify the optimal quantity and placement of radar sensors to achieve accurate sleep posture estimation. We invited 70 participants to assume nine different sleep postures under blankets of varying thicknesses. This was conducted in a setting equipped with a baseline of eight radars-three positioned at the headboard and five along the side. We proposed a novel technique for generating radar maps, Spatial Radio Echo Map (SREM), designed specifically for data fusion across multiple radars. Sleep posture estimation was conducted using a Multiview Convolutional Neural Network (MVCNN), which serves as the overarching framework for the comparative evaluation of various deep feature extractors, including ResNet-50, EfficientNet-50, DenseNet-121, PHResNet-50, Attention-50, and Swin Transformer. Among these, DenseNet-121 achieved the highest accuracy, scoring 0.534 and 0.804 for nine-class coarse- and four-class fine-grained classification, respectively. This led to further analysis on the optimal ensemble of radars. For the radars positioned at the head, a single left-located radar proved both essential and sufficient, achieving an accuracy of 0.809. When only one central head radar was used, omitting the central side radar and retaining only the three upper-body radars resulted in accuracies of 0.779 and 0.753, respectively. This study established the foundation for determining the optimal sensor configuration in this application, while also exploring the trade-offs between accuracy and the use of fewer sensors.

Efficient multi-level cross-modal fusion and detection network for infrared and visible image

With the rapid development of uncrewed aerial vehicle (UAV) technology, detecting aerial images has found significant applications across various domains. However, existing algorithms overlook the impact of illumination on target detection, resulting in less satisfactory detection performance under low-light conditions. We propose EfficientFuseDet, a visible and infrared image fusion detection network to overcome this issue. First, an effective multilevel cross-modal fusion network called EfficientFuse is presented to combine complementary information from both modalities better. EfficientFuse captures local dependencies and global contextual information in shallow and deep layers, seamlessly combining complimentary local and global features throughout the network. The generated fused images can exhibit clear target contours and abundant texture information. Second, we propose a detection network called AFI-YOLO, which employs an inverted residual vision transformer backbone (IRViT) to effectively address the challenges associated with background interference in fused images. We design an efficient feature pyramid network (EFPN) that efficiently integrates multiscale information, enhancing multiscale detection capability using aerial images. A reparameterization decoupling head (RepHead) is proposed to further improve target classification and localization precision. Finally, experiments on the DroneVehicle dataset indicate that the detection accuracy using fused images can reach 47.2 %, which is higher than that observed with visible light images of 45 %. Compared to state-of-the-art detection algorithms, EfficientFuseDet exhibits a slight decrease in speed. However, it demonstrates superior detection capabilities and effectively enhances the detection accuracy using aerial images under low-light conditions.

Nighttime wildlife object detection based on YOLOv8‐night

AbstractMonitoring nocturnal animals in the field is an important task in ecological research and wildlife conservation, but the complexity of nocturnal images and low light conditions make it difficult to cope with traditional image processing methods. To address this problem, researchers have introduced infrared cameras to improve the accuracy of nocturnal animal behaviour observations. Object detection in nighttime images captured by infrared cameras faces several challenges, including low image quality, animal scale variations, occlusion, and pose changes. This study proposes the YOLOv8‐night model, which effectively overcomes these challenges by introducing a channel attention mechanism in YOLOv8. The model is more focused on capturing animal‐related features by dynamically adjusting the channel weights, which improves the saliency of key features and increases the accuracy rate in complex backgrounds. The main contribution of this study is the introduction of the channel attention mechanism into the YOLOv8 framework to create a YOLOv8‐night model suitable for object detection in nighttime images. When tested on nighttime images, the model performs well with a significantly higher mAP (0.854) than YOLOv8 (0.831), and YOLOv8‐night scores 0.856 on mAP_l, which is obviously better than YOLOv8 (0.833) in terms of processing large objects. The study provides a reliable technical tool for ecological research, wildlife conservation and environmental monitoring, and offers new methods and insights for the study of nocturnal animal behaviour.

ST-Phys: Unsupervised Spatio-Temporal Contrastive Remote Physiological Measurement.

Remote photoplethysmography (rPPG) is a non-contact method that employs facial videos for measuring physiological parameters. Existing rPPG methods have achieved remarkable performance. However, the success mainly profits from supervised learning over massive labeled data. On the other hand, existing unsupervised rPPG methods fail to fully utilize spatio-temporal features and encounter challenges in low-light or noise environments. To address these problems, we propose an unsupervised contrast learning approach, ST-Phys. We incorporate a low-light enhancement module, a temporal dilated module, and a spatial enhanced module to better deal with long-term dependencies under the random low-light conditions. In addition, we design a circular margin loss, wherein rPPG signals originating from identical videos are attracted, while those from distinct videos are repelled. Our method is assessed on six openly accessible datasets, including RGB and NIR videos. Extensive experiments reveal the superior performance of our proposed ST-Phys over state-of-the-art unsupervised rPPG methods. Moreover, it offers advantages in parameter reduction and noise robustness.

High-frequency monitoring reveals phytoplankton succession patterns and the role of cryptophyte in a subtropical river reservoir

Shifts in phytoplankton biodiversity drive frequent community successions within aquatic ecosystems. While significant progress has been made in understanding the factors influencing phytoplankton growth under controlled conditions, the processes driving phytoplankton succession in natural waters remain poorly understood. To address this gap, we conducted high-frequency monitoring of the phytoplankton community in the Jiangdong River Reservoir (Southeast China) from 2021 to 2022. High-frequency data captured 17 complete cycles of phytoplankton growth and decline. Our analysis showed that cryptophytes, though not the dominant group, played a key role in shaping the temporal dynamics of phytoplankton community structure and abundance. The relative abundance of cryptophytes consistently mirrored fluctuations in total chlorophyll-a concentrations. Rainfall-induced low-light conditions enhanced the competitive advantage of cryptophytes. Throughout each cycle of phytoplankton changes, cryptophytes leveraged this short-term advantage to proliferate by utilizing available ammonia nitrogen, ascending from the fourth to the second most abundant group. However, as ammonia nitrogen levels declined to 0.1 mg/L, cryptophyte populations began to decline. These new findings highlight the critical role of cryptophytes in driving phytoplankton succession patterns.

SRCNN: Stacked-Residual Convolutional Neural Network for Improving Human Activity Classification Based on Micro-Doppler Signatures of FMCW Radar

Current methods for daily human activity classification primarily rely on optical images from cameras or wearable sensors. Despite their high detection reliability, camera-based approaches suffer from several drawbacks, such as low-light conditions, limited range, and privacy concerns. To address these limitations, this article proposes the use of a frequency-modulated continuous wave radar sensor for activity recognition. A stacked-residual convolutional neural network (SRCNN) is introduced to classify daily human activities based on the micro-Doppler features of returned radar signals. The model employs a two-layer stacked-residual structure to reuse former features, thereby improving the classification accuracy. The model is fine-tuned with different hyperparameters to find a trade-off between classification accuracy and inference time. Evaluations are conducted through training and testing on both simulated and measured datasets. As a result, the SRCNN model with six stacked-residual blocks and 64 filters achieves the best performance, with accuracies exceeding 95% and 99% at 0 dB and 10 dB, respectively. Remarkably, the proposed model outperforms several state-of-the-art CNN models in terms of classification accuracy and execution time on the same datasets.

Multi-model fusion of DSFD and Zero-DCE for low-light face detection

Due to the wide application of face detection, face recognition task under extreme lighting conditions inevitably becomes one of the challenges. Under low-light conditions, the face detection task becomes more complex and difficult due to the presence of more noise and lower visibility in the image. For this extreme environment, this paper proposes a multi-model-assisted solution that combines image enhancement and face detection models to handle this task. The DarkFace dataset was used in this study's validation of a number of distinct face identification models and picture enhancement techniques. The experimental findings demonstrate that, on this dataset, the DSFD (Dual Shot Face Detector) and Zero-DCE (Zero-Reference Deep Curve Estimation) methods perform noticeably better than the other techniques, achieving a recall of 24.61%. Therefore, these two algorithms are selected as the core processing flow in this paper to improve the facial detection system's low-light performance. This combination offers a workable answer to the problem of face detection in low light by effectively increasing the visibility of the picture and raising the accuracy of face detection.