Low-light Environments Research Articles

Autonomous Quadrotor Flight Despite Rotor Failure With Onboard Vision Sensors: Frames vs. Events

Fault-tolerant control is crucial for safety-critical systems, such as quadrotors. State-of-art flight controllers can stabilize and control a quadrotor even when subjected to the complete loss of a rotor. However, these methods rely on external sensors, such as GPS or motion capture systems, for state estimation. To the best of our knowledge, this has not yet been achieved with only onboard sensors. In this paper, we propose the first algorithm that combines fault-tolerant control and onboard vision-based state estimation to achieve position control of a quadrotor subjected to complete failure of one rotor. Experimental validations show that our approach is able to accurately control the position of a quadrotor during a motor failure scenario, without the aid of any external sensors. The primary challenge to vision-based state estimation stems from the inevitable high-speed yaw rotation (over 20 rad/s) of the damaged quadrotor, causing motion blur to cameras, which is detrimental to visual inertial odometry (VIO). We compare two types of visual inputs to the vision-based state estimation algorithm: standard frames and events. Experimental results show the advantage of using an event camera especially in low light environments due to its inherent high dynamic range and high temporal resolution. We believe that our approach will render autonomous quadrotors safer in both GPS denied or degraded environments. We release both our controller and VIO algorithm open source.

Photogrammetric Digital Surface Model Reconstruction in Extreme Low-Light Environments

Digital surface models (DSM) have become one of the main sources of geometrical information for a broad range of applications. Image-based systems typically rely on passive sensors which can represent a strong limitation in several survey activities (e.g., night-time monitoring, underground survey and night surveillance). However, recent progresses in sensor technology allow very high sensitivity which drastically improves low-light image quality by applying innovative noise reduction techniques. This work focuses on the performances of night-time photogrammetric systems devoted to the monitoring of rock slopes. The study investigates the application of different camera settings and their reliability to produce accurate DSM. A total of 672 stereo-pairs acquired with high-sensitivity cameras (Nikon D800 and D810) at three different testing sites were considered. The dataset includes different camera configurations (ISO speed, shutter speed, aperture and image under-/over-exposure). The use of image quality assessment (IQA) methods to evaluate the quality of the images prior to the 3D reconstruction is investigated. The results show that modern high-sensitivity cameras allow the reconstruction of accurate DSM in an extreme low-light environment and, exploiting the correct camera setup, achieving comparable results to daylight acquisitions. This makes imaging sensors extremely versatile for monitoring applications at generally low costs.

Depth distribution of canopy‐forming algae of the order Fucales is related to their photosynthetic features

AbstractPhotosynthetic features and dark respiration of different species of dominant canopy‐forming algae of the Order Fucales (Phaeophyceae) consistently change with depth. Photosynthetic efficiency (α) and light at compensation (Ic) change according to the amount of light available at each depth, α increasing and Ic decreasing with depth. However, the values of light at saturation (Ik) do not show any depth pattern. Deep‐water species show higher photosynthesis at saturation (Pmax) and lower dark respiration (Rd) than shallow water ones. This result contrasts with the expected reduction of Pmax in low light environments shown by terrestrial plants but is consistent with other studies performed in macroalgae, suggesting that Pmax and α show the same trend of variation in the marine environment. The positive correlation between Pmax and α cannot be explained by differences in thickness as all tested species show an extremely high morphological similarity. The increase in Pmax with depth is also coupled to an increase of N and P content in branches, which points to nutrient‐limited photosynthesis in shallow water species.

Abundance and distribution of invasive woody shrub, Mahonia bealei, in the urban forest fragments of the southern Piedmont, USA

Woody invasive plants are a pernicious threat to the structure and function of ecosystems worldwide. Due to the growing number of invasive species and the variable timeline of invasion, many invasive plants are unrecognized or underreported, leading to long-term ecological damage that is beyond the capability of cost-efficient management programs. Mahonia bealei is a woody shrub native to China that is invading urban forests of the southeastern U.S. and is likely underreported. This study (a product of a thesis research paper, https://scholarworks.gsu.edu/geosciences_theses/137/ ) identified and mapped all occurrences of M. bealei across 40 woodlots throughout DeKalb County, Georgia, and analyzed vital indicators in the habitats that may promote colonization and establishment. M. bealei was found in 90% of sample sites, frequently in large numbers within the forest interior. The high abundance of M. bealei combined with success in low-light environments indicates an aggressive and successful long-term invasion of the Southeastern Piedmont.

Monitoring social distancing under various low light conditions with deep learning and a single motionless time of flight camera.

The purpose of this work is to provide an effective social distance monitoring solution in low light environments in a pandemic situation. The raging coronavirus disease 2019 (COVID-19) caused by the SARS-CoV-2 virus has brought a global crisis with its deadly spread all over the world. In the absence of an effective treatment and vaccine the efforts to control this pandemic strictly rely on personal preventive actions, e.g., handwashing, face mask usage, environmental cleaning, and most importantly on social distancing which is the only expedient approach to cope with this situation. Low light environments can become a problem in the spread of disease because of people’s night gatherings. Especially, in summers when the global temperature is at its peak, the situation can become more critical. Mostly, in cities where people have congested homes and no proper air cross-system is available. So, they find ways to get out of their homes with their families during the night to take fresh air. In such a situation, it is necessary to take effective measures to monitor the safety distance criteria to avoid more positive cases and to control the death toll. In this paper, a deep learning-based solution is proposed for the above-stated problem. The proposed framework utilizes the you only look once v4 (YOLO v4) model for real-time object detection and the social distance measuring approach is introduced with a single motionless time of flight (ToF) camera. The risk factor is indicated based on the calculated distance and safety distance violations are highlighted. Experimental results show that the proposed model exhibits good performance with 97.84% mean average precision (mAP) score and the observed mean absolute error (MAE) between actual and measured social distance values is 1.01 cm.

$$l_{1/2}$$ regularized joint low rank and sparse recovery technique for illumination map estimation in low light image enhancement

Insufficient illumination causes poor discernibility of visuals captured under low light environment. Often, computer vision algorithms designed to process visuals taken under normal light struggle to handle visuals with low visibility. The present work proposes a low rank approximation (LRA) based optimization model in producing the brightness enhanced version from the input low light image. The work adopts $$l_{1/2}$$ nuclear norm and sparsity minimization combined with total variation (TV) regularization to estimate the illumination map required to reconstruct the clear image. The $$l_{1/2}$$ nuclear norm serves primarily for smoothing the coarse illumination map, while TV regularization helps to preserve the prominent edge information. Additionally, $$l_{1/2}$$ regularization aids to retain the sparse structural details. The intelligence of the proposed method is bounded in the efficient formulation of a unified optimization model for the estimation of illumination map with low rank and sparsity concepts which is never reported in the direction of low light image enhancement. Furthermore, extensive experiments confirm that the proposed method outperforms the current state of art methods in low light image enhancement.

Decreased low-light regulates plant morphogenesis through the manipulation of hormone biosynthesis in Solanum lycopersicum

Low light intensity limits the growth and development of plants. How plant morphogenesis and adaptation are generated in response to reduced low-light are important for tomato plants. To investigate the light-dependent and long-term adapted regulations on tomato plants, four simulations of gradually reduced light intensity (LI), referred to as 100 % LI, 80 % LI, 67 % LI, and 40 % LI, are designed and performed in four-week-old tomato plants for two more weeks. We find that plant biomass was decreased in response to the reduced light intensity. Interestingly, the stem diameters, internode lengths, light- and CO2- response curves, maximum velocity of RuBP carboxylation by Rubisco (Vcmax), and the maximum potential rate of electron transport contributing to RuBP regeneration (Jmax) are largely affected in 40 % LI, but slightly changed in 80 % LI treatments when compared to 100 % LI control, suggesting moderately decreasing the light intensity did not substantially influence the growth and morphological phenotypes of tomato plants. The mRNA-seq results indicate that the differentially expressed genes (DEGs) are mainly enriched in the pathways corresponding to signal transductions and biosynthesis of plant hormone. We further find that the contents of jasmonic acid (JA), salicylic acid (SA) and zeatin (ZT) are significantly decreased, while content of aminocyclopropane-1-carboxylic acid (ACC) are increased in the plants treated with decreased and low light. Our results present a hypothesis that ZT functions in the regulating the plant stem diameter and internode length in decreased low-light treatments. Our study indicates that multiple plant hormones direct the morphogenesis and adaption in tomato plants in response to reduced low-light environment by manipulation of hormone biosynthesis. Hence, this study updates the understanding of formation of plant morphogenesis and regulation of endogenous hormones in the long-term adapted plants grown under decreased low-light environmental condition.

AgNWs@TiO2 and AgNPs@TiO2 Double-Layer Photoanode Film Improving Light Capture and Application under Low Illumination

In this article, silver nanowires (AgNWs) were prepared and introduced into the double-layer photoanode of dye-sensitized solar cells (DSSCs). Silver nanowires with a diameter of about 50–60 nm and a length of 1–2 mm were prepared by the polyol method. The power conversion efficiency of the double-layer photoanode DSSC made of AgNWs@TiO2 and AgNPs@TiO2 composite materials is 6.38%. Compared with the unmodified DSSC, the composite double-layer photoanode combined with AgNWs and AgNPs increased the efficiency of DSSC by 58.7%. This increased efficiency was mainly due to the localized surface plasmon resonance effect caused by AgNPs and AgNWs. The increased light collection was caused by the plasma effect of AgNPs, and it increased the short-circuit photocurrent density (JSC). The conductive properties of AgNWs improved interface charge transfer and delay charge recombination. The effect of a low light environment on DSSC efficiency was also investigated, and the best photovoltaic conversion efficiency under an irradiance of 10 mW/cm2 was found to be 8.78%.

Genomic adaptations to an endolithic lifestyle in the coral-associated alga Ostreobium

The green alga Ostreobium is an important coral holobiont member, playing key roles in skeletal decalcification and providing photosynthate to bleached corals that have lost their dinoflagellate endosymbionts. Ostreobium lives in the coral's skeleton, a low-light environment with variable pH and O2 availability. We present the Ostreobium nuclear genome and a metatranscriptomic analysis of healthy and bleached corals to improve our understanding of Ostreobium's adaptations to its extreme environment and its roles as a coral holobiont member. The Ostreobium genome has 10,663 predicted protein-coding genes and shows adaptations for life in low and variable light conditions and other stressors in the endolithic environment. This alga presents a rich repertoire of light-harvesting complex proteins but lacks many genes for photoprotection and photoreceptors. It also has a large arsenal of genes for oxidative stress response. An expansion of extracellular peptidases suggests that Ostreobium may supplement its energy needs by feeding on the organic skeletal matrix, and a diverse set of fermentation pathways allows it to live in the anoxic skeleton at night. Ostreobium depends on other holobiont members for vitamin B12, and our metatranscriptomes identify potential bacterial sources. Metatranscriptomes showed Ostreobium becoming a dominant agent of photosynthesis in bleached corals and provided evidence for variable responses among coral samples and different Ostreobium genotypes. Our work provides a comprehensive understanding of the adaptations of Ostreobium to its extreme environment and an important genomic resource to improve our comprehension of coral holobiont resilience, bleaching, and recovery.

An under-ice bloom of mixotrophic haptophytes in low nutrient and freshwater-influenced Arctic waters

The pelagic spring bloom is essential for Arctic marine food webs, and a crucial driver of carbon transport to the ocean depths. A critical challenge is understanding its timing and magnitude, to predict its changes in coming decades. Spring bloom onset is typically light-limited, beginning when irradiance increases or during ice breakup. Here we report an acute 9-day under-ice algal bloom in nutrient-poor, freshwater-influenced water under 1-m thick sea ice. It was dominated by mixotrophic brackish water haptophytes (Chrysochromulina/ Prymnesium) that produced 5.7 g C m−2 new production. This estimate represents about half the annual pelagic production, occurring below sea ice with a large contribution from the mixotrophic algae bloom. The freshwater-influenced, nutrient-dilute and low light environment combined with mixotrophic community dominance implies that phagotrophy played a critical role in the under-ice bloom. We argue that such blooms dominated by potentially toxic mixotrophic algae might become more common and widespread in the future Arctic Ocean.

Low-Light Image Restoration With Short- and Long-Exposure Raw Pairs

Low-light imaging with handheld mobile devices is a challenging issue. Limited by the existing models and training data, most existing methods cannot be effectively applied in real scenarios. In this paper, we propose a new low-light image restoration method by using the complementary information of short- and long-exposure images. We first propose a novel data generation method to synthesize realistic short- and long-exposure raw images by simulating the imaging pipeline in low-light environment. Then, we design a new long-short-exposure fusion network (LSFNet) to deal with the problems of low-light image fusion, including high noise, motion blur, color distortion and misalignment. The proposed LSFNet takes pairs of short- and long-exposure raw images as input, and outputs a clear RGB image. Using our data generation method and the proposed LSFNet, we can recover the details and color of the original scene, and improve the low-light image quality effectively. Experiments demonstrate that our method can outperform the state-of-the-art methods.

Multi-scale joint network based on Retinex theory for low-light enhancement

Due to the limitations of devices, images taken in low-light environments are of low contrast and high noise without any manual intervention. Such images will affect the visual experience and hinder further visual processing tasks, such as target detection and target tracking. To alleviate this issue, we propose a multi-scale joint low-light enhancement network based on the Retinex theory. The network consists of a decomposition part and an enhancement part. As a joint network, the decomposition and enhancement parts are mutually constrained, and the parameters are updated at the same time so that the image processing results are more excellent in detail. Our algorithm avoids the separation and recombination of decomposition and enhancement. Therefore, less information is lost in the processing of low-light images, and the enhancement result of the proposed algorithm is very close to the ground truth. In addition, in the enhancement part, we adopt a multi-scale network to fully extract image features. The multi-scale network maintains a balance between the global and local luminance of the illumination image. Retinex theory can effectively solve the problem of noise amplification and color distortion. At the same time, we have added color loss to solve the problem of color distortion, so that the enhancement result is closer to the normal-light image in color. The enhancement results are intuitively excellent, and the peak signal-to-noise ratio and structural similarity index results also reflect the reliability of the algorithm.

Research on pedestrian detection using optimized mask R-CNN algorithm in low-light road environment

Aiming to the performance degradation of the object detection algorithms in low-light environment, the image fusion module (MSRCR-IF) proposed in this paper is introduced into the object detection network based on the object detection algorithm of Mask R-CNN. This proposed fusion algorithm adjusts Region Proposal Network (RPN) and delete instance mask branch to achieve better pedestrian detection performance of algorithm in low-light environment. The experimental results reveal that the algorithm proposed in this paper has better detection performance than other current mainstream algorithms in COCO2017 data set, and the average detection accuracy of 85.05% was achieved under the self-built low-light road environment data set, which was 4.66% higher than before improvement. In order to verify the effectiveness of the improved algorithm, a real car data test was conducted, and the test results showed that the improved method can effectively improve the detection effect of pedestrians in low light conditions.

Learning-based low-illumination image enhancer for underwater live crab detection

Abstract Swift, non-destructive detection approaches should address the problem of insufficient sensitivity when attempting to obtain and perceive live crab information in low-light environments caused by the crab’s phototaxis. We propose a learning-based low-illumination image enhancer (LigED) for effective enhanced lighting and elimination of darkness in images. The camera response function was combined with the reflectance ground-truth mechanism of image decomposition. Self-attention units were then introduced in the reflectance restoration network to adjust the illumination to avoid visual defects, thus jointly strengthening the adaptability of dark-light enhancement and ability to perceive crab information. Convolutional neural network (CNN)-based detection methods can further enhance the algorithm’s robustness to light and adaptability to different environments, which motivated the development of a scalable lightweight live crab detector (EfficientNet-Det0) utilizing the two-stage compound scaling CNN approach. The lightness order error and natural image quality evaluator based on the proposed methods were 251.26 and 11.60, respectively. The quality of average precision detection increased by 13.84–95.40%. The fastest detection speed of a single image was 91.74/28.41 f·s−1 using a common GPU/CPU, requiring only 15.1 MB of storage, which advocates for the utilization of LigED and EfficientNet-Det0 for the efficient detection of underwater live crabs.

Comparative study on the adaptability of four submerged macrophytes to low light environment in Lake Erhai

水下弱光环境是水生态系统沉水植被恢复面临的常见问题，因此研究沉水植物在弱光胁迫下的适应性是必要的.本研究以云南大理州洱海南部重度退化的湖区（惯称湖心平台，曾分布有大面积沉水植物）作为实验地点，选取4种沉水植物——苦草（Vallisneria natans）、轮叶黑藻（Hydrilla verticillata）、穗花狐尾藻（Myriophyllum spicatum）、竹叶眼子菜（Potamogeton wrightii）作为研究对象，开展原位盆栽实验，通过比较4种沉水植物的生长情况来选取潜在的目标物种，为洱海湖心平台沉水植被恢复提供科学依据.结果表明：1）水下弱光环境对4种沉水植物生长均造成胁迫，生物量和株高显著性降低，物种间的相对生物量差异表现为：苦草和竹叶眼子菜>轮叶黑藻和穗花狐尾藻，相对株高差异表现为：苦草>竹叶眼子菜>穗花狐尾藻>轮叶黑藻；2）生理特征方面，苦草、穗花狐尾藻和竹叶眼子菜有相似的胁迫反应，均表现为：氮（N）和游离氨基酸含量均升高，碳氮比（C∶N比）、可溶性糖含量与淀粉含量均下降，仅苦草的叶绿素含量升高，而轮叶黑藻的生理胁迫反应表现为：氨基酸含量下降，可溶性糖和淀粉含量升高；3）四者最终存活率为：苦草>竹叶眼子菜>穗花狐尾藻>轮叶黑藻，且苦草相对生物量和相对株高高于其他3种植物.以上原位实验结果表明，与其他3种植物相比，苦草更能适应湖心平台的水下弱光环境，可考虑作为湖心平台沉水植被恢复的目标物种.;Low light environment under water is a common problem in the revegetation of submerged macrophytes. Therefore, it is necessary to study their adaptability to low light stress. The central part of the southern Lake Erhai (a mesotrophic lake in Yunnan Province, China) is flat, with an average annual water depth of about 6.3 m, which was called the “lake center platform” (LCP). Large areas of aquatic vegetation were once existed in the LCP, which had important roles in the lake ecological function but completely disappeared since 2003. Therefore, we chose the LCP as our experimental site, and select four common submerged macrophytes (Vallisneria natans, Hydrilla verticillata, Myriophyllum spicatum, and Potamogeton wrightii) as the research objects for the in-situ pot experiment, then select the potential species for revegetation by comparing their growth and final survival rates. Our results showed that: 1) the growth of the four species was stressed under the low light environment, and the biomass and plant height significantly reduced. The relative differences in biomass among species were as follows: V. natans and P. wrightii>M. spicatum and H. verticillata, and the relative differences in plant height were as follows: V. natans>P. wrightii>M. spicatum>H. verticillate; 2) In terms of physiological characteristics, V. natans, P. wrightii and M. spicatum had similar physiological response to low light stress-the contents of N and free amino acids increased, the ratio of C∶N, soluble carbohydrate and starch content decreased. Only the chlorophyll content of V. natans increased. The physiological response of H. verticillata to low light stress was as follows: the content of free amino acids decreased, and those of soluble carbohydrate and starch increased; 3) the final survival rates of four species were as follows: V. natans>P. wrightii>M. spicatum>H. verticillata. In summary, the relative biomass and plant height and final survial rate of V. natans were highest, therefore it was more suitable for underwater low light environment and could be considered as a pioneer species for revegetation in LCP.

Low irradiance amplifies negative effects of ocean acidification on recruitment of coralline algae communities

Coralline algae play foundational roles in coastal ecosystems and are globally significant components of benthic habitats down to the limits of the photic zone. Despite their vulnerability to ocean acidification (OA) and importance in low light environments, there is a limited understanding of how the interplay between irradiance and OA influences coralline reproduction and recruitment. To better understand this interaction, a 212 d experiment was run exposing coralline communities to 2 pH levels (present-day pH 8.03, OA pH 7.65) and a gradient of daily light dose (0.35, 0.17 and 0.1 mol m-2 d-1), based on in situ measurements. In the highest light dose treatment, lowered seawater pH projected for 2100 (pH(T) 7.65) reduced recruitment by 56%. This OA-driven reduction in recruitment was amplified under reduced light, with recruitment near zero in the lowest light treatment. This study shows, for the first time, the increased vulnerability of coralline community recruitment to OA under low light. Coralline algae are known to be the deepest growing macroalgae and thus, in these low light zones, OA may have the potential to reduce coralline depth distribution.

悬浮物对造礁石珊瑚营养方式的影响及其适应性研究进展

营养方式是造礁石珊瑚获取能量与营养物质的基础，影响其生长与分布。近年来珊瑚礁区悬浮物含量与组分结构发生显著变化，其对造礁石珊瑚营养方式的诸多影响正成为当前研究热点。研究系统综述了珊瑚礁区悬浮物变化特征、悬浮物对造礁石珊瑚营养方式的影响及其适应性研究现状。发现近年来人类活动加剧与强降雨事件频发是驱动珊瑚礁，尤其是近岸珊瑚礁区悬浮物含量递增、组分改变与变频加剧的主因；悬浮物变化对造礁石珊瑚光合自养与异养营养的影响存在显著的种间差异，这主要与悬浮物消光效应、生物可利用性及造礁石珊瑚种类密切相关。虽然少数种类造礁石珊瑚具光合可塑性或异养可塑性，能在高含量悬浮物存在的弱光环境中较好生长。然而对绝大多数造礁石珊瑚而言，其营养方式适应性较差，无法在悬浮物影响下较好地获取生命活动所需的能量与营养物质，进而难以生存。总体来说，悬浮物被认为是近年来影响造礁石珊瑚生长与分布的重要环境因子之一，而关于造礁石珊瑚营养方式对悬浮物变化的响应及其适应机制，当前研究仍较薄弱，需要进一步加强相关研究。;The nutritional mode is the basis for scleractinian corals to obtain energy and nutrients, which affects their growth and distribution. In recent years, the content and composition of suspended sediment in coral reefs have changed significantly, and many effects on the nutritional mode of scleractinian corals are becoming the focus of current research. This article reviewed the characteristics of the suspended sediment changes in coral reefs, the effects of suspended sediment on the nutritional mode of scleractinian corals and their adaptability. The results show that the intensification of human activities and the frequent occurrence of heavy rainfall events in recent years are the main reasons driving the increase in suspended sediment concentration, composition changes and frequency conversion in coral reefs, especially in fringing reefs. The influences of suspended sediment on autotrophy and heterotrophy of scleractinian corals are significantly different, which are mainly related to the extinction effect and bioavailability of suspended sediment, and the species of scleractinian corals. Although some species of scleractinian corals have photosynthetic plasticity or heterotrophic plasticity, they can grow well in low-light environments where high levels of suspended sediment exist. However, for the majority of scleractinian corals, the adaptability of nutritional mode is poor, so they can't obtain the energy and nutrients needed for living better under the influence of suspended sediment, which makes them difficult to survive. Overall, suspended sediment is considered to be one of the important environmental factors affecting the growth and distribution of scleractinian corals in recent years. However, the present studies on the response and adaptability mechanism of the nutritional mode of scleractinian corals to the change of suspended sediment are still rare, and further research is needed.

Red-Green Tactical Lighting Is Preferred for Suturing Wounds in a Simulated Night Environment.

Delivering medical care in nighttime conditions is challenging, as 25% of Special Operations medical Operators have reported that problems with lighting contributed to poor casualty outcomes. Red light is often used in nighttime operations but makes blood detection difficult and diminishes depth perception and visual acuity. Red-green combination lighting may be superior for differentiating blood from tissue and other fluids but had not been tested versus red-only or green-only lighting for combat-related medical procedures, such as wound suturing. Dark-adapted medical resident physicians (N = 24) sutured 6cm long, 3cm deep, full-thickness lacerations in deceased swine under red-only, green-only, and red-green lighting provided by a tactical flashlight using a randomized within-subjects design. Time to suture completion, suture quality, user ratings, and user preference data were contrasted at p < .05. This study was approved by Naval Medical Center Portsmouth IRB. Suture completion time and suture quality were similar across all lighting conditions. Participants rated red-green lighting as significantly easier for identifying blood, identifying instruments, and performing suturing (p < .01). Red-green lighting was preferred by 83% of participants compared to 8% each for red-only and green-only (p < .001). Pending further study under tactical conditions, red-green lighting is tentatively recommended for treating battlefield wounds in low-light environments.

Visible-to-Thermal Transfer Learning for Facial Landmark Detection

There has been increasing interest in face recognition in the thermal infrared spectrum. A critical step in this process is face landmark detection. However, landmark detection in the thermal spectrum presents a unique set of challenges compared to in the visible spectrum: inherently lower spatial resolution due to longer wavelength, differences in phenomenology, and limited availability of labeled thermal face imagery for algorithm development and training. Thermal infrared imaging does have the advantage of being able to passively acquire facial heat signatures without the need for active or ambient illumination in low light and nighttime environments. In such scenarios, thermal imaging must operate by itself without corresponding/paired visible imagery. Mindful of this constraint, we propose visible-to-thermal parameter transfer learning using a coupled convolutional network architecture as a means to leverage visible face data when training a model for thermal-only face landmark detection. This differentiates our approach from models trained either solely on thermal images or models which require a fusion of visible and thermal images at test time. In this work, we implement and analyze four types of parameter transfer learning methods in the context of thermal face landmark detection: Siamese (shared) layers, Linear Layer Regularization (LLR), Linear Kernel Regularization (LKR), and Residual Parameter Transformations (RPT). These transfer learning approaches are compared against a baseline version of the network and an Active Appearance Model (AAM), both of which are trained only on thermal data. We achieve a 6.5% - 9.5% improvement on the DEVCOM ARL Multi-modal Thermal Face Dataset and a 4% improvement on the RWTH Aachen University Thermal Face Dataset over the baseline model. We show that LLR, LKR, and RPT all result in improved thermal face landmark detection performance compared to the baseline and AAM, demonstrating that transfer learning leveraging visible spectrum data improves thermal face landmarking.

Robust 3D Reconstruction Using HDR-Based SLAM

3D reconstruction is an important topic in the field of the emerging applications such as smart robotics, virtual reality (VR), augmented reality (AR), and autonomous driving. RGB-D simultaneous localization and mapping (SLAM) technique is widely used in the reconstruction process. However, low light and low textured environment often results in insufficient point features and fails the reconstruction. To address this problem, we propose a robust RGB-D SLAM system using high dynamic range (HDR) image information called HDR-based SLAM. The deep learning based HDR generation method is adopted to map a single low dynamic range (LDR) image into a radiance map which is normalized to exclude the influence of exposure time. We retrained the ORB descriptor patch to fit the normalized radiance maps in the feature matching step. The proposed method can improve the quantitative camera trajectory accuracy and qualitative result of geometry reconstruction. Experimental results show that the proposed method has better performance compared to that of the standard range imaging SLAM under challenging low light environment, which helps expand the applicability of 3D reconstruction system.