Thermal Infrared Images Research Articles

Mitigating Modality Discrepancies for RGB-T Semantic Segmentation.

Semantic segmentation models gain robustness against adverse illumination conditions by taking advantage of complementary information from visible and thermal infrared (RGB-T) images. Despite its importance, most existing RGB-T semantic segmentation models directly adopt primitive fusion strategies, such as elementwise summation, to integrate multimodal features. Such strategies, unfortunately, overlook the modality discrepancies caused by inconsistent unimodal features obtained by two independent feature extractors, thus hindering the exploitation of cross-modal complementary information within the multimodal data. For that, we propose a novel network for RGB-T semantic segmentation, i.e. MDRNet+, which is an improved version of our previous work ABMDRNet. The core of MDRNet+ is a brand new idea, termed the strategy of bridging-then-fusing, which mitigates modality discrepancies before cross-modal feature fusion. Concretely, an improved Modality Discrepancy Reduction (MDR+) subnetwork is designed, which first extracts unimodal features and reduces their modality discrepancies. Afterward, discriminative multimodal features for RGB-T semantic segmentation are adaptively selected and integrated via several channel-weighted fusion (CWF) modules. Furthermore, a multiscale spatial context (MSC) module and a multiscale channel context (MCC) module are presented to effectively capture the contextual information. Finally, we elaborately assemble a challenging RGB-T semantic segmentation dataset, i.e., RTSS, for urban scene understanding to mitigate the lack of well-annotated training data. Comprehensive experiments demonstrate that our proposed model surpasses other state-of-the-art models on the MFNet, PST900, and RTSS datasets remarkably.

Modeling Thermal Infrared Image Degradation and Real-World Super-Resolution Under Background Thermal Noise and Streak Interference

Modeling Thermal Infrared Image Degradation and Real-World Super-Resolution Under Background Thermal Noise and Streak Interference

High-resolution thermal infrared contrails images identification and classification method based on SDGSAT-1

With the rapid development of the aviation industry, the impact of contrails on global warming is becoming increasingly significant. However, due to limitations in sensor technology and data sources, precise, all-weather observations of contrails remain a significant challenge. To address this, this study utilizes thermal imagery data from the SDGSAT-1 to classify newly formed contrails based on their length, creating SDGCONTR, the first three-band thermal infrared contrail dataset with a 30-meter spatial resolution. Based on this dataset, we propose an effective contrail image classification method: the SDGContrail Images Classification Method. This method builds on the existing Multi-axis Vision Transformer deep learning model. In the preprocessing stage, it employs an innovative contrail enhancement method by creating masks based on contrail brightness temperature and brightness temperature ratio percentiles in thermal infrared images. Our method achieved optimal classification performance compared to other methods, with the F1 Score of 95.2%. Using this model, we further analyzed the geographic and diurnal distribution patterns of long and short contrails, validating the mechanism of contrail formation. Our method not only enhances the ability to conduct large-scale, all-weather, and detailed contrail observations, but also sets a new standard for environmental monitoring and aviation safety.

Experimental investigation on a microfluidic U-turn channel for heat transfer applications

In the present work, an experimental set-up designed to allow investigations of the carrier fluid local behaviours in terms of velocity and temperature fields is presented. The study is carried out on a U-turn microfluidic channel, to bear a complete view of the thermo-fluidic system. The experiments have been performed by Infrared Thermal Imaging, which allows deriving the thermal field on the microchannel external walls, and inferring the internal thermal profiles through specific transfer functions in laminar and transitional regimes. Maps of thermal transients allow deriving cooling performances, which are used to identify the local thermal efficiency of the microchannel and relate it to the global efficiency. Velocimetry measurements have been conducted with a Micro Particle Image Velocimetry (μPIV) setup at different flow rates. This analysis has been coupled with thermal results to obtain a description of the effects of local fluid flow phenomena in transitional turbulent regime on the global heat transfer efficiency of a U shaped microchannel. The interactions among thermal and flow fields are specifically related to secondary recirculating flows, close to the turns, where high cooling rates and high magnitude of local velocity fluctuations are measured.

Destabilization damage characteristics and infrared radiation response of coal-rock complexes

To investigate the characteristics of destabilization damage in coal-rock complexes. Mechanical property tests were conducted on coal, rock, and their complexes. An infrared thermal camera was employed to real-time monitor the infrared (IR) radiation response signals during the destabilization damage process. A numerical model of coal-rock destabilization damage was developed, and its validity was verified. Deformed stress fields and displacement contours were obtained during the destabilization damage process. Upon destabilization, numerous cracks form at the base of the “coal” section, extending towards the interface, resulting in the formation of a wave-like deformation region. The differentiation in infrared thermal images is more pronounced in the “coal” section compared to the “rock” section. A high-stress region is evident at the interface, resulting in an area of high stress differentials. However, the bottom of the “coal” section also exhibits a region with high stress differentials and a more pronounced tendency towards destabilization damage. Displacement contours revealed that numerous units at the bottom of the “coal” section had slipped and misaligned, leading to the accumulation of damage and an elevation in the local damage level. It is a crucial factor contributing to the notable phenomenon of IR thermal image differentiation.

Impact of the Turbulent Vertical Mixing on Chemical and Cloud Species in the Venus Cloud Layer

AbstractThe Venusian atmosphere hosts a 10 km deep convective layer that has been studied by various spacecrafts. However, the impact of the strong vertical mixing on the chemistry of this region is still unknown. This study presents the first realistic coupling between resolved small‐scale turbulence and a chemical network. The resulting vertical mixing is different for each species: those with longer chemical timescales will tend to be well‐mixed. Vertical eddy diffusion due to resolved convection motions was estimated, ranging from 102 to 104 m2/s for the 48–55 km convective layer, several orders of magnitude above the typically used value. In the 48–55 km convective layer, the impact of the small‐scale turbulence on the cloud layer boundaries was between 200 m and 1 km. The impact of turbulence on cloud chemistry is consistent with Venus Express/Visible and Infrared Thermal Imaging Spectrometer observations. The observability at the cloud‐top of small‐scale turbulence by VenSpec‐U spectrometer would be challenging.

Synergistic retrieval of mangrove vital functional traits using field hyperspectral and satellite data

Mangroves play a pivotal role in the attainment of United Nations Sustainable Development Goals. Leaf equivalent water thickness (EWT) and leaf mass per area (LMA) are vital functional trait parameters for monitoring heath status of mangroves. However, their inherent spectral response characteristics remain unclear, which presents a challenge for high-precision estimating EWT and LMA using remote sensing images. In this study, we attempted to synergistic retrieval of mangrove EWT and LMA using field hyperspectral and satellite data. Specifically, we proposed a novel Fractional-order assisted Spectral angle Analysis (FSA) approach to accurately identify sensitive spectral domains of EWT and LMA for three mangrove species based on in situ full-spectrum hyperspectral (350–2500 nm) data. We constructed a new spectral matching index (SMI) to screen the satellite-based sensitive spectral bands based on the field-based spectral domains, and further verified their feasibility of estimating EWT and LMA using three optical satellite sensors (Sentinel-2A, SDGSAT-1, and OHS-3D). Finally, we explored the effectiveness of combining SAR and thermal infrared (TIR) images with sensitive spectral bands on improving retrieval performance in estimating EWT and LMA, respectively. Results indicated that FSA approach could capture the sensitive spectral domains of EWT (1125–1868 nm) and LMA (1419–1999 nm) for three mangrove species. The SMI could effectively realize the spectral matching between field-based sensitive domains and satellite-based bands, and perform the synergistic retrieval of mangrove EWT (R2 = 0.65) and LMA (R2 = 0.70). Intriguing finding is that the additional TIR bands significantly enhanced the EWT and LMA inversion accuracy (R2 increased by 9.8 %–68.3 %) when compared to SAR images. We confirmed that EWT displayed an inverse trend of LMA with the flooding frequency increasing. Our works provide a scientific basis for protection and sustainable management of mangroves.

Using Passive Multi-Modal Sensor Data for Thermal Simulation of Urban Surfaces

Abstract. This paper showcases an integrated workflow hinged on passive airborne multi-modal sensor data for the simulation of the thermal behavior of built-up areas with a focus on urban heat islands. The geometry of the underlying parametrized model, or digital twin, is derived from high-resolution nadir and oblique RGB, near-infrared and thermal infrared imagery. The captured bitmaps get photogrammetrically processed into comprehensive surface models, terrain, dense 3D point clouds and true-ortho mosaics. Building geometries are reconstructed from the projected point sets with procedures presupposing outlining, analysis of roof and fac¸ade details, triangulation, and texturing mapping. For thermal simulation, the composition of the ground is determined using supervised machine learning based on a modified multi-modal DeepLab v3+ architecture. Vegetation is retrieved as individual trees and larger tree regions to be added to the meshed terrain. Building materials are assigned from the available visual, infrared and surface planarity information as well as publicly available references. With actual weather data, surface temperatures can be calculated for any period of time by evaluating conductive, convective, radiative and emissive energy fluxes for triangular layers congruent to the faces of the modeled scene. Results on a sample dataset of the Moabit district in Berlin, Germany, showed the ability of the simulator to output surface temperatures of relatively large datasets efficiently. Compared to the thermal infrared images, several insufficiencies in terms of data and model caused occasional deviations between measured and simulated temperatures. For some of these shortcomings, improvement suggestions within future work are presented.

Thermal Infrared-Image-Enhancement Algorithm Based on Multi-Scale Guided Filtering

Obtaining thermal infrared images with prominent details, high contrast, and minimal background noise has always been a focal point of infrared technology research. To address issues such as the blurriness of details and low contrast in thermal infrared images, an enhancement algorithm for thermal infrared images based on multi-scale guided filtering is proposed. This algorithm fully leverages the excellent edge-preserving characteristics of guided filtering and the multi-scale nature of the edge details in thermal infrared images. It uses multi-scale guided filtering to decompose each thermal infrared image into multiple scales of detail layers and a base layer. Then, CLAHE is employed to compress the grayscale and enhance the contrast of the base layer image. Then, detail-enhancement processing of the multi-scale detail layers is performed. Finally, the base layer and the multi-scale detail layers are linearly fused to obtain an enhanced thermal infrared image. Our experimental results indicate that, compared to other methods, the proposed method can effectively enhance image contrast and enrich image details, and has higher image quality and stronger scene adaptability.

An Integrated Way to Monitor the Health Condition of Building Facades by Analyzing Thermal Infrared Images

Thermography is widely used for defect detection through analysing thermography. Thermography is used to record the surface temperatures with a thermal infrared camera. Visual interpretation can be employed to identify defects according to the differences in surface temperatures. Recently, image-processing techniques have provided a quantitative way to detect defects rather than visual interpretation. Robust principal component analysis (RPCA) is employed to analyze a series of thermal infrared images [1]. Traditionally, principal component analysis (PCA) is used to investigate a series of thermal infrared images by projecting the thermal data onto a low-dimension space such that a template image inherits the significant characteristics of the given thermal data. However, PCA introduces noise effects such that the template generated with PCA contains the noise effects. RPCA offers a similar way to project the given data onto a low-dimension space and a feature space and provides a robust way to depress the noise effects. With the introduction of RPCA, a template image can be selected from the low- dimension space, and image segmentation is employed to segment the template. Those segmented regions illustrated on the segmented template indicate the surface temperature differences, and those differences can be used as clues to identify the defects.

A Novel Multi-Effect Photosensitizer for Tumor Destruction via Multimodal Imaging Guided Synergistic Cancer Phototherapy.

How to ingeniously design multi-effect photosensitizers (PSs), including multimodal imaging and multi-channel therapy, is of great significance for highly spatiotemporal controllable precise phototherapy of malignant tumors. Herein, a novel multifunctional zinc(II) phthalocyanine-based planar micromolecule amphiphile (ZnPc 1) was successfully designed and synthesized, in which N atom with photoinduced electron transfer effect was introduced to enhance the near-infrared absorbance and nonradiative heat generation. After simple self-assembling into nanoparticles (NPs), ZnPc 1 NPs would exhibit enhanced multimodal imaging properties including fluorescence (FL) imaging (FLI) /photoacoustic (PA) imaging (PAI) /infrared (IR) thermal imaging, which was further used to guide the combined photodynamic therapy (PDT) and photothermal therapy (PTT). It was that under the self-guidance of the multimodal imaging, ZnPc 1 NPs could precisely pinpoint the tumor from the vertical and horizontal boundaries achieving highly efficient and accurate treatment of cancer. Accordingly, the integration of FL/PA/IR multimodal imaging and PDT/PTT synergistic therapy pathway into one ZnPc 1 could provide a blueprint for the next generation of phototherapy, which offered a new paradigm for the integration of diagnosis and treatment in tumor and a promising prospect for precise cancer therapy.

Nighttime Thermal Infrared Image Colorization With Feedback-Based Object Appearance Learning

Nighttime Thermal Infrared Image Colorization With Feedback-Based Object Appearance Learning

Thermal image visualization using multi-discriminator CycleGAN with unpaired thermal–visible image training set

Thermal imaging utilizes the differences in thermal radiation within a scene to capture images, effectively compensating for the limited imaging capabilities of visible-light cameras in low-light environments. In order to further enhance thermal imaging, we propose a novel approach that transforms the visualization of thermal images into an image conversion task. Our method, named TIVNet (Thermal Images Visualization Network), is based on a multi-discriminator cycle-consistent adversarial network (CycleGAN) and operates in the YUV color space to accurately colorize and represent thermal infrared images. The utilization of color and texture discriminators in our framework ensures that the converted images exhibit appropriate color styles and capture fine details, respectively. In the generator, we employed a Dense Residual Module (DRM) to facilitate the reuse of features and enhance the flow of information. By utilizing the DRM, we aim to enhance the overall performance and efficiency of the generator network. Additionally, we incorporated the Dual-channel Self-Attention Module(DCSA) to enhance the feature representation. By leveraging self-attention, the generator can effectively learn and exploit the inherent relationships between different regions, leading to improved feature representation and, consequently, more accurate colorization and representation of thermal infrared images. Notably, our proposed method leverages unsupervised learning, eliminating the need for a fully paired thermal infrared-visible image dataset. Through extensive experimentation on publicly available datasets as well as our own, we demonstrate that our approach outperforms state-of-the-art methods in thermal image visualization. Furthermore, the converted images produced by TIVNet exhibit a high level of realism, thereby enhancing the visual impact of thermal imaging. This improvement is particularly beneficial for human perception and subsequent target recognition tasks. Our findings suggest that the application of our method yields superior results both quantitatively and qualitatively, showcasing its potential for advancing thermal imaging capabilities.

Performance Analysis: Object Detection System for Low Illumination

Abstract: Abnormal activity detection plays a very important role in surveillance applications. The existing research on surveillance for daytime has achieved better performance by detecting and tracking objects using deep learning algorithms. However, it is difficult to achieve the same performance for night vision mainly due to low illumination. Deep learning is a powerful machine learning technique in which the object detector automatically learns image features required for detection tasks. It is required to generate a model that detects objects under low illumination. The approach is to use thermal infrared images and detect external objects, if any, and classify whether it is human or animal in an isolation area. With the rapid growth of deep learning, more efficient techniques will be implemented to solve the problems of object detection using neural networks and deep learning

Enhancing Diagnostic Accuracy with a Novel Computer Application for Quantitative Analysis of Bio-Medical Infrared Thermal Images

Temperature is a parameter that acts as a valuable indicator for understanding persisting disorders and illnesses in the human body. Body surface temperature is measured through the skin and the body’s internal temperature is measured through the mouth or rectum, which are used as vital information reflecting the state of thermo-regulation, a sub-process of the body's homeostasis, which is required for its normal functioning. In a state of functional imbalance, the affected region emits thermal radiation that is above or below the normal range. Thermal imaging of body regions is a beneficial means of detection of such thermal imbalances and the temperature data of each image can be analyzed quantitatively to be able to correlate the results clinically. In this article, a computer-based GUI – MedTherm Image Viewer and Analysis Tool developed in MATLAB is proposed for the processing and quantitative evaluation of thermal images for the purpose of providing supportive aid to the existing medical diagnostic procedures. The suggested graphical user interface (GUI) is beneficial in computing statistical features based on histograms of thermal images that have been recognized in numerous other studies as valuable parameters that assist in clinical diagnostic procedures.

RGBT tracking: A comprehensive review

In recent years, visual object tracking, as a prominent research area in computer vision, has garnered significant attention. To bolster the robustness of trackers across a spectrum of complex scenarios, researchers actively explore the synergistic potential of visible and thermal infrared images, aiming to design more potent tracking systems. This paper presents a comprehensive review of target tracking technology based on visible and thermal infrared information, encompassing three key aspects. Firstly, we categorize existing RGBT tracking methods into two main categories: traditional-based methods and deep learning-based methods. This classification facilitates a systematic understanding and comparison of the strengths and weaknesses of different approaches, providing a solid foundation for future research. Secondly, we focus on the evolution of RGBT datasets and analyze the performance of diverse tracking methods on these datasets. Research in this domain aids in evaluating the applicability of existing methods in real-world scenarios and offers guidance for future dataset construction. Finally, we delve into future research directions from multiple perspectives, including model design and dataset construction. In terms of model design, researchers are encouraged to explore more efficient feature extraction methods and innovative model fusion structures to further enhance tracker performance. Regarding dataset construction, increased attention should be given to ensure diversity in real-world scenarios, guaranteeing optimal tracker performance across a variety of complex conditions. In conclusion, this review makes a comprehensive analysis of the development of RGBT tracking from different perspectives, and provides a valuable reference for researchers in related fields such as multi-modal tracking and image fusion. By systematically classifying and analyzing existing research while outlining future research prospects, this review aims to foster the continued development of this field and inspire the emergence of more innovative work.

Personnel Detection in Dark Aquatic Environments Based on Infrared Thermal Imaging Technology and an Improved YOLOv5s Model.

This study presents a novel method for the nighttime detection of waterborne individuals using an enhanced YOLOv5s algorithm tailored for infrared thermal imaging. To address the unique challenges of nighttime water rescue operations, we have constructed a specialized dataset comprising 5736 thermal images collected from diverse aquatic environments. This dataset was further expanded through synthetic image generation using CycleGAN and a newly developed color gamut transformation technique, which significantly improves the data variance and model training effectiveness. Furthermore, we integrated the Convolutional Block Attention Module (CBAM) at the end of the last encoder's feedforward network. This integration maximizes the utilization of channel and spatial information to capture more intricate details in the feature maps. To decrease the computational demands of the network while maintaining model accuracy, Ghost convolution was employed, thereby boosting the inference speed as much as possible. Additionally, we applied hyperparameter evolution to refine the training parameters. The improved algorithm achieved an average detection accuracy of 85.49% on our proprietary dataset, significantly outperforming its predecessor, with a prediction speed of 23.51 FPS. The experimental outcomes demonstrate the proposed solution's high recognition capabilities and robustness, fulfilling the demands of intelligent lifesaving missions.

Estimating Fire Radiative Energy Density with Repeat-Pass Aerial Thermal-Infrared Imaging of Actively Progressing Wildfires

Studies on estimating cumulative fire intensity from spreading wildland fires based on fire radiative energy density (FRED) have primarily been conducted through controlled experiments. The objective of this study was to assess the potential for estimating FRED for freely-burning wildfires at landscape scales. Airborne thermal infrared image sequences collected 8 and 9 December 2017 during the Thomas Fire were used for surface temperature derivation and FRED estimation. Sensitivity of varying ambient temperatures, and a newly developed method that adjusts for ash radiances on fire radiative flux density (FRFD) and FRED estimates were tested. Pixel-level image classification was run to identify FRFD time sequences that were complete or incomplete because of cloud obscuration and provided the basis for an obscuration gap filling technique. Variations in estimated ambient temperature used to estimate FRFD had little impact on FRED estimates, while our ash adjustment led to notable differences. An exponential decay model characterized FRFD time sequences well, providing a basis for gap filling irregular sequences caused by atmospheric obscuration. FRED estimates were regressed on rate of spread (ROS) magnitudes and found to be positively and significantly correlated. FRED magnitudes were higher on 9 December when the Thomas Fire burned under higher wind speeds and lower relative humidity levels (Santa Ana weather conditions) than on 8 December.

Advancing human-robot collaboration: Predicting operator trajectories through AI and infrared imaging

In industrial applications demanding close interaction between humans and robots, accurate perception of the robot's surroundings is vital to adjust its behavior appropriately. This need becomes especially pertinent when manipulators operate autonomously and execute both separate and common tasks with human operators. To ensure operator safety in such Human-Robot Collaboration (HRC) environments, the present paper proposes an innovative approach that leverages advanced collaborative robotic systems capable of sensing the operator's position to effectively prevent or mitigate potential contact. Our solution combines infrared-thermal imaging technology with a workspace monitoring system, utilizing thermal and depth cameras, to track the operator's current location and predict their future trajectory using a multi-layer Long Short-Term Memory (LSTM) neural network. We also introduce a technique for simulating both human and robot movements to generate a synthetic dataset. This data is shared with the motion planning system to generate trajectories that do not intersect with the operator's future paths, minimizing downtime due to safety stoppages. The solution is integrated and evaluated within an industrial high payload collaborative robot context, and the results have shown improvements in cycle time efficiency and operator approval. This approach offers a promising direction for enhancing safety and productivity in HRC workspaces.

Geometric accuracy evaluation and analysis of ZY-1 02E IRS thermal infrared image data using GCP extraction based on phase correlation matching method

Abstract. The Ziyuan-1 (ZY-1) 02E launched on December 26, 2021 is equipped with a thermal infrared sensor (IRS), which has a ground resolution of better than 16m and a width priority of 115km, balancing the advantages of high resolution and large wide observation. The geometric performance of image data is the premise of remote sensing application, and the difficulty in evaluating the geometric performance of thermal infrared image data lies in the extraction of well-distributed, reliable and accurate GCPs. To extract GCP from high-precision reference images, it is necessary to overcome the feature differences between images caused by different spectral responses. This paper adopts a phase correlation matching method based on frequency domain to realize the fine registration of the data obtained by the emission thermal spectral band with the data from the reflectance spectral band, which can not only solve the GCP extraction of conventional thermal infrared images collected during the day, but also obtain satisfactory GCP data from thermal infrared data acquired at night. In order to test the GCP method proposed in this paper, three typical areas are selected as the experimental areas, including Yiyang City in Hunan, Nagqu City in Xizang and Hami City in Xinjiang, and the internal geometric accuracy and absolute geolocation accuracy of the thermal infrared data spanning one year are evaluated and analyzed by using the reference data composed of the DOM with an accuracy of 2m and the DEM with an accuracy of 10m. The research results indicate that the internal geometric accuracy of ZY-1 02E IRS satellite image data is better than 1.0 pixels, and the performance is satisfactory. However, its absolute geolocation accuracy needs to be continuously improved, especially there are systematic errors in the ascending data at night that require further research. Overall, it meets the design accuracy indicators of satellites and can meet the application requirements of thermal infrared remote sensing.