Thermal Infrared Images Research Articles

Geometric accuracy evaluation and analysis of ZY-1 02E IRS thermal infrared image data using GCP extraction based on phase correlation matching method

Abstract. The Ziyuan-1 (ZY-1) 02E launched on December 26, 2021 is equipped with a thermal infrared sensor (IRS), which has a ground resolution of better than 16m and a width priority of 115km, balancing the advantages of high resolution and large wide observation. The geometric performance of image data is the premise of remote sensing application, and the difficulty in evaluating the geometric performance of thermal infrared image data lies in the extraction of well-distributed, reliable and accurate GCPs. To extract GCP from high-precision reference images, it is necessary to overcome the feature differences between images caused by different spectral responses. This paper adopts a phase correlation matching method based on frequency domain to realize the fine registration of the data obtained by the emission thermal spectral band with the data from the reflectance spectral band, which can not only solve the GCP extraction of conventional thermal infrared images collected during the day, but also obtain satisfactory GCP data from thermal infrared data acquired at night. In order to test the GCP method proposed in this paper, three typical areas are selected as the experimental areas, including Yiyang City in Hunan, Nagqu City in Xizang and Hami City in Xinjiang, and the internal geometric accuracy and absolute geolocation accuracy of the thermal infrared data spanning one year are evaluated and analyzed by using the reference data composed of the DOM with an accuracy of 2m and the DEM with an accuracy of 10m. The research results indicate that the internal geometric accuracy of ZY-1 02E IRS satellite image data is better than 1.0 pixels, and the performance is satisfactory. However, its absolute geolocation accuracy needs to be continuously improved, especially there are systematic errors in the ascending data at night that require further research. Overall, it meets the design accuracy indicators of satellites and can meet the application requirements of thermal infrared remote sensing.

Convolutional Neural Network to Classify Infrared Thermal Images of Fractured Wrists in Pediatrics.

Convolutional neural network (CNN) models were devised and evaluated to classify infrared thermal (IRT) images of pediatric wrist fractures. The images were recorded from 19 participants with a wrist fracture and 21 without a fracture (sprain). The injury diagnosis was by X-ray radiography. For each participant, 299 IRT images of their wrists were recorded. These generated 11,960 images (40 participants × 299 images). For each image, the wrist region of interest (ROI) was selected and fast Fourier transformed (FFT) to obtain a magnitude frequency spectrum. The spectrum was resized to 100 × 100 pixels from its center as this region represented the main frequency components. Image augmentations of rotation, translation and shearing were applied to the 11,960 magnitude frequency spectra to assist with the CNN generalization during training. The CNN had 34 layers associated with convolution, batch normalization, rectified linear unit, maximum pooling and SoftMax and classification. The ratio of images for the training and test was 70:30, respectively. The effects of augmentation and dropout on CNN performance were explored. Wrist fracture identification sensitivity and accuracy of 88% and 76%, respectively, were achieved. The CNN model was able to identify wrist fractures; however, a larger sample size would improve accuracy.

Sparse matching via point and line feature fusion for robust aerial triangulation of photovoltaic power stations’ thermal infrared imagery

Abstract. In this paper, we present a novel matching method tailored for unmanned aerial vehicle (UAV) thermal infrared images of photovoltaic (PV) panels characterized by highly repetitive textures. This method capitalizes on the integration of point and line features within the image to obtain reliable corresponding points. Furthermore, it employs multiple constraints to eliminate mismatched features and get rid of the interference of repetitive textures on feature matching. To verify the effectiveness of the proposed method, we used an UAV equipped with the DJI Zenmuse H20T thermal infrared gimbal to capture 3767 images of a PV power station in Guangzhou, China. Experiments demonstrate that, for UAV thermal infrared images of PV panels, our method outperforms the state-of-the-art techniques in terms of the density of matching points, matching success rate and matching reliability, consequently leading to robust aerial triangulation results.

Effect of Incidence Angle on Temperature Measurement of Solar Panel with Unmanned Aerial Vehicle-Based Thermal Infrared Camera

This study utilizes Thermal Infrared (TIR) imaging technology to detect hotspots in photovoltaic (PV) modules of solar power plants. Unmanned aerial vehicle (UAV)-based TIR imagery is crucial for efficiently analyzing fault detection in solar power plants. This research explores optimal operational parameters for generating high-quality TIR images using UAV technology. In addition to existing variables such as humidity, emissivity, height, wind speed, irradiance, and ambient temperature, newly considered variables including the angle of incidence between the target object and the thermal infrared camera are analyzed for their impact on TIR images. Based on the solar power plant’s tilt (20°) and the location coordinate data of the hotspot modules, the inner and outer products of the vectors were used to obtain the normal vector and angle of incidence of the solar power plant. It was discovered that the difference between measured TIR temperature data and Land Surface Temperature (LST) data varies with changes in the angle of incidence. The analysis presented in this study was conducted using multiple regression analysis to explore the relationships between dependent and independent variables. The Ordinary Least Squares (OLS) regression model employed was able to explain 63.6% of the variability in the dependent variable. Further, the use of the Condition Number (Cond. No.) and the Variance Inflation Factor (VIF) revealed that the multicollinearity among all variables was below 10, ensuring that the independence among variables was well-preserved while maintaining statistically significant correlations. Furthermore, a positive correlation was observed with the actual measured temperature values, while a negative correlation was observed between the TIR image data values and the angle of incidence. Moreover, it was found that an angle of incidence between 15° and 20° yields the closest similarity to LST temperature data. In conclusion, our research emphasizes the importance of adjusting the angle of incidence to 15–20° to enhance the accuracy of TIR imaging by mitigating overestimated TIR temperature values.

Digital Infrared Thermal Imaging Assistance for Laryngectomy and Neck Dissection Flap Monitoring in Elderly Patients

Digital Infrared Thermal Imaging Assistance for Laryngectomy and Neck Dissection Flap Monitoring in Elderly Patients

Highly compact adaptive network based on transformer for RGBT tracking

RGBT tracking is a challenging task that requires robust fusion of visible images (RGB) and thermal infrared images (TIR) to handle various scenarios, such as illumination changes, occlusions, and camouflage. The current popular RGBT trackers mostly based on two stream Siamese trackers. They tend to separately extracts template and search images region features, which neglects relationship between target and background. Moreover, they do not fuse RGB and TIR features properly, limiting their ability to utilize complementary features. To address these issues, we introduce a highly compact adaptive transformer-based network that unifies the process of feature extracting and correlation between template and search images. In this way, the compact network has dual-branches. They can combine feature extraction and correlation for different modalities of template images and search images. Meanwhile, we introduce a cross-modal weight redistribution module (CMWR) for multi-modal fusion. This adaptive fusion scheme learns discriminative features of RGB and TIR data and assigns weights to them, enabling them to complement each other. Furthermore, to address the issue of tracking targets of different scales, we design scale-adaptive optimization pyramid module (SAOP) that adapt to objects of different sizes. Our method achieves exceptional performance on the GTOT, RGBT234 and LasHeR datasets, surpassing most of the existing methods. The results are consistent across multiple datasets, demonstrating the effectiveness and superiority of our approach. And our code is released at: https://github.com/ELOESZHANG/HCANet.

LAGSwin: Local attention guided Swin-transformer for thermal infrared sports object detection.

Compared with visible light images, thermal infrared images have poor resolution, low contrast, signal-to-noise ratio, blurred visual effects, and less information. Thermal infrared sports target detection methods relying on traditional convolutional networks capture the rich semantics in high-level features but blur the spatial details. The differences in physical information content and spatial distribution of high and low features are ignored, resulting in a mismatch between the region of interest and the target. To address these issues, we propose a local attention-guided Swin-transformer thermal infrared sports object detection method (LAGSwin) to encode sports objects' spatial transformation and orientation information. On the one hand, Swin-transformer guided by local attention is adopted to enrich the semantic knowledge of low-level features by embedding local focus from high-level features and generating high-quality anchors while increasing the embedding of contextual information. On the other hand, an active rotation filter is employed to encode orientation information, resulting in orientation-sensitive and invariant features to reduce the inconsistency between classification and localization regression. A bidirectional criss-cross fusion strategy is adopted in the feature fusion stage to enable better interaction and embedding features of different resolutions. At last, the evaluation and verification of multiple open-source sports target datasets prove that the proposed LAGSwin detection framework has good robustness and generalization ability.

3-D Reconstruction for Thermal Infrared Images of Substation Equipment Based on Visual SLAM and Cascade MVSNet

3-D Reconstruction for Thermal Infrared Images of Substation Equipment Based on Visual SLAM and Cascade MVSNet

A Defect Data Compensation Model for Infrared Thermal Imaging Based on Bi-LSTM with Attention Mechanism

A Defect Data Compensation Model for Infrared Thermal Imaging Based on Bi-LSTM with Attention Mechanism

Retraction Note: Memory-guided visual attention generative adversarial network for colorization of nighttime thermal infrared images

Retraction Note: Memory-guided visual attention generative adversarial network for colorization of nighttime thermal infrared images

In defense and revival of Bayesian filtering for thermal infrared object tracking

Deep learning-based methods monopolize the latest research in the field of thermal infrared (TIR) object tracking. However, relying solely on deep learning models to obtain better tracking results requires carefully selecting feature information that is beneficial to representing the target object and designing a reasonable template update strategy, which undoubtedly increases the difficulty of model design. Thus, recent TIR tracking methods face many challenges in complex scenarios. This paper introduces a novel Deep Bayesian Filtering (DBF) method to enhance TIR tracking in these challenging situations. DBF is distinctive in its dual-model structure: the system and observation models. The system model leverages motion data to estimate the potential positions of the target object based on two-dimensional Brownian motion, thus generating a prior probability. Following this, the observation model comes into play upon capturing the TIR image. It serves as a classifier and employs infrared information to ascertain the likelihood of these estimated positions, creating a likelihood probability. According to the guidance of the two models, the position of the target object can be determined, and the template can be dynamically updated. Experimental analysis across several benchmark datasets reveals that DBF achieves competitive performance, surpassing most existing TIR tracking methods in complex scenarios.

Boron nitride nanosheets from high shear exfoliation for high-performance thermal conductivity films

Boron nitride nanosheets (BNNSs) with intrinsically ultra-high thermal conductivity have attracted more and more attention and various methods have been proposed to prepare BNNSs-based thermally conductive adhesives. In this work, a liquid phase shear exfoliation process was proposed to prepare BNNSs. BNNSs can be directly prepared by exfoliating h-BN particles dispersed in isopropanol at a controlled rotational speed range of 16,000–22,000 rpm and the maximum yield of BNNSs is up to 32.1%. After the BNNSs were modified with polydopamine (M-BNNSs), the dispersion performance of BNNSs in water was significantly improved. BNNSs were incorporated into a cellulose nanofiber-based adhesive, and the highly thermally conductive films were prepared by a flow-delay process. A tested in-plane thermal conductivity of up to 23.23 W·m−1·K−1 and excellent mechanical properties are demonstrated and thermal infrared images provide a more visual representation of the high thermal conductivity and heat dissipation properties of the film, indicating the promising application of the film in the field of thermal conductivity.

Identification of Industrial Heat Source Production Areas Based on SDGSAT-1 Thermal Infrared Imager

Industrial heat sources (IHSs) are key contributors to anthropogenic heat, air pollution, and carbon emissions. Accurately and automatically detecting their production areas (IHSPAs) on a large scale is vital for environmental monitoring and decision making, yet this is challenged by the lack of high-resolution thermal data. Sustainable Development Science Satellite 1 (SDGSAT-1) thermal infrared spectrometer (TIS) data with the highest resolution (30 m) in the civilian field and a three-band advantage were first introduced to detect IHSPAs. In this study, an IHSPA identification model using multi-features extracted from SDGSAT-1 TIS and Landsat OLI data and support vector machine (SVM) was proposed. First, three brightness temperatures and four thermal radiation indices using SDGSAT-1 TIS and Landsat OLI data were designed to enlarge the temperature difference between IHSPAs and the background. Then, 10 features combined with three indices from Landsat OLI images with the same spatial resolution (30 m) and stable data were extracted. Second, an IHSPA identification model based on SVM and multi-feature extraction was constructed to identify IHSPAs. Finally, the IHS objects were manually delineated and verified using the identified IHSPAs and Google Earth images. Some conclusions were obtained from different comparisons in Wuhai, China: (1) IHSPA identification based on SVM using thermal and optical features can detect IHSPAs and obtain the best results compared with different features and identification models. (2) The importance of using thermal features from the SDGSAT-1 TIS to detect IHSPAs was demonstrated by different importance analysis methods. (3) Our proposed method can detect more IHSs, with greater spatial coverage and smaller areas, compared with the methods of Ma and Liu. This new way to detect IHSPAs can obtain higher-spatial-resolution emissions of IHSs on a large scale and help decision makers target environmental monitoring, management, and decision making in industrial plant processing.

PiDiNet-TIR: An improved edge detection algorithm for weakly textured thermal infrared images based on PiDiNet

In order to solve the problems such as difficulties in extracting image edge information caused by low contrast and blurred edges of thermal infrared images, a thermal infrared image edge detection algorithm PiDiNet-TIR constructed based on the PiDiNet model is proposed. On the basis of the visible light image dataset BSDS500, the visible light image is grayed out, noise is added, and contrast is reduced to construct the thermal infrared image edge dataset I R-BSDS500, which provides image data for model training and testing. Deeply study the theory of PiDiNet model, add rich convolution mechanism to its backbone network, improve the activation function, pooling layer and up-sampling module, study the SimAM attention mechanism and add it to the side network. Using the open-source thermal infrared image dataset and homemade thermal infrared images for testing, we compare and analyze the mean square error (MSE), structural similarity of image (SSIM), feature similarity of image (FSIM), and Frames Per Second (FPS)of different detection algorithms, such as HED, RCF, DexiNet, PiDiNet, and the algorithm proposed in this paper. The results of image evaluation parameters prove that the edge detection algorithm proposed in this paper has high feature extraction efficiency and feature expression ability, and the algorithm detection performance is higher than other algorithms.

Microwave fracturing of frozen coal with different water content: Pore-structure evolution and temperature characteristics

Liquid nitrogen (LN2) freezing has widely been studied for the production of coal-bed methane (CBM) in coal reservoirs, but there is little study involving the thawing process. In this paper, the effects of microwave thawing on the LN2-freezing coals with different water content are studied through magnetic resonance imaging (NMR), p-wave velocity, and infrared thermal imaging (ITI). The results prove that the increase rates of the seepage pores of the air-thawing samples increase with the increasing water content. Compared with air thawing, the increase rates of the seepage pores after microwave thawing increase by 17.37%, 9.56%, and 9.32% for the dried, wet, and saturated conditions, respectively. The results of the wave velocity are similar to the increased rates of the seepage pores. The uniformity of the surface temperature of the samples thawed by microwave is related to the initial water content. The low initial water content contributes to the development of surface cracks during LN2 freezing. After microwave thawing, the pore water reduces by 21.46%, 44.85%, and 21.31% for the dried sample, wet sample, and saturated sample, respectively. The research shows that microwave thawing of the frozen coal not only enhances the pore structures but also removes the pore water.

Fine Pitch Micro Indium Bump Interconnect Flip Chip Bonding

Quantum computing processors, micro LED displays and Focal Plane Array (FPA) imaging and detector devices such as infrared (IR) thermal imaging sensors are seeing higher demand as more practical applications requiring these components are coming into research and development, industrial, military and consumer markets. This paired with higher pixel and Qubit count and interconnect density, on larger and larger chips is driving hybridization and monolithic integration in these technologies. This is showing a marked increase in demand for fine pitch micro bump interconnection flip chip die bonding. However, some critical challenges facing these technologies are; larger component sizes mean higher density interconnections over increasing surface area. Submicron accuracy required to align fine pitch micro interconnect arrays. This together with the challenges facing the materials that are becoming the industry standard for these applications, such as the requirement for the assembled components to remain stable in extreme conditions such as cryogenic application environments. Combined with low loss high strength mechanical/ electrical interconnect requirements on components containing sensitive materials, structures and unmatched coefficient of thermal expansion (CTE) means that processing gases such as formic acid or high temperature reflow bonding can no longer be used to bond these devices. These challenges mean that the industry is fast approaching the limitations of even state-of-the-art die bonders and die bonding methods on the market today. This paper is going to highlight these challenges and the methods used to address them to produce large format, high density IR thermal imaging FPA devices, Quantum processors and micro LED displays using fine pitch micro Indium bump array interconnections that meet today’s industry requirements.

Feasibility study of UAV based ecological monitoring and habitat assessment of cervids in the floating meadows of Keibul Lamjao National Park in Manipur, India

Technological interventions such as remote sensing can play a crucial role in conserving inaccessible wetlands and their associated species. The floating meadows of Keibul Lamjao National Park (KLNP) in Manipur, India, are the only home to the last remaining wild population of the endangered and endemic Eld’s Deer, locally known as Sangai (Rucervus eldii eldii). Besides the rampant change in land use and altering water regimes induced by anthropogenic pressure, monitoring the population and habitat quality remains challenging. Moreover, conventional in-situ surveys that involve infiltrating animal habitats could adversely affect the habits and behavior of such shy species as a response to human presence. This study assesses the scope of Uncrewed Aerial Vehicles (UAVs) to non-invasively access such a challenging habitat and detect the ungulate individuals of interest. A quadcopter, mounted with an optical (RGB) and a thermal camera, was tested in this landscape conditions for its durability, failsafe range, flexibility, camera capabilities for visualization and detection, and resolution. The results of the drone survey based on three different time frames suggest that it is feasible to soar through the floating meadows to detect deer, best during dawn and dusk. Detection yielded the best results without creating a disturbance in animal behavior, with thermal infrared (TIR) imaging at night and both TIR and optical during the early morning. A flying height of nearly 90 m with RGB for vegetation and 20–25 m with TIR for animal presence yielded the best results. We have also established the least distracting sampling technique by analyzing the behavioral response of the deer to the presence of a drone. This study has paved a safe and practical way for acquiring accurate ground information that can assist in population estimation, habitat assessment, and burnt area estimation due to wildfire, to aid the conservation of an invaluable natural asset.

A Review of Techniques and Bio-Heat Transfer Models Supporting Infrared Thermal Imaging for Diagnosis of Malignancy

The present review aims to analyze the application of infrared thermal imaging, aided by bio-heat models, as a tool for the diagnosis of skin and breast cancers. The state of the art of the related technical procedures, bio-heat transfer modeling, and thermogram post-processing methods is comprehensively reviewed. Once the thermal signatures of different malignant diseases are described, the updated thermographic techniques (steady-state and dynamic) used for cancer diagnosis are discussed in detail, along with the recommended best practices to ensure the most significant thermal contrast observable between the cancerous and healthy tissues. Regarding the dynamic techniques, particular emphasis is placed on innovative methods, such as lock-in thermography, thermal wave imaging, and rotational breast thermography. Forward and inverse modeling techniques for the bio-heat transfer in skin and breast tissues, supporting the thermographic examination and providing accurate data for training artificial intelligence (AI) algorithms, are reported with a special focus on real breast geometry-based 3D models. In terms of inverse techniques, different data processing algorithms to retrieve thermophysical parameters and growth features of tumor lesions are mentioned. Post-processing of infrared images is also described, citing both conventional processing procedures and applications of AI algorithms for tumor detection.

Optimization of multi-dimensional indices for kiwifruit orchard soil moisture content estimation using UAV and ground multi-sensors

Accurately estimating of soil moisture content (SMC) is essential for effective irrigation water management and optimizing plant water productivity. Recent advancements in multi-sensor platforms and ensemble learning (EL) algorithms such as the use of UAV-based multispectral (MS) and thermal infrared (TIR) images, have enabled precise mapping of SMC at the field level. The purpose of this study is to optimize the multi-dimensional (n-D) index set and EL model to achieve accurate inversion mapping of SMC in the 0–60 cm root zone at a kiwifruit orchard during the fruit growth period (May-Sep 2022). The n-D index set was computed using kiwifruit canopy MS and TIR data from UAV along with in-field temperature data (air temperature [Ta], kiwifruit canopy temperature [Tc], soil temperature [Ts], and Ta-Tc, Ta-Ts, Tc-Ts). The primary findings of this study are as follows: (1) The Three-Dimensional Drought Index (TDDI), Temperature Vegetation Drought Index (TVDI), and Crop Water Stress Index (CWSI) demonstrated superior performance in capturing temporal variations of root zone SMC compared to Normalized Difference Vegetation Index (NDVI) and Enhance Vegetation Index (EVI). (2) The optimal Categorical Boosting (Catboost) model with two optimal TDDIs (TDDI22: (Tc-Ts)-EVI10,6,2-CWSI space and TDDI7: Tc-NDVI7,3-CWSI space), exhibited exceptional performance for SMC estimation, with determination coefficient (R2) and root-mean-square error (RMSE) of 0.966 ± 0.010 and 0.046 ± 0.003%, respectively. (3) The planted-by-planted mapping performed well in estimating root zone SMC at different growth stages and under different irrigation treatments, with a correlation coefficient (R) of 0.936 ± 0.074 (p < 0.001). This study demonstrated that the integration of multi-sensor indicators and EL models can improve the accuracy of SMC estimation due to its robust adaptability to various field conditions. Furthermore, the planted-grid-based SMC mapping framework can be considered as a valuable approach for monitoring dynamic SMC, facilitating informed irrigation decisions at the individual planting grid.

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Nighttime Thermal InfraRed (NTIR) image colorization, also known as the translation of NTIR images into Daytime Color Visible (DCV) images, can facilitate human and intelligent system perception of nighttime scenes under weak lighting conditions. End-to-end neural networks have been used to learn the mapping relationship between temperature and color domains, and translate NTIR images with one channel into DCV images with three channels. However, this mapping relationship is an ill-posed problem with multiple solutions without constraints, resulting in blurred edges, color disorder, and semantic errors. To solve this problem, an NTIR2DCV method that includes two steps is proposed: firstly, fuse Nighttime Color Visible (NCV) images with NTIR images based on an Illumination-Aware, Multilevel Decomposition Latent Low-Rank Representation (IA-MDLatLRR) method, which considers the differences in illumination conditions during image fusion and adjusts the fusion strategy of MDLatLRR accordingly to suppress the adverse effects of nighttime lights; secondly, translate the Nighttime Fused (NF) image to DCV image based on HyperDimensional Computing Generative Adversarial Network (HDC-GAN), which ensures feature-level semantic consistency between the source image (NF image) and the translated image (DCV image) without creating semantic label maps. Extensive comparative experiments and the evaluation metrics values show that the proposed algorithms perform better than other State-Of-The-Art (SOTA) image fusion and translation methods, such as FID and KID, which decreased by 14.1 and 18.9, respectively.