Feasibility study of UAV based ecological monitoring and habitat assessment of cervids in the floating meadows of Keibul Lamjao National Park in Manipur, India

Abstract. This paper deals with two aspects of photogrammetric processing of thermal images: image quality and 3D reconstruction quality. The first aspect of the paper relates to the influence of day light on Thermal InfraRed (TIR) images captured by an Unmanned Aerial Vehicle (UAV). Environmental factors such as ambient temperature and lack of sun light affect TIR image quality. We acquire image sequences of the same object during day and night and compare the generated orthophotos according to different metrics like contrast and signal-to-noise ratio (SNR). Our experiments show that performing TIR image acquisition during night time provides a better thermal contrast, regardless of whether we compute contrast over the whole image or over small patches. The second aspect investigated in this work is the potential of using TIR images for photogrammetric tasks such as the automatic generation of Digital Surface Models (DSM) and orthophotos. Due to the low geometrical resolution of a TIR camera and the low image quality in terms of contrast and noise compared to RGB images, the TIR DSM suffers from reconstruction errors and an orthophoto generated using the TIR DSM and TIR images is visibly influenced by those errors. We therefore include measurements of the UAVs positions during image capturing provided by a Global Navigation Satellite System (GNSS) receiver to retrieve position and orientation of TIR and RGB images in the same world coordinate system. To generate an orthophoto from TIR images, they are projected onto the DSM reconstructed from RGB images. This procedure leads to a TIR orthophoto of much higher quality in terms of geometrical correctness.

A thermal infrared (TIR) image is a measure of the Earth's surface temperature and TIR emittance; however, its low spatial resolution severely limits its potential applications. Image fusion techniques can be used to fuse a TIR image with higher spatial resolution reflective bands to generate a synthetic TIR image. Because of the weak correlation between TIR and reflective data, such a synthetic image typically contains significant spectral distortion. In this paper, a multivariate analysis technique is used to derive a variable as a linear function of multiple reflective bands and their non-linearly transformed versions, to produce the maximum correlation with the TIR image. The spatial details of the variable are then injected into the TIR image to yield a synthetic image with reduced spectral distortion. In an experiment on Landsat Thematic Mapper (TM) TIR and reflective data, the fusion method proposed in this paper outperforms several existing methods in preserving the spectral characteristics of TIR data.

Experimental exploration to thermal infrared imaging for detecting the transient process of solid impact

At active volcanoes, the surface temperature and its spatial distribution can indicate changes in the underlying magmatic and hydrothermal system. Surface temperature monitoring has been widely performed using thermal remote sensing with thermal infrared (TIR) cameras. One of the drawbacks of this method is that the unknown viewing orientation of TIR images inhibits quantitative evaluations of the spatial extent and distributions of thermal anomalies. Therefore, many studies have performed 3D temperature-field reconstructions by processing TIR images photogrammetrically. However, these studies have not included the correction of TIR wave atmospheric attenuation and viewing-angle variation in emissivity in the reconstruction procedure, which can result in significant temperature misestimation. We propose a simple method that incorporates the correction into the reconstruction process, which can improve the estimation of surface temperature, especially in rugged terrains. We demonstrate our method for the active crater of Aso volcano in Japan. We create digital elevation models by applying the Structure from Motion–Multi-view Stereo algorithm to aerial visible images taken at the same time as the TIR images and then project the TIR images onto the DEM with the direct georeferencing method. The correction is carried out using the geometric parameters acquired in the process. We create 1-m resolution orthorectified thermal images of the crater on two separate dates (18 August 2020 and 16 March 2022) and calculate the heat discharge rates from steaming grounds and a volcanic lake. We find that the heat discharge rate from the fumarolic field on the south crater wall showed a sevenfold increase after the phreatic explosions that occurred on 14 and 20 October 2021.Graphical

This study utilizes Thermal Infrared (TIR) imaging technology to detect hotspots in photovoltaic (PV) modules of solar power plants. Unmanned aerial vehicle (UAV)-based TIR imagery is crucial for efficiently analyzing fault detection in solar power plants. This research explores optimal operational parameters for generating high-quality TIR images using UAV technology. In addition to existing variables such as humidity, emissivity, height, wind speed, irradiance, and ambient temperature, newly considered variables including the angle of incidence between the target object and the thermal infrared camera are analyzed for their impact on TIR images. Based on the solar power plant’s tilt (20°) and the location coordinate data of the hotspot modules, the inner and outer products of the vectors were used to obtain the normal vector and angle of incidence of the solar power plant. It was discovered that the difference between measured TIR temperature data and Land Surface Temperature (LST) data varies with changes in the angle of incidence. The analysis presented in this study was conducted using multiple regression analysis to explore the relationships between dependent and independent variables. The Ordinary Least Squares (OLS) regression model employed was able to explain 63.6% of the variability in the dependent variable. Further, the use of the Condition Number (Cond. No.) and the Variance Inflation Factor (VIF) revealed that the multicollinearity among all variables was below 10, ensuring that the independence among variables was well-preserved while maintaining statistically significant correlations. Furthermore, a positive correlation was observed with the actual measured temperature values, while a negative correlation was observed between the TIR image data values and the angle of incidence. Moreover, it was found that an angle of incidence between 15° and 20° yields the closest similarity to LST temperature data. In conclusion, our research emphasizes the importance of adjusting the angle of incidence to 15–20° to enhance the accuracy of TIR imaging by mitigating overestimated TIR temperature values.

Semantic segmentation in urban scenes is widely used in applications of intelligent transportation systems (ITS). In urban scenes, thermal infrared (TIR) images can be captured in weak illumination conditions or in the presence of obscuration (e.g., light fog, smoke). Therefore, TIR images have great potential to endow automated intelligent vehicles or assist navigation systems. However, TIR imaging is blurry and low-contrast due to the absorption by atmospheric gases and heat transfer effect. Hence, TIR semantic segmentation in urban scenes has rarely been explored even though it has a wide range of scenarios in ITS. To overcome this limitation, we analyze the light transport of TIR light. Our analysis reveals that contours are the reliable features shared by TIR and Visible Spectrum (VS) light. Inspired by this, we attempt to transfer joint features from VS domain to TIR domain. Thus, we propose a curriculum domain adaptation method to guide the TIR urban scene semantic segmentation task from VS domain through contours. Moreover, to evaluate the proposed model, we build TIR-SS: an open-for-request dataset consisting of TIR images and pixel level annotations of 8 classes in urban scenes. Qualitative and quantitative experimental results on the dataset indicate that the proposed domain adaptation method outperforms related methods on this TIR semantic segmentation task.

Thermal InfraRed (TIR) image data at high temporal and spatial resolution are required to monitor the rapid development of crops during the growing season, taking into account the fragmentation of most agricultural landscapes. Moreover, integrating high-resolution satellite TIR data to calibrate hydrological models is a powerful information to efficiently monitor crop water use. Conversely, no single sensor meets these combined requirements in the TIR spectral region. Data fusion approaches offer an alternative to exploit observations from multiple sensors, providing image data to meet the combined requirements on spatial and temporal resolution.A novel spatio-temporal data fusion workflow based on a multi-sensor multi-resolution algorithm was developed and applied to generate TIR synthetic image data at high temporal and spatial resolution. The workflow includes two steps: in the first step, synthetic daily radiance images at Top of Atmosphere (TOA) and 30-m spatial resolution (at the ground) are generated using TIR radiometric data at TOA collected by the Moderate Resolution Imaging Spectroradiometer (MODIS) daily 1-km and Landsat 8/TIRS 16-day 30-m. This procedure is applied to two image pairs on different dates. The workflow yields an estimator to generate TIR TOA radiance data on any given date, provided a MODIS radiance image is available. The next step applies constrained unmixing of the 30 m (now considered as low-resolution) TIR images using the information about sub-pixel land-cover obtained from co-registered images at higher spatial resolution in the VNIR (Visible Near InfraRed) spectrum. In our case study, the L8/TIRS synthetic image data were unmixed to the Sentinel 2/MSI with 10 m × 10 m spatial resolution. Two geographically diverse experiments were carried out using the same procedure: one in The Netherlands to evaluate the procedure and other in Puglia (Italy) to generate a time series of the 10-m × 10-m TIR image data product. The validation experiment, where an actual TIRS image was applied as a reference, gave a RMSE value of 35.3 W/(m2 μm sr), which corresponds to a relative value of 8.5% against the TIRS reference values. The results confirm the feasibility of the proposed methodology, which yields a synthetic thermal band to integrate with the multi-spectral data provided by the S2/MSI at 10 m resolution.

Abstract. Thermal anomaly detection has an important role in remote sensing. One of the most widely used instruments for this task is a Thermal InfraRed (TIR) camera. In this work, thermal anomaly detection is formulated as a salient region detection, which is motivated by the assumption that a hot region often attracts attention of the human eye in thermal infrared images. Using TIR and optical images together, our working hypothesis is defined in the following manner: a hot region that appears as a salient region only in the TIR image and not in the optical image is a thermal anomaly. This work presents a two-step classification method for thermal anomaly detection based on an information fusion of saliency maps derived from both, TIR and optical images. Information fusion, based on the Dempster-Shafer evidence theory, is used in the first phase to find the location of regions suspected to be thermal anomalies. This classification problem is formulated as a multi-class problem and is carried out in an unsupervised manner on a pixel level. In the following phase, classification is formulated as a binary region-based problem in order to differentiate between normal temperature variations and thermal anomalies, while Random Forest (RF) is chosen as the classifier. In the seconds phase, the classification results from the previous phase are used as features along with temperature information and height details, which are obtained from a Digital Surface Model (DSM). We tested the approach using a dataset, which was collected from a UAV with TIR and optical cameras for monitoring District Heating Systems (DHS). Despite some limitations outlined in the paper, the presented innovative method to identify thermal anomalies has achieved up to 98.7 percent overall accuracy.

Abstract. Obtaining accurate 3d descriptions in the thermal infrared (TIR) is a quite challenging task due to the low geometric resolutions of TIR cameras and the low number of strong features in TIR images. Combining the radiometric information of the thermal infrared with 3d data from another sensor is able to overcome most of the limitations in the 3d geometric accuracy. In case of dynamic scenes with moving objects or a moving sensor system, a combination with RGB cameras of Time-of-Flight (TOF) cameras is suitable. As a TOF camera is an active sensor in the near infrared (NIR) and the thermal infrared camera captures the radiation emitted by the objects in the observed scene, the combination of these two sensors for close range applications is independent from external illumination or textures in the scene. This article is focused on the fusion of data acquired both with a time-of-flight (TOF) camera and a thermal infrared (TIR) camera. As the radiometric behaviour of many objects differs between the near infrared used by the TOF camera and the thermal infrared spectrum, a direct co-registration with feature points in both intensity images leads to a high number of outliers. A fully automatic workflow of the geometric calibration of both cameras and the relative orientation of the camera system with one calibration pattern usable for both spectral bands is presented. Based on the relative orientation, a fusion of the TOF depth image and the TIR image is used for scene segmentation and people detection. An adaptive histogram based depth level segmentation of the 3d point cloud is combined with a thermal intensity based segmentation. The feasibility of the proposed method is demonstrated in an experimental setup with different geometric and radiometric influences that show the benefit of the combination of TOF intensity and depth images and thermal infrared images.

Crops are greatly affected by the temperature of farmland surface during their growing period. It is feasible to investigate the growth status of crops based on temperature information. For serving the research of crop growth status, the component temperature (e.g. temperature of vegetation and temperature of soil) are in need to be obtained. In this study, an unmanned aerial vehicle (UAV) temperature measurement system with a thermal infrared (TIR) imager and a charge-coupled device (CCD) camera is assembled and applied to measure the brightness temperatures of farmland surface. The target areas were photographed by the UAV temperature measurement system according to a pre-set route, and obtain TIR and visible images. The component temperatures are obtained from the TIR image as following processes: (1) When shaded components are negligible at noon, two components, i.e. vegetation and soil, are divided by the OTSU algorithm; and (2) When shaded components cannot be ignored in the morning and afternoon, various components, i.e. vegetation, soil and concrete, the TIR image is divided into soil, vegetation and concrete by the corresponding classified visible images; Then, each of the components is divided into light and shaded components by the OTSU algorithm; thus, four components are obtained, including sunlit vegetation, shaded vegetation, sunlit soil, and shaded soil. The derived component temperatures can serve as inputs to agricultural and water resource models.

The impact of land use- and climate change on the managed eco-geomorphic balance in the Alps

In some applications of thermography, spatial orientation of the thermal infrared information can be desirable. By the photogrammetric processing of thermal infrared (TIR) images, it is possible to create 2D and 3D results augmented by thermal infrared information. On the augmented 2D and 3D results, it is possible to locate thermal occurrences in the coordinate system and to determine their scale, length, area or volume. However, photogrammetric processing of TIR images is difficult due to negative factors which are caused by the natural character of TIR images. Among the negative factors are the lower resolution of TIR images compared to RGB images and lack of visible features on the TIR images. To eliminate these negative factors, two methods of photogrammetric co-processing of TIR and RGB images were designed. Both methods require a fixed system of TIR and RGB cameras and for each TIR image a corresponding RGB image must be captured. One of the methods was termed sharpening and the result of this method is mainly an augmented orthophoto, and an augmented texture of the 3D model. The second method was termed reprojection and the result of this method is a point cloud augmented by thermal infrared information. The details of the designed methods, as well as the experiments related to the methods, are presented in this article.

This paper examines the recognition of real persons, mirrored persons and other objects using thermal infrared (TIR) images and radar micro-Doppler (µ-D). Mirrored persons lead to confusion of firefighters, when only a TIR camera is used. However, mirrored persons exhibit the µ- D of the mirroring objects, hence radar can resolve this ambiguity. In this paper, multiple sensor fusion architectures are investigated for this classification task. The first approach uses an attention stage, where bounding boxes of candidates for real/mirrored persons are determined in TIR images. These bounding boxes are associated to the radar targets and subsequently classified. A joint classification of the radar µ- D and TIR image at measurement level is compared to a separate classification with subsequent combination (object level). Furthermore, a classification of the complete scene is proposed, omitting the TIR attention stage and data association. Experiments with real measurements are used for an evaluation of the presented approaches.

The topsoil moisture content, its spatial distribution and dynamics, are important variables in understanding the response of arid eco‐geomorphic hill slope systems to rainfall. This study presents a method of measuring the soil moisture within a shrub's microenvironment by employing thermal infrared (TIR) imaging. A model for converting soil moisture (SMCM) is based on multi‐temporal TIR images. The method incorporates data from in situ measurements of soil temperature, moisture content and local meteorological variables collected simultaneously with TIR imaging. The results obtained, together with the high spatial resolution of the TIR images, demonstrated the efficacy of this method for mapping soil moisture on micro‐scale and also the potential for spatial analysis and change detection over time.

Fusing multi-modal information from visible (VIS) and thermal infrared (TIR) images is crucial for object detection in fully adapting to varied lighting conditions. However, the existing models usually treat VIS and TIR images as independent information and extract corresponding features from separate networks due to the scarcity of training data with labeled instances from both VIS and TIR registration images. To fill this gap, a novel Multi-Modal Fusion NETwork (M2FNet) based on the Transformer architecture is proposed in this paper, which contains two effective modules: the Union-Modal Attention (UMA) and the Cross-Modal Attention (CMA). The UMA module aggregates multi-spectral features from VIS and TIR images and then extracts multi-modal features via a convolutional neural network (CNN) backbone. The CMA module is designed to learn cross-attention features from VIS and TIR pairwise features by Transformer architecture. Evaluation results by the mean average precision (mAP) metric show that the M2FNet method significantly advances the baseline methods trained using only VIS or TIR images by 10.71 % and 2.97 %, respectively. The increments in mAP are observed in the M2FNet method compared with the existing multi-modal methods on two public datasets. Sensitivity analysis of eight illumination thresholds shows that the M2FNet method presents robustness performance on varied illumination conditions and achieves the maximum increase in accuracy of 25.6 %. Moreover, this method is subsequently applied to a new testing dataset, VI2DA (Visible-Infrared paired Video and Image DAtaset), observed by diverse sensors and platforms for testing the generalization ability of object detectors, which will be publicly available at https://github.com/TIR-OD/Datasets.