Uncooled Thermal Camera Research Articles

Gas Imaging with Uncooled Thermal Imager.

Gas imaging has become one of the research hotspots in the field of gas detection due to its significant advantages, such as high efficiency, large range, and dynamic visualization. It is widely used in industries such as natural gas transportation, chemical, and electric power industries. With the development of infrared detector technology, uncooled thermal imagers are undergoing a developmental stage of technological advancement and widespread application. This article introduces a gas imaging principle and radiation transfer model, focusing on passive imaging technology and active imaging technology. Combined with the actual analysis, the application scenarios using uncooled thermal imaging cameras for gas imaging measurement are analyzed. Finally, the limitations and challenges of the development of gas imaging technology are analyzed.

Self-Calibration of UAV Thermal Imagery Using Gradient Descent Algorithm

Unmanned aerial vehicle (UAV) thermal imagery offers several advantages for environmental monitoring, as it can provide a low-cost, high-resolution, and flexible solution to measure the temperature of the surface of the land. Limitations related to the maximum load of the drone lead to the use of lightweight uncooled thermal cameras whose internal components are not stabilized to a constant temperature. Such cameras suffer from several unwanted effects that contribute to the increase in temperature measurement error from ±0.5 °C in laboratory conditions to ±5 °C in unstable flight conditions. This article describes a post-processing procedure that reduces the above unwanted effects. It consists of the following steps: (i) devignetting using the single image vignette correction algorithm, (ii) georeferencing using image metadata, scale-invariant feature transform (SIFT) stitching, and gradient descent optimisation, and (iii) inter-image temperature consistency optimisation by minimisation of bias between overlapping thermal images using gradient descent optimisation. The solution was tested in several case studies of river areas, where natural water bodies were used as a reference temperature benchmark. In all tests, the precision of the measurements was increased. The root mean square error (RMSE) on average was reduced by 39.0% and mean of the absolute value of errors (MAE) by 40.5%. The proposed algorithm can be called self-calibrating, as in contrast to other known solutions, it is fully automatic, uses only field data, and does not require any calibration equipment or additional operator effort. A Python implementation of the solution is available on GitHub.

Evaluation of Thermal Imaging on Embedded GPU Platforms for Application in Vehicular Assistance Systems

This study is focused on evaluating the real-time performance of thermal object detection for smart and safe vehicular systems by deploying the trained networks on GPU & single-board EDGE-GPU computing platforms for onboard automotive sensor suite testing. A novel large-scale thermal dataset comprising of > 35,000 distinct frames is acquired, processed, and open-sourced in challenging weather and environmental scenarios. The dataset is a recorded from lost-cost yet effective uncooled LWIR thermal camera, mounted stand-alone and on an electric vehicle to minimize mechanical vibrations. State-of-the-art YOLO-V5 networks variants are trained using four different public datasets as well newly acquired local dataset for optimal generalization of DNN by employing SGD optimizer. The effectiveness of trained networks is validated on extensive test data using various quantitative metrics which include precision, recall curve, mean average precision, and frames per second. The smaller network variant of YOLO is further optimized using TensorRT inference accelerator to explicitly boost the frames per second rate. Optimized network engine increases the frames per second rate by 3.5 times when testing on low power edge devices thus achieving 11 fps on Nvidia Jetson Nano and 60 fps on Nvidia Xavier NX development boards.

A Multi-Spectral Thermal Gas Detection Imager Using Uncooled Infrared Camera

Gas remote detection is useful for early warning of gas leakage and toxic chemicals. Optical gas imaging (OGI) built with an uncooled infrared camera is superior to cooled detectors in terms of cost. Current mainstream OGI technologies fall short in their detection of gases at ambient temperature and their ability to classify multiple gases. A multi-spectral uncooled imager is developed to try to solve these problems, which is constructed from a commercial uncooled thermal camera and wide band filters. To solve filter self-radiation and unevenness, a correction method is devised, with an ambient temperature blackbody placed in front and subtracted from the measured image. Based on waveband cutoffs, filters are classified into target-sensitive filters and background filters. Multi-spectra are simulated according to wide band filter transmittance, which can be used in gas classification. A sulfur hexafluoride (SF6) experiment is conducted outdoors at a distance of 10 m. An SVM model is trained to classify gas release in real time. Detection with a cold sky background is improved with the aid of data cube differences in a time sequence. The SF6 outdoor experiment concluded with preliminary effective results of ambient temperature gas remote detection.

A Case Study of Vignetting Nonuniformity in UAV-Based Uncooled Thermal Cameras

Uncooled thermal cameras have been employed as common UAV payloads for aerial temperature surveillance in recent years. Due to the lack of internal cooling systems, such cameras often suffer from thermal-drift-induced nonuniformity or vignetting despite having built-in mechanisms to minimize the noise. The current study examined a UAV-based uncooled thermal camera vignetting regarding camera warmup time, ambient temperature, and wind speed and direction, and proposed a simple calibration-based vignetting migration method. The experiments suggested that the camera needed to undergo a warmup period to achieve stabilized performance. The required warmup duration ranged from 20 to 40 min depending on ambient temperature. Camera vignetting severity increased with camera warmup time, decreasing ambient temperature, and wind presence, while wind speed and direction did not make a difference to camera vignetting during the experiments. Utilizing a single image of a customized calibration target, we were able to mitigate vignetting of outdoor images captured in a 30 min duration by approximately 70% to 80% in terms of the intra-image pixel standard deviation (IISD) and 75% in terms of the pixel-wise mean (PWMN) range. The results indicated that outdoor environmental conditions such as air temperature and wind speed during short UAV flights might only minimally influence the thermal camera vignetting severity and pattern. Nonetheless, frequent external shutter-based corrections and considering the camera nonlinear temperature response in future studies have the potential to further improve vignetting correction efficacy for large scene temperature ranges.

Pixel-wise radiometric calibration approach for infrared focal plane arrays using multivariate polynomial correction

Advances in infrared focal plane arrays allow thermal cameras to operate without active cooling equipment, however, the outputs of uncooled thermal cameras are closely related to their own temperatures, which degrades the temperature retrieval accuracy when ambient environment varies. In this paper, a novel radiometric calibration approach is proposed to decrease the errors derived from ambient temperature drift and internal heating. Firstly, a multivariate polynomial correction plus a second-order polynomial correction are implemented to stabilize the camera output derived from temporal non-uniformity. Afterwards, multi-point correction is applied to remove the fixed pattern noise caused by spatial non-uniformity. Finally, object temperature is calculated from thermal image gray values using the Planck-like approximation function. Experimental results demonstrated that the proposed method outperforms existing shutterless and shutter-based methods especially when uncooled thermal cameras are in unstable status.

Development of a Compact and Robust Mid-Infrared Spectrometer by Using a Silicon/Air Hyperspectral Filter

The molecular absorption spectrum in the mid-infrared (mid-IR) region is typically measured by Fourier-transform infrared spectroscopy (FTIR). Such spectrometers require complex and delicately aligned interferometers, which increases their size and cost. Here, we present an alternative compact mid-IR spectrometer, which uses a variable mid-IR filter with a gradual change of cavity length between two silicon/air dielectric mirrors. When combined with a modern uncooled thermal imaging camera, hyperspectral filtering can provide a powerful solution for the miniaturization of mid-IR spectrometers. By using the hyperspectral filter, light from a broadband light source can be dispersed and assigned to individual pixels of a microbolometer array of the thermal imaging camera. This technology offers an inexpensive and compact mid-IR spectrometer design with no moving parts and rapid acquisition time.

A novel method for sensitivity modelling of optical gas imaging thermal cameras with warm filters

ABSTRACT The work proposes a novel method for sensitivity modelling of uncooled thermal cameras for optical gas imaging purposes. Such cameras use warm interference filters for better gas leak contrast at the cost of decreased sensitivity. With the presented method, it is possible to estimate this sensitivity without the need for physically installing a filter inside the camera. It can be done for any chosen background temperature and an arbitrary filter with known spectral transmission characteristic, which is often found in the filter manufacturer’s documentation. The proposed method requires prior measurement of the camera calibration curve before filter installation. In addition, this method may be used for estimating, how camera noise equivalent temperature difference will change after filter installation. With the aid of new parameter gas equivalent blackbody digital level difference, one may also verify, whether in particular measurement scenario gas leak will be visible or not. The performance of the proposed method is validated with five different filters and broadband uncooled thermal imaging camera.

USING THERMAL AND RGB UAV IMAGERY TO MEASURE SURFACE FLOW VELOCITIES OF RIVERS

Abstract. This study assesses the suitability to use RGB and thermal infrared imagery acquired from an UAV to measure surface flow velocities of rivers. The reach of a medium-scale river in Hungary is investigated. Image sequences with a frame rate of 2 Hz were captured with two sensors, a RGB and an uncooled thermal camera, at a flying height that ensures the visibility of both shores. The interior geometry of both cameras were calibrated with an in-house designed target field. The image sequences were automatically co-registered to account for UAV movements during the image acquisition. The TIR data was processed to keep loss-free image information solely in the water area and to enhance the signal to noise ratio. Image velocimetry with PIV applied to the TIR data and PTV applied to the RGB data was utilised to retrieve surface flow velocities. Comparison between RGB and TIR data reveal an average deviation of about 0.01 m/s. Future studies are needed to evaluate the transferability to other non-regulated river reaches.

Pixel-Based Calibration and Atmospheric Correction of a UAS-Mounted Thermal Camera for Land Surface Temperature Measurements

HighlightsA novel pixel-based calibration algorithm and an atmospheric correction method are developed.Application of the calibration methods reduces the RMSE of measurements to less than 1.32°C.The calibrations facilitate stitching of images together to form whole-field mosaics.Abstract. Thermal imagery can be used to provide insight into the water stress status and evapotranspiration demand of crops, but satellite-based sensors are generally too coarse spatially and too infrequent temporally to provide information of use for the management of specific fields. Thermal cameras mounted on small unmanned aerial systems (UAS) have potential to provide canopy temperature information at high spatial and temporal resolutions useful for crop management; however, without appropriate camera corrections, the measurement biases of these uncooled thermal cameras can be larger than ±5°C. Such uncertainty can render such camera measurements useless. In this research, a pixel-based (non-uniformity) calibration algorithm and an atmospheric correction method based on in-field approximate blackbody sources (water targets) were developed for a thermal camera. The objective was to improve the temperature measurement accuracy of the thermal camera on various land surfaces including soil and vegetation. With sufficient accuracy, temperature measurements can be used for the estimation of latent heat flux of field crops in the future. The thermal camera was first calibrated in a laboratory setting where the camera and environmental conditions were controlled. The results indicated that in the range between 10°C and 45°C, the calibrated temperatures were accurate, with an average bias of 1.76°C, and had a high linear correlation with reference temperatures (water target temperatures) (R2 > 0.99). Variability of measurements was also better constrained. In-field atmospheric correction is also important for obtaining high-accuracy thermal imagery. By applying both pixel-based calibration and atmospheric corrections, the RMSE (root mean square error) of validation targets from two dates in 2017 was reduced from 4.56°C and 6.36°C before calibration to 1.32°C and 1.24°C after calibration. The calibration process also increased the range of temperatures in the imagery, which enhanced contrast and may help with identification of tie-points and stitching of images together to form whole-field mosaics. Keywords: Atmospheric correction, Pixel-based calibration, Thermal remote sensing, UAS, Water targets.

Performance estimation of the state-of-the-art convolution neural networks for thermal images-based gender classification system

Gender classification has found many useful applications in the broader domain of computer vision systems including in-cabin driver monitoring systems, human–computer interaction, video surveillance systems, crowd monitoring, data collection systems for the retail sector, and psychological analysis. In previous studies, researchers have established a gender classification system using visible spectrum images of the human face. However, there are many factors affecting the performance of these systems including illumination conditions, shadow, occlusions, and time of day. Our study is focused on evaluating the use of thermal imaging to overcome these challenges by providing a reliable means of gender classification. As thermal images lack some of the facial definition of other imaging modalities, a range of state-of-the-art deep neural networks are trained to perform the classification task. For our study, the Tufts University thermal facial image dataset was used for training. This features thermal facial images from more than 100 subjects gathered in multiple poses and multiple modalities and provided a good gender balance to support the classification task. These facial samples of both male and female subjects are used to fine-tune a number of selected state-of-the-art convolution neural networks (CNN) using transfer learning. The robustness of these networks is evaluated through cross validation on the Carl thermal dataset along with an additional set of test samples acquired in a controlled lab environment using prototype uncooled thermal cameras. Finally, a new CNN architecture, optimized for the gender classification task, GENNet, is designed and evaluated with the pretrained networks.

Evapotranspiration Estimation with Small UAVs in Precision Agriculture.

Estimating evapotranspiration (ET) has been one of the most critical research areas in agriculture because of water scarcity, the growing population, and climate change. The accurate estimation and mapping of ET are necessary for crop water management. Traditionally, researchers use water balance, soil moisture, weighing lysimeters, or an energy balance approach, such as Bowen ratio or eddy covariance towers to estimate ET. However, these ET methods are point-specific or area-weighted measurements and cannot be extended to a large scale. With the advent of satellite technology, remote sensing images became able to provide spatially distributed measurements. However, the spatial resolution of multispectral satellite images is in the range of meters, tens of meters, or hundreds of meters, which is often not enough for crops with clumped canopy structures, such as trees and vines. Unmanned aerial vehicles (UAVs) can mitigate these spatial and temporal limitations. Lightweight cameras and sensors can be mounted on the UAVs and take high-resolution images. Unlike satellite imagery, the spatial resolution of the UAV images can be at the centimeter-level. UAVs can also fly on-demand, which provides high temporal imagery. In this study, the authors examined different UAV-based approaches of ET estimation at first. Models and algorithms, such as mapping evapotranspiration at high resolution with internalized calibration (METRIC), the two-source energy balance (TSEB) model, and machine learning (ML) are analyzed and discussed herein. Second, challenges and opportunities for UAVs in ET estimation are also discussed, such as uncooled thermal camera calibration, UAV image collection, and image processing. Then, the authors share views on ET estimation with UAVs for future research and draw conclusive remarks.

In situ calibration of an uncooled thermal camera for the accurate quantification of flower and stem surface temperatures

This paper presents an in situ calibration method for an uncooled thermal camera to reduce bias and root mean square error (RMSE) for the observed surface temperatures of Gerbera jamesonii plants. Surface temperature bias and RMSE were quantified for the calibrated camera constants and compared with respect to the default constant statistics. The averaged calibrated camera constant bias and RMSE values decreased by at least 89.1% relative to the averaged default camera constant bias and RMSE values for individual plant stems and flowers. This calibration approach has the potential to be suitable for other uncooled thermal cameras in fields beyond horticultural applications.

A Calibration Procedure for Field and UAV-Based Uncooled Thermal Infrared Instruments.

Thermal infrared cameras provide unique information on surface temperature that can benefit a range of environmental, industrial and agricultural applications. However, the use of uncooled thermal cameras for field and unmanned aerial vehicle (UAV) based data collection is often hampered by vignette effects, sensor drift, ambient temperature influences and measurement bias. Here, we develop and apply an ambient temperature-dependent radiometric calibration function that is evaluated against three thermal infrared sensors (Apogee SI-11(Apogee Electronics, Santa Monica, CA, USA), FLIR A655sc (FLIR Systems, Wilsonville, OR, USA), TeAx 640 (TeAx Technology, Wilnsdorf, Germany)). Upon calibration, all systems demonstrated significant improvement in measured surface temperatures when compared against a temperature modulated black body target. The laboratory calibration process used a series of calibrated resistance temperature detectors to measure the temperature of a black body at different ambient temperatures to derive calibration equations for the thermal data acquired by the three sensors. As a point-collecting device, the Apogee sensor was corrected for sensor bias and ambient temperature influences. For the 2D thermal cameras, each pixel was calibrated independently, with results showing that measurement bias and vignette effects were greatly reduced for the FLIR A655sc (from a root mean squared error (RMSE) of 6.219 to 0.815 degrees Celsius (℃)) and TeAx 640 (from an RMSE of 3.438 to 1.013 ℃) cameras. This relatively straightforward approach for the radiometric calibration of infrared thermal sensors can enable more accurate surface temperature retrievals to support field and UAV-based data collection efforts.

Passive Autofocusing System for a Thermal Camera

Most of the Thermal (Infrared) cameras nowadays are equipped with a motorized lens for focusing a scene manually. The subjective nature of manual focusing makes it an inefficient and cumbersome process. In contrast, Autofocusing (AF) obtains the best focused image based on a quantitative measure with the benefits of convenience and intelligence. Various AF systems for visual cameras have been developed, but relatively less amount of work has been done for thermal imaging systems. This paper presents a Vision and Control based Autofocusing System (VCAFS) comprising: (1) an uncooled thermal camera with motorized lens, (2) a passive contrast-based focus measure, (3) a smoothing operator to avoid local extrema, and (4) two different lens motion controllers. Experimental results show the efficacy of the proposed system on live videos even when the scene and its depth are continuously changing.

Measurement of land surface temperature from oblique angle airborne thermal camera observations

ABSTRACTThis paper presents a novel airborne remote sensing method using thermal imaging to directly georeference and calculate Earth surface temperature with a high spatiotemporal resolution. A tethered balloon is used to elevate an uncooled thermal camera in the field. When deployed, images with oblique view angles of the surrounding Earth surface are collected. Images recorded from a field environmental monitoring campaign in a northern Canadian mining facility are processed with open source software, and it is shown that they successfully represent the diurnal and spatial surface temperature variations within the facility. Furthermore, in comparison to MODerate resolution Imaging Spectroradiometer (MODIS) satellite images, the approach results in a median absolute error of 0.64 K, with a bias and Root Mean Square Error (RMSE) of 0.5 K and 5.45 K, respectively.

Optical Gas Imaging With Uncooled Thermal Imaging Camera - Impact of Warm Filters and Elevated Background Temperature

The goal of the presented article is to investigate the impact of warm filters and elevated background temperature on the operation of an uncooled wideband camera used for optical gas imaging (OGI). The noise equivalent temperature difference (NETD) parameter is measured for a VOx wideband OGI camera built for the purpose of the experiments, with and without an interference filter. The results demonstrate that warm filters have a clear effect on the performance of the OGI camera. Based on the standard NETD measurement procedures, a camera-scene noise equivalent temperature difference (CSNETD) parameter is proposed to quantify both the camera and scene noise. The CSNETD measurements are used to evaluate the impact of elevated background temperature on the performance of the OGI camera. Finally, the camera is used to image methane leaks in the mid-wave infrared (MWIR) and long-wave infrared (LWIR) bands. The results are consistent with the presented NETD/CSNETD measurements and confirm the impact of warm filters and elevated background temperature on the operation of the uncooled thermal imaging camera.

Evaluating Image Tracking Approaches for Surface Velocimetry With Thermal Tracers

AbstractIn this paper an automatic approach is proposed to measure flow velocity with an uncooled thermal camera. Hot water is used as thermal tracer. The introduced tracking algorithm utilizes the pyramidal Lucas‐Kanade method and is especially suitable for thermal image data. The performance of the new tool is compared to traditional image‐based tracking tools, that is, PIVlab and PTVlab. Experiments are performed in the laboratory for three different flow velocities and tests are conducted in a small stream to illustrate the suitability of the tool for field measurements. Results of the laboratory experiments as well as of the field experiments show that our tracking algorithm, applied to imagery from a thermal camera, outperforms commonly used tracking methods. Our tool provides velocity fields with very high resolution and is in close agreement with reference measurements, whereas PTVlab and PIVlab tend to overestimate and underestimate flow velocities, respectively.

(Invited) Hermetic Wafer-Level Packaging of Microbolometers for Uncooled Thermal Cameras

Infrared (IR) and thermal imaging industry has been increasingly expanding outside of its traditional military scope to cover a variety of consumer products, surveillance, and security applications. The market for thermal imaging is expected to double in the next 5 years and reach an amount of $8 billion by 2021. This increasing demand for thermal imaging technology is expected to reduce the unit price per device, which puts pressure on the industry to lower production costs. For low-cost consumer products, thermal imaging is based on microbolometer technology which does not require cooling, works in short-range, and provides live-video feed of surroundings. Microbolometers comprise of IR sensitive material that undergoes a change in electrical resistance upon absorbing IR radiation. In order to minimize losses, improve the resolution of the thermal imager, and extend its lifetime, the material is thermally isolated from its surrounding environment by enclosing it in a low vacuum micro-cavity. The micro-cavity is formed by bonding a cap wafer to the device wafer, which allows encapsulating tens of devices at the same time. This form of bonding, known as wafer-level packaging, significantly cuts the microbolometer production cost. When used for consumer products, the thermal imaging device must operate for 5-10 years, depending on the targeted application, before its performance deteriorates. To prolong the lifetime of the device, the vacuum in micro-cavities must be sustained. Several steps must be implemented in order to achieve and maintain proper operation conditions: i) a hermetic sealing of the cavity should be realized to minimize gas leakage from surrounding environment; ii) degassing of the system before seal-off is needed to dislodge and release trapped gases within the materials; and if necessary, iii) incorporating a non-evaporable getter which acts as a chemical pump to rid the system of residual gases during the device lifetime. In this presentation, we provide an overview of current challenges in hermetic wafer-level packaging of microbolometers, which is a crucial step in the large scale production of uncooled thermal cameras. The presentation will address advantages and limitations of different bonding processes. Special focus will be made on fluxless AuSn solder and its use to achieve hermetic wafer-level packaging. Results of temperature-dependent surface analysis studies including in situ and ex situ spectroscopic and microscopic techniques (LEEM/PEEM, XPS, SEM-NanoAuger, and TEM) will be discussed to highlight key mechanisms in wafer preprocessing (e.g. patterning of the seal rings) and establish key parameters controlling the quality of the bonding. The role of pre-processing at elevated temperatures in degassing the cavity from non-getterable gases and its influence on Sn and Ti surface diffusion will be discussed. This surface diffusion yields to the formation surface oxide, which is detrimental to the bonding process.

THERMAL TEXTURE SELECTION AND CORRECTION FOR BUILDING FACADE INSPECTION BASED ON THERMAL RADIANT CHARACTERISTICS

Abstract. An automatic building façade thermal texture mapping approach, using uncooled thermal camera data, is proposed in this paper. First, a shutter-less radiometric thermal camera calibration method is implemented to remove the large offset deviations caused by changing ambient environment. Then, a 3D façade model is generated from a RGB image sequence using structure-from-motion (SfM) techniques. Subsequently, for each triangle in the 3D model, the optimal texture is selected by taking into consideration local image scale, object incident angle, image viewing angle as well as occlusions. Afterwards, the selected textures can be further corrected using thermal radiant characteristics. Finally, the Gauss filter outperforms the voted texture strategy at the seams smoothing and thus for instance helping to reduce the false alarm rate in façade thermal leakages detection. Our approach is evaluated on a building row façade located at Dresden, Germany.