Infrared Image Sensor Research Articles

Virtual IR Sensing for Planetary Rovers: Improved Terrain Classification and Thermal Inertia Estimation

Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.

Evaporated Sex Te1- x Thin Films with Tunable Bandgaps for Short-Wave Infrared Photodetectors.

Semiconducting absorbers in high-performance short-wave infrared (SWIR) photodetectors and imaging sensor arrays are dominated by single-crystalline germanium and III-V semiconductors. However, these materials require complex growth and device fabrication procedures. Here, thermally evaporated Sex Te1- x alloy thin films with tunable bandgaps for the fabrication of high-performance SWIR photodetectors are reported. From absorption measurements, it is shown that the bandgaps of Sex Te1- x films can be tuned continuously from 0.31 eV (Te) to 1.87 eV (Se). Owing to their tunable bandgaps, the peak responsivity position and photoresponse edge of Sex Te1- x film-based photoconductors can be tuned in the SWIR regime. By using an optical cavity substrate consisting of Au/Al2 O3 to enhance its absorption near the bandgap edge, the Se0.32 Te0.68 film (an optical bandgap of ≈0.8 eV)-based photoconductor exhibits a cut-off wavelength at ≈1.7 μm and gives a responsivity of 1.5 AW-1 and implied detectivity of 6.5 × 1010 cm Hz1/2 W-1 at 1.55 μm at room temperature. Importantly, the nature of the thermal evaporation process enables the fabrication of Se0.32 Te0.68 -based 42 × 42 focal plane arrays with good pixel uniformity, demonstrating the potential of this unique material system used for infrared imaging sensor systems.

Wearable infra-red pre-screening smartbra for early detection of breast cancer

Breast cancer is the most common cancer in women worldwide, comprising 16% of all female cancers and early diagnosis remains an important detection strategy. The aim of this study was to design and implement a user-friendly SmartBra. An infra-red (IR) imaging sensor was deployed to determine the temperature profile of the breast for this application. The device was tested with approval from the Health Research Ethics Committee of the Lagos University Teaching Hospital using healthy persons and persons already diagnosed with different stages of breast cancer. The results showed high sensitivity and specificity with good intra-examiner reliability for absolute values of mean temperature for the right breast and very good reproducibility for the left breast. Data for healthy participants revealed that the difference in absolute temperature between the left and right breast was less than 1oC, while that of the sick (cancer) participants indicated values greater than 1oC. The device is safe and easy to use and therefore can serve as an adjunct diagnostic device for early detection of breast cancer.
 Keywords: breast cancer, early detection, infra-red thermography, mammography.

Landsat 8-Based Surface Temperature Anomaly and Hydrocarbon Prospectivity: A Study in Part of Assam–Arakan Fold Belt, North East India

Subsurface hydrocarbon reservoirs act as effective thermal barriers to the Earth’s heat flow from the interior to the surface. As a result, a positive thermal anomaly below a hydrocarbon reservoir and a negative thermal anomaly on the surface above the reservoir are observed. The use of remote sensing satellite images is a rapid, cost-effective and accurate method of determining land surface temperature of a region. The present study uses recent Landsat 8 operational land imager-thermal infrared sensor images to detect land surface temperature distribution in a part of the Assam–Arakan Fold Belt, North East India, using a single-channel algorithm. Two anomalous negative surface temperature zones to the south of the study area are found to be important. High-resolution Landsat 8 panchromatic image, surface geological map, NDVI map and SRTM data rule out the effects of artefacts, urban settlements, and variations in lithology, vegetation or topography on these anomalous zones. The superimposition of the surface temperature map over the previously determined hydrocarbon prospect map reveals that these negative surface temperature anomalies lie over two significant hydrocarbon prospect zones. Thus, the effect of subsurface petroleum reservoirs is evident on the land surface temperature distribution of the area. Therefore, satellite image-based land surface temperature mapping can be used as an aid in detecting potential target areas for hydrocarbon exploration in the entire basin.

Up-Conversion Sensing of 2D Spatially-Modulated Infrared Information-Carrying Beams with Si-Based Cameras.

Up-conversion sensing based on optical heterodyning of an IR (infrared) image with a local oscillator laser wave in a nonlinear optical sum-frequency mixing (SFM) process is a practical solution to circumvent some limitations of IR image sensors in terms of signal-to-noise ratio, speed, resolution, or cooling needs in some demanding applications. In this way, the spectral content of an IR image can become spectrally shifted to the visible/near infrared (VIS/NWIR) and then detected with silicon focal plane arrayed sensors (Si-FPA), such as CCD/CMOS (charge-coupled and complementary metal-oxide-semiconductor devices). This work is an extension of a previous study where we recently introduced this technique in the context of optical communications, in particular in FSOC (free-space optical communications). Herein, we present an image up-conversion system based on a 1064 nm Nd3+: YVO4 solid-state laser with a KTP (potassium titanyl phosphate) nonlinear crystal located intra-cavity where a laser beam at 1550 nm 2D spatially-modulated with a binary Quick Response (QR) code is mixed, giving an up-converted code image at 631 nm that is detected with an Si-based camera. The underlying technology allows for the extension of other IR spectral allocations, construction of compact receivers at low cost, and provides a natural way for increased protection against eavesdropping.

Mechanical and Electronic Video Stabilization Strategy of Mortars with Trajectory Correction Fuze Based on Infrared Image Sensor.

For a higher attack accuracy of projectiles, a novel mechanical and electronic video stabilization strategy is proposed for trajectory correction fuze. In this design, the complexity of sensors and actuators were reduced. To cope with complex combat environments, an infrared image sensor was used to provide video output. Following the introduction of the fuze’s workflow, the limitation of sensors for mechanical video stabilization on fuze was proposed. Particularly, the parameters of the infrared image sensor that strapdown with fuze were calculated. Then, the transformation relation between the projectile’s motion and the shaky video was investigated so that the electronic video stabilization method could be determined. Correspondingly, a novel method of dividing sub-blocks by adaptive global gray threshold was proposed for the image pre-processing. In addition, the gray projection algorithm was used to estimate the global motion vector by calculating the correlation between the curves of the adjacent frames. An example simulation and experiment were implemented to verify the effectiveness of this strategy. The results illustrated that the proposed algorithm significantly reduced the computational cost without affecting the accuracy of the motion estimation. This research provides theoretical and experimental basis for the intelligent application of sensor systems on fuze.

Analytical study on the relationship between land surface temperature and land use/land cover indices

ABSTRACT The present study focuses on the estimating land surface temperature (LST) of Raipur City in India emphasizing the urban heat island (UHI) and non-UHI inside the city boundary and their relationship with normalized difference vegetation index, normalized difference water index, normalized difference built-up index, and normalized multi-band drought index. The entire study has been performed on four multi-date Landsat 8 Operational Land Imager and Thermal Infrared Sensor images taken from four different seasons; pre-monsoon, monsoon, post-monsoon, and winter in the same year. The UHI has mainly been developed along the northern and southern periphery of the city. The range of LST in the common UHI for four different seasons varies between 25.72° C and 35.69° C. The results present the strongest correlation between LST and the land use/land cover indices in monsoon and post-monsoon images while winter and pre-monsoon images show a comparatively weak correlation.

An Atmospheric Correction Using High Resolution Numerical Weather Prediction Models for Satellite-Borne Single-Channel Mid-Wavelength and Thermal Infrared Imaging Sensors

This paper presents a single-channel atmospheric correction method for remotely sensed infrared (wavelength of 3–15 μm) images with various observation angles. The method is based on basic radiative transfer equations with a simple absorption-focused regression model to calculate the optical thickness of each atmospheric layer. By employing a simple regression model and re-organization of atmospheric profiles by considering viewing geometry, the proposed method conducts atmospheric correction at every pixel of a numerical weather prediction model in a single step calculation. The Visible Infrared Imaging Radiometer Suite (VIIRS) imaging channel (375 m) I4 (3.55~3.93 μm) and I5 (10.50~12.40 μm) bands were used as mid-wavelength and thermal infrared images to demonstrate the effectiveness of the proposed single-channel atmospheric correction method. The estimated sea surface temperatures (SSTs) obtained by the proposed method with high resolution numerical weather prediction models were compared with sea-truth temperature data from ocean buoys, multichannel-based SST products from VIIRS/MODIS, and results from MODerate resolution atmospheric TRANsmission 5 (MODTRAN 5), for validation. High resolution (1.5 km and 12 km) numerical weather prediction (NWP) models distributed by the Korea Meteorological Administration (KMA) were employed as input atmospheric data. Nighttime SST estimations with the I4 band showed a root mean squared error (RMSE) of 0.95 °C, similar to that of the VIIRS product (RMSE: 0.92 °C) and lower than that of the MODIS product (RMSE: 1.74 °C), while estimations with the I5 band showed an RMSE of 1.81 °C. RMSEs from MODTRAN simulations were similar (within 0.2 °C) to those of the proposed method (I4: 0.81 °C, I5: 1.67 °C). These results demonstrated the competitive performance of a regression-based method using high-resolution numerical weather prediction (NWP) models for atmospheric correction of single-channel infrared imaging sensors.

Cloud Detection Based on High Resolution Stereo Pairs of the Geostationary Meteosat Images

Due to the considerable impact of clouds on the energy balance in the atmosphere and on the earth surface, they are of great importance for various applications in meteorology or remote sensing. An important aspect of the cloud research studies is the detection of cloudy pixels from the processing of satellite images. In this research, we investigated a stereographic method on a new set of Meteosat images, namely the combination of the high resolution visible (HRV) channel of the Meteosat-8 Indian Ocean Data Coverage (IODC) as a stereo pair with the HRV channel of the Meteosat Second Generation (MSG) Meteosat-10 image at 0° E. In addition, an approach based on the outputs from stereo analysis was proposed to detect cloudy pixels. This approach is introduced with a 2D-scatterplot based on the parallax value and the minimum intersection distance. The mentioned scatterplot was applied to determine/detect cloudy pixels in various image subsets with different amounts of cloud cover. Apart from the general advantage of the applied stereography method, which only depends on geometric relationships, the cloud detection results are also improved because: (1) The stereo pair is the HRV bands of the Spinning Enhanced Visible and InfraRed Imager (SEVIRI) sensor, with the highest spatial resolution available from the Meteosat geostationary platform; and (2) the time difference between the image pairs is nearly 5 s, which improves the matching results and also decreases the effect of cloud movements. In order to prove this improvement, the results of this stereo-based approach were compared with three different reflectance-based target detection techniques, including the adaptive coherent estimator (ACE), constrained energy minimization (CEM), and matched filter (MF). The comparison of the receiver operating characteristics (ROC) detection curves and the area under these curves (AUC) showed better detection results with the proposed method. The AUC value was 0.79, 0.90, 0.90, and 0.93 respectively for ACE, CEM, MF, and the proposed stereo-based detection approach. The results of this research shall enable a more realistic modelling of down-welling solar irradiance in the future.

An automatic computer vision pipeline for the in-line monitoring of freeze-drying processes

Infrared imaging sensors were proposed in the past years for the real-time monitoring and control of the Vacuum Freeze Drying (VFD) process. In order to extract reliable, real-time, and quantitative features from images collected by these sensors, a robust and automated image processing pipeline is required. Two major issues that have to be addressed concern the object detection and segmentation step, which locates and segments the objects whose temperature needs to be measured, and the tracking step, which is about following the movement of objects of interest to correlate information across subsequent images. Traditional intensity-based image analysis techniques are not reliable in this specific application, since the temperature, which is the intensity of a thermal picture, varies with time, whereas deep learning-based techniques are more robust to such changes.In this work, an object detector network based on a Faster Region Convolutional Neural Network (Faster R-CNN), and a Kernelized Correlation Filter (KCF) tracker were combined to monitor the VFD process of products contained in glass vials, which is a common scenario in the pharmaceutical domain. The object detector was trained on nine experimental acquisitions (7981 images) and tested on nine more (7201 images). Data augmentation methods consisting in the generation of synthetic sequences were used to effectively increase the performance while reducing the cost of data acquisition and annotation. The proposed technique achieved a recall and precision equal to 99.6 %, with execution time compatible with a real-time application. The localization of the vials was precise enough to measure the mean temperature with an acceptable error.

A Thermalized Bias Circuit for Wider Operating Temperature Range in Bolometric Infrared Imagers

This letter presents a device for biasing bolometric infrared imaging sensors to increase the operating temperature range and enhance performance. It addresses the market's need for uncooled readout circuits that are easier to use. Here, we propose an integrated biasing system capable of managing the bolometric sensor's high-temperature dependence. We demonstrated that a biasing architecture using thermalized bolometers can automatically offset resistance variations in the active bolometer due to temperature variations of the focal plane array (FPA). Our research concerns a 12-μm pixel pitch QVGA matrix that operates at 50-mK noise equivalent temperature difference (NETD).

Downscaling Land Surface Temperature Using Multiscale Geographically Weighted Regression Over Heterogeneous Landscapes in Wuhan, China

The deficiency of fine-resolution satellite-derived land surface temperature (LST) has impeded the meticulous exploration of the urban thermal environment at micro spatial scales. Although the LST downscaling methods are well-documented, the scale dependence between LST and environmental factors has been rarely considered in the regression establishment. This article proposes a new method (MGWRK) coupling multiscale geographically weighted regression (MGWR) and area-to-point kriging (ATPK) to generate fine-resolution LST maps over heterogeneous landscapes. Technically, normalized difference vegetation index (NDVI), normalized difference built-up index (NDBI), and albedo are selected using random forests (RF) as scale factors. First, the spatial and temporal robustness of MGWRK is evaluated by downscaling aggregated Landsat-8 Thermal Infrared Sensor images (1000-m) into finer resolutions of 500, 400, 300, 200, and 100-m in different seasons (spring, summer, and winter). Results reveal that MGWRK is superior to GWR and DisTrad methods for spatial details improvement (SSIM > 0.93) and LST fidelity ( ${\boldsymbol{R}^2}$ > 0.91) under all scales and in the seasons. Second, MGWRK is utilized to downscale the synchronous 1000-m Moderate Resolution Imaging Spectroradiometer (MODIS) LST map into 100-m. The quantitative and qualitative result substantiates that MGWRK obtains ideal performance when applied on MODIS data with few LST distortion ( ${\boldsymbol{R}^2}\; = \;0.915$ , RMSE = 1.607 K) and obvious spatial information enrichment (SSIM = 0.903).

Analysis of urban heat island intensity using multi temporal landsat data; case study of Kendari City, Indonesia

The urban air temperature has gradually increased in almost all cities in the world, including Kendari City. This is indicated by the increase in building materials and reduced vegetation biomass in urban areas that have consequences of increasing surface temperatures and forming a micro-climate phenomenon called urban heat island (UHI). The aim of this study is to analyse of UHI intensity in Kendari region for the periods 2001-2014-2019, based on the distribution of land surface temperature (LST), which was analysed through thermal infrared (TIRS) and operational land imager (OLI) sensors onboard Landsat-7 and Landsat-8, each image has an atmospheric correction and brightness temperature. The results show the intensity of UHI during the 2001 to 2014 period increased by 2.44 °C, while in 2019 the intensity decreased by 2.27 °C from the previous period. These fluctuations are closely related to the land cover (LC) changes especially in built-up areas, vegetation, and bare soil as the effects of the urbanization process, and parameters of the normalized difference vegetation index (NDVI).

Precise Inspection Method of Solar Photovoltaic Panel Using Optical and Thermal Infrared Sensor Image Taken by Drones

The inspection of the solar panel using the drone has already been put into practical use. However, this method requires an initial investment cost as compared with the conventional method, and it may take a cost such as image processing. In addition, there are challenges such as the ability to carry out the necessary maintenance of the law even if technically possible. Nowadays, the introduction of drones is proceeding in such a way as to check the points of abnormalities through simple flight. The advantage of this method is that safety enhancement and inspection speed are greatly improved compared to the case of performing the inspection by the conventional method. For companies, the use of drones and thermal infrared sensors for inspection can be used to reduce the population and to reduce the problems of recruitment, and to improve the efficiency and reduce the cost by simplifying the inspection procedure with fewer personnel will be. The purpose of this study is to develop a method to accurately detect abnormal points by making thermal infrared images of orthographic images by improving the inspection method through simple flight of drone. The study investigated whether it is possible to precisely check the point where the problem is located under the condition of insufficient solar radiation at the site installed by a small developer. We propose a method that can accurately distinguish between normal and abnormal panel points in images taken using drones and thermal infrared sensors, and how to accurately detect abnormal points by focusing on problem panels. Abnormal points were found to be very accurate and efficient to find and inspect panel more than the temperature range value of each panel.

Tunable dual-frequency cross polarization converters based on graphene metasurface in infrared range

A tunable dual-frequency metamaterial graphene polarization converter in the infrared range is put forward in this paper. The proposed converter is made up of 3 layers—the bottom gold layer, the middle silicon dioxide layer and the upper graphene layer. The simulated results show that the polarization converter can simultaneously work at two resonant frequencies of 36.82 and 41.38 THz. The polarization conversion rates are up to 0.9992 at 36.82 THz and 0.9969 at 41.38 THz under the certain normal condition when Fermi energy is given to 1.0 eV, showing the high efficiency of the polarization converter. Moreover, the converter shows its stable working performance under the change of the incident angle and the geometrical size. Meanwhile, without varying the nanostructure, the resonant frequencies can be flexibly adjusted by controlling the Fermi energy. The superior performance of the proposed graphene polarization converter indicates a broad prospect that a large number of applications such as infrared imaging sensors, modulators, switches, photonic and biochemical sensors, light harvesting, thermal detection and electromagnetic energy converter will apply it.

A Novel Infrared Image Enhancement Based on Correlation Measurement of Visible Image for Urban Traffic Surveillance Systems

Infrared imaging sensors are widely employed in urban traffic systems since they are not affected by lighting conditions. However, due to the limitation of hardware and imaging environment, it is difficult to obtain infrared (IR) images at the desired quality. IR images always lack detailed information, which leads to unsatisfying IR image enhancement results with the conventional method. Compared with the IR images, the visible (VIS) images contain detailed information, which could help to enhance the quality of the corresponding IR images. In this article, we propose an effective method to enhance IR images by applying the multi-sensors image. First, we adopt the edge-preserving filter to decompose the IR and VIS images into illumination and reflectance components according to Retinex theory. Second, each region in the IR and VIS image is classified into the related region and non-related region according to the correlation between IR and VIS images. Finally, an adaptive fuzzy plateau HE (AFPHE) is utilized to enhance the illumination component, and a strategy is employed to enhance the detail of the IR reflectance component with the help of VIS images. Experimental results demonstrate that the proposed method can effectively improve the contrast and enhance the detail of the IR images.

Influence of passivation layer on thermal stability of Nb:TiO2−x samples for shutter-less infrared image sensors

Test devices of unpassivated, single-layer passivated, and double-layer passivated samples were fabricated to analyze the enhancement of thermal stability for shutter-less infrared (IR) image sensors. In this study, double-layer passivation (SiOx/SiNx) was proposed to avoid the oxygen-diffusion from unpassivated or single-layer (SiOx) passivated Nb:TiO2−x (TNO) samples. Before fabricating the test device, high-temperature (450 °C) annealing was carried out under oxygen-rich ambient conditions to improve the thermal stability by reducing the number of oxygen vacancies in the as-deposited TNO samples. The double-layer passivated sample showed higher thermal stability than the other samples. The structural and bolometric properties were also studied for all passivated samples and the obtained results attested that the properties seemed to be almost similar for different passivation layers. It was found that the proposed double-layer passivated TNO sample could provide enhanced thermal stability for future shutter-less IR sensors.

Discriminative Fusion Correlation Learning for Visible and Infrared Tracking

Discriminative correlation filter- (DCF-) based trackers are computationally efficient and achieve excellent tracking in challenging applications. However, most of them suffer low accuracy and robustness due to the lack of diversity information extracted from a single type of spectral image (visible spectrum). Fusion of visible and infrared imaging sensors, one of the typical multisensor cooperation, provides complementarily useful features and consistently helps recognize the target from the background efficiently in visual tracking. Therefore, this paper proposes a discriminative fusion correlation learning model to improve DCF-based tracking performance by efficiently combining multiple features from visible and infrared images. Fusion learning filters are extracted via late fusion with early estimation, in which the performances of the filters are weighted to improve the flexibility of fusion. Moreover, the proposed discriminative filter selection model considers the surrounding background information in order to increase the discriminability of the template filters so as to improve model learning. Extensive experiments showed that the proposed method achieves superior performances in challenging visible and infrared tracking tasks.

Averaging Pixel Current Adjustment Technique for Reducing Fixed Pattern Noise in the Bolometer-Type Uncooled Infrared Image Sensor.

In this paper, we propose an averaging pixel current adjustment technique for reducing fixed pattern noise (FPN) in the bolometer-type uncooled infrared image sensor. The averaging pixel current adjustment technique is composed of active pixel, reference pixel, and calibration circuit. Polysilicon resistors were used in each active pixel and reference pixel. Resistance deviation among active pixels integrated with the same resistance value cause FPN. The principle of the averaging pixel current adjustment technique for removing FPN is based on the subtraction of dark current of the active pixel from the dark current of the reference pixel. The subtracted current is converted into the voltage, which contains pixel calibration information. The calibration circuit is used to adjust the calibration current. After calibration, the nano-ampere current is output with small deviation. The proposed averaging pixel current adjustment technique is implemented by a chip composed of a pixel array, a calibration circuit, average current generators, and readout circuits. The chip was fabricated using a standard 0.35 μm CMOS process and its performance was evaluated.

An Analog-Front ROIC with On-Chip Non-Uniformity Compensation for Diode-Based Infrared Image Sensors.

This paper proposes a CMOS front-end readout-integrated circuit (ROIC) with on-chip non-uniformity compensation technique for a diode-based uncooled infrared image sensor. Two techniques are adopted to achieve on-chip non-uniformity compensation: a reference dummy metal line is introduced to alleviate the dominant non-uniformity with IR-drop presented in large pixel array, and a current splitting architecture-based variable current source for diode bias is proposed to compensate other residual non-uniformity. A differential integrator is chosen as the main amplifier of readout circuit for its superior noise performance. For low power design, a pulse-powered row buffer is designed in this work. The proposed ROIC for 384 × 288 diode-based detector array is fabricated with a 0.35-m CMOS process. It occupies an area of 4.4 mm × 15 mm, and the power consumption is 180 mW. The measured result shows that with the proposed on-chip non-uniformity compensation, the output voltage variation is greatly reduced from 2.5 V to 60 mV.