Thermal Pixel Research Articles

Salient Features for Moving Object Detection in Adverse Weather Conditions During Night Time

Foreground segmentation of moving objects in adverse atmospheric conditions such as fog, rain, low light, and dust is a challenging task in computer vision. The advantages of thermal infrared imaging at night time under adverse atmospheric conditions have been demonstrated, which are due to the long wavelength. However, the existing state-of-the-art object detection techniques have not been useful in such scenarios. In this paper, we propose an improved background model that utilizes both thermal pixel intensity features and spatial video salient features. The proposed spatial video salient features are represented as an Akin-based per-pixel Boolean string over a local region block, and depend on the effect of neighboring pixels on a center pixel. The result of this Boolean procedure is referred to as the- Akin-Based Local Whitening Boolean Pattern (ALWBP), which differentiates foreground and background region accurately, even against a cluttered background. The background model is controlled via 1) the automatic adaptation of parameters such as the decision threshold $\text{R}_{\mathrm {T}}$ and, learning parameter L, and 2) the updating of background samples $\text{B}_{\mathrm {sample\_{}int}}$ and,- $\text{B}_{\mathrm {sample\_{}ALWBP}}$ to minimize 1) the effect of the background dynamics of outdoor scenes and 2) the temperature polarity changes during the maiden appearance of a moving object in thermal frame sequences. The performance of this model is evaluated using nine existing standard segmentation performance metrics on our newly created- Tripura University Video Dataset at Night Time (TU-VDN) and on the publicly available CDnet-2014 dataset. Our newly created weather-degraded video dataset, namely, TU-VDN, consists of sixty video sequences that represent four atmospheric conditions, namely, low light, dust, rain, and fog. The results of a performance comparison with 14 state-of-the-art detection techniques also demonstrate the high accuracy of the proposed technique.

Enhancing the retrieval of stream surface temperature from Landsat data

Thermal images of water bodies often show a radiance gradient perpendicular to the banks. This effect is frequently due to mixed land and water thermal pixels. In the case of the Landsat images, radiance mixing can also affect pure water pixels due the cubic convolution resampling of the native thermal measurements. Some authors recommended a general-purpose margin of two thermal pixels to the banks or a minimum river width of three pixels, to avoid near bank effects in water temperature retrievals. Given the relatively course spatial resolution of satellite thermal sensors, the three pixel margin severely restricts their application to temperature mapping in many rivers. This study proposes a new algorithm to enhance the retrieval of stream surface temperature using Landsat 8 thermal data, although it is also applicable to Landsat 7 and Landsat 5. The aim is not to perform a subpixel radiance unmixing but to refine the selection of unmixed, reliable pixels for temperature mapping. For this purpose, the spatial arrangement of native Landsat thermal pixels is approximated, and pure water pixels in the downscaled thermal band are selected accordingly. The least-favourable cubic convolution near-bank radiance mixing is simulated on image basis. Only pure thermal water pixels unaffected by the simulated worst-case resampling are selected. The algorithm allowed retrieving water surface temperature in reaches down to 120 m wide, clearly improving the existing three pixel, i.e. 300 m for Landsat 8, recommendation. The enhancing algorithm was applied to a reach in the Ebro River reach, Spain. It provided spatially distributed temperatures in narrow parts, upstream and downstream of a wide reservoir, offering new insight of the overall impact of the reservoir over the river thermal regime.

Digital thermal metasurface with arbitrary infrared thermogram

An object exhibits the infrared thermogram of another object, which is called thermal illusion as extensively investigated in the field of thermal metamaterials. However, almost all the existing thermal illusion behaviors were theoretically designed by using unconventional thermal conductivities, which means that the conductivities must be anisotropic, graded, or even singular due to the analytical methods in use. This problem largely limits fabrications for applications. By suggesting two discretization steps, here we put forward a numerical method instead to design thermal illusion, for which unconventional conductivities are no longer needed. In the meantime, more importantly, we reveal different thermal illusion behaviors. By tailoring the joint effects of thermal conduction and convection, we design a thermal pixel of cuboidal shape. We show that the assembly of such pixels into different two-dimensional arrays could generate infrared thermograms of different objects, which is thus called the digital thermal metasurface. Also, the metasurface is reconfigurable, and it can apparently produce all the existing thermal illusion behaviors reported in the literature. Finally, we experimentally fabricate a prototype. This work opens a door for applying conventional thermal conductivities of commercially available materials to thermal illusion, and we expect it to stimulate more exciting developments in electromagnetic disguise and confrontation.

Performance evaluation strategy for cool roof based on pixel dependent variable in multiple spatial regressions

Performance evaluation for cool roof has been mainly based on field sampling, which relies on attributes of an area at one point in time, reflecting an emphasis on the small number of in situ data. One of the major disadvantages of traditional field monitoring is that it is costly, laborious and time consuming due to the large number of samples required. Acknowledging these constraints, this research was inspired by the fact that pixel of remotely-sensed imagery could play a key role in presenting rooftop temperature remarkably due to thermal sensitivity channel. The aim of this research is primarily intended to show how to evaluate energy saving performance of color changing roof, based on pixel of remotely-sensed imagery, together with multiple spatial regressions. An empirical study for a case study site was suggested to demonstrate how a standard remote sensing and multiple spatial regression approach can be used to assist in monitoring performance of cool roof based on thermal pixel in terms of various specific settings such as season, sunset, sunrise timing, solar facing and sitting orientation etc. It is expected that the pixel-based analysis would make it possible to identify area-wide patterns of cool roof performance subject to many different type of artificial structures, which cannot be acquired by traditional field sampling. This research will be used as a valuable reference in implementing performance evaluation for color roofing since the quantitative data generated from remote sensing and multiple spatial regressions can present area-wide visual evidences by permanent record.

Forest fire risk modeling in Uttarakhand Himalaya using TERRA satellite datasets

Forest fire is one of the major causes of degradation in western Himalaya, and is an annual phenomenon in more than 50% in the forests of Uttarakhand state. Fire danger models are useful for the fire managers to mitigate and suppress the fire activates. MODIS 8 day products viz. MODIS Terra Land surface reflectance (MOD09A1), MODIS Terra Land surface Temperature (MOD11A2) and ASTER digital Elevation Model (DEM) were used to develop fire danger model in this paper. Three parameters Modified Normalized Difference Fire Index (MNDFI), Perpendicular Moisture Index (PMI) and potential surface temperature were computed from the above mentioned satellite products. MNDFI has been used for determining the actual fire occurrence in thermal anomaly pixels and PMI has been used for the estimation of live fuel moisture content in the vegetation and litter. The Potential surface temperature was computed using the MODIS Land Surface Temperature and ASTER DEM. Spatial model was developed based on the above parameters and MODIS terra and Aqua thermal anomaly product (fire location) was used for the validation of the model in the study area. The fire danger models showed an accuracy of 87.31%, i.e. the model accurately predict the fire danger over the study area. Further analysis was done based on composite fire danger image and vegetation types; composite fire danger image and fragmentation map of the study area.

Debris thickness of glaciers in the Everest area (Nepal Himalaya) derived from satellite imagery using a nonlinear energy balance model

Abstract. Debris thickness is an important characteristic of debris-covered glaciers in the Everest region of the Himalayas. The debris thickness controls the melt rates of the glaciers, which has large implications for hydrologic models, the glaciers' response to climate change, and the development of glacial lakes. Despite its importance, there is little knowledge of how the debris thickness varies over these glaciers. This paper uses an energy balance model in conjunction with Landsat7 Enhanced Thematic Mapper Plus (ETM+) satellite imagery to derive thermal resistances, which are the debris thickness divided by the thermal conductivity. Model results are reported in terms of debris thickness using an effective thermal conductivity derived from field data. The developed model accounts for the nonlinear temperature gradient in the debris cover to derive reasonable debris thicknesses. Fieldwork performed on Imja–Lhotse Shar Glacier in September 2013 was used to compare to the modeled debris thicknesses. Results indicate that accounting for the nonlinear temperature gradient is crucial. Furthermore, correcting the incoming shortwave radiation term for the effects of topography and resampling to the resolution of the thermal band's pixel is imperative to deriving reasonable debris thicknesses. Since the topographic correction is important, the model will improve with the quality of the digital elevation model (DEM). The main limitation of this work is the poor resolution (60 m) of the satellite's thermal band. The derived debris thicknesses are reasonable at this resolution, but trends related to slope and aspect are unable to be modeled on a finer scale. Nonetheless, the study finds this model derives reasonable debris thicknesses on this scale and was applied to other debris-covered glaciers in the Everest region.

Mapping daily evapotranspiration at field scales over rainfed and irrigated agricultural areas using remote sensing data fusion

Continuous monitoring of daily evapotranspiration (ET) at field scale can be achieved by combining thermal infrared remote sensing data information from multiple satellite platforms, given that no single sensor currently exists today with the required spatiotemporal resolution. Here, an integrated approach to field-scale ET mapping is described, combining multi-scale surface energy balance evaluations and a data fusion methodology, namely the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), to optimally exploit spatiotemporal characteristics of image datasets collected by the Landsat and Moderate resolution Imaging Spectroradiometer (MODIS) sensors, as well as geostationary platforms. Performance of this methodology is evaluated over adjacent irrigated and rainfed fields, since mixed conditions are the most challenging for data fusion procedures, and in two different climatic regions: a semi-arid site in Bushland, TX and a temperate site in Mead, NE. Daytime-total ET estimates obtained for the Landsat overpass dates suggest that the intrinsic model accuracy is consistent across the different test sites (and on the order of 0.5mmd−1) when contemporaneous Landsat imagery at 30-m resolution is available. Comparisons between tower observations and daily ET datastreams, reconstructed between overpasses by fusing Landsat and MODIS estimates, provide a means for assessing the strengths and limitations of the fused product. At the Mead site, the model performed similarly for both irrigated and rainfed fields, with an accuracy of about 0.9mmd−1. This similarity in performance is likely due to the relatively large size of the fields (≈50ha), suggesting that the soil moisture dynamics of the irrigated fields are reasonably well captured at the 1-km MODIS thermal pixel scale. On the other hand, the accuracy of daily retrievals for irrigated fields at the Bushland site was lower than that for the rainfed field (errors of 1.5 and 1.0mmd−1, respectively), likely due to the inability of the model to capture ET spikes right after irrigation events for fields substantially smaller than MODIS data resolution. At this site, the irrigated fields were small (≈5ha) compared to the MODIS pixel size, and sparsely distributed over the landscape, so sporadic contributions to ET from soil evaporation due to irrigation were not captured by the 1-km MODIS ET retrievals. However, due the semiarid environment at Bushland, these irrigation-induced spikes in soil evaporation are not long-lived and these underestimations generally affect the irrigation dates only and they do not seem to influence negatively the estimates at the seasonal scale. ET data fusion is expected to perform better over agricultural areas where irrigation is more spatially continuous, resulting in moisture fluxes that are more uniform at the MODIS pixel scale. Overall, the model accurately reproduces the ET temporal dynamics for all the experimental sites, and is able to capture the main differences that were observed between irrigated and rainfed fields at both daily and seasonal time scales.

Correction to "Using thermal time and pixel purity for enhancing biophysical variable time series: An interproduct comparison" [Apr 13 2119-2127

There is an error in the above-named article [ibid.,vol. 51, no. 4, pp. 2119-2127, Apr. 2013] regarding the definition and the implementation of equation (3), defining the proposed temporal smoothing index (TSI). The correct formula is provided. These corrections do not change any of the general conclusions of the paper, but some of the comments regarding the interpretation of this table are revised.

Using Thermal Time and Pixel Purity for Enhancing Biophysical Variable Time Series: An Interproduct Comparison

This paper presents a multiannual comparison at regional scale of currently available 1-km global leaf area index (LAI) products with crop-specific green area index (GAI) retrieved from 250-m spatial resolution imagery from the Moderate Resolution Imaging Spectroradiometer (MODIS). The crop-specific GAI product benefits from the following extra processing steps: 1) spatial filtering of time series based on pixel purity; 2) transforming the time scale to thermal time; and 3) fitting a canopy structural dynamic model to smooth out the signal. In order to perform a rigorous comparison, these steps were also applied to the 1-km LAI products, namely, MODIS LAI (MCD15) and LAI produced in the CYCLOPES (Carbon cYcle and Change in Land Observational Products from an Ensemble of Satellites) project. A simple indicator was also designed to quantify the increase in temporal smoothness that can thus be obtained. The results confirm that, for winter wheat, the 250-m GAI product provides a more realistic description of the time course of the biophysical variable in terms of reaching higher values, grasping the variability, and providing smoother time series. However, the use of thermal time and pixel purity also improves the temporal consistency and coherence of the 1-km products. Overall, the results of this study suggest that these techniques could be valuable in harmonizing remote sensing data coming from different sources with varying spatial and temporal resolution for enhanced vegetation monitoring.

Accuracy increase of temperature measurement on thermal images

Multiresponse regression model is used for dependence of thermal image pixel amplitudes from temperature to increase accuracy of remote indirect method of temperature measurement. Obtained model allows to calculate integral estimation of temperature and its dispersion on all pixels taking into account correlation relationships of amplitude observation errors.

Fabrication of amorphous InGaZnO thin-film transistor-driven flexible thermal and pressure sensors

In this work, we present the results concerning the use of amorphous indium–gallium–zinc–oxide (a-IGZO) thin-film transistor (TFT) as a driving transistor of the flexible thermal and pressure sensors which are applicable to artificial skin systems. Although the a-IGZO TFT has been attracting much attention as a driving transistor of the next-generation flat panel displays, no study has been performed about the application of this new device to the driving transistor of the flexible sensors yet. The proposed thermal sensor pixel is composed of the series-connected a-IGZO TFT and ZnO-based thermistor fabricated on a polished metal foil, and the ZnO-based thermistor is replaced by the pressure sensitive rubber in the pressure sensor pixel. In both sensor pixels, the a-IGZO TFT acts as the driving transistor and the temperature/pressure-dependent resistance of the ZnO-based thermistor/pressure-sensitive rubber mainly determines the magnitude of the output currents. The fabricated a-IGZO TFT-driven flexible thermal sensor shows around a seven times increase in the output current as the temperature increases from 20 °C to 100 °C, and the a-IGZO TFT-driven flexible pressure sensors also exhibit high sensitivity under various pressure environments.

An iterative integrated framework for thermal–visible image registration, sensor fusion, and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal–visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal–visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.

A SOI-based CMOS-MEMS IR image sensor with partially released reference pixels

We have developed a 22 µm pitch and 320 × 240 pixel uncooled IR (infrared) image sensor. For IR detection, we utilized single crystal silicon series p–n junctions, which were fabricated on a SOI (silicon on insulator) wafer utilizing 8 inch CMOS technology and MEMS processes. The p–n junctions were passivated with buried and laminated oxide layers from wet crystalline etching of the silicon substrate. The oxide layers were also utilized to absorb the IR radiation and to form supporting beams. The partially released pixels were utilized as thermal black pixels (TBs) instead of optical black pixels (OBs) for correlated double sampling. The IR image sensor utilizing TBs obtained a thermal image of the human body stably without the smearing phenomenon.

The influence of topography on the forest surface temperature retrieved from Landsat TM, ETM + and ASTER thermal channels

The objective of this study was to assess the influence of topography on the surface temperature (ST) of spruce forest in the central part of the Šumava Mountains, Czech Republic. The aim was to design a method for surface temperature normalisation of forest stands in rugged relief. In this study, two Landsat scenes, ETM + and TM, and one TERRA ASTER scene were used. The different spatial (60 m–90 m–120 m) and/or radiometric (8–12 bits) resolutions of these scenes enabled the assessment of the influence of these parameters on the accuracy of surface temperature models at the mesoscale landscape context. These models are based on the effects of complex topography (digital elevation model — DEM, and illumination — Hillshade) on surface temperature. Only homogeneous spruce forest stands were used for surface temperature modeling. The influence of topography on surface temperature in spruce forest was confirmed in all types of satellite data used. Three different sampling approaches were used to increase the accuracy of the models. Predictability increased with forest content in the thermal pixel (sampling approach 1). The resulting R 2 values (0.47–0.49) were similar between all three scenes. Sampling approach 2 is based on the weighting of thermal pixels by the forest content ( R 2 : 0.32–0.39). We also assessed the influence of spruce forest edge effect on the accuracy of thermal models (sampling approach 3). Removing forest buffer zones resulted in greater statistical significance (approximately by 25%–40%). The optimal width of forest edge removed was determined to be 90 m. The resulting explained variability ( R 2 ) improved by forest edge removal was 0.57, 0.52, 0.47 in the case of ETM + , ASTER, and TM, respectively. These values correspond with the spatial resolution. However, the differences are not significant indicating that all these data are useful for ST modeling of spruce forest. The potential use of ST modeling is to identify temperature anomalies caused by different types of forest disturbance (e.g. harvesting, disease, insect attack) or changes in the condition of stands (e.g. water stress).

Modelling surface energy fluxes over maize using a two-source patch model and radiometric soil and canopy temperature observations

Models estimating surface energy fluxes over partial canopy cover with thermal remote sensing must account for significant differences between the radiometric temperatures and turbulent exchange rates associated with the soil and canopy components of the thermal pixel scene. Recent progress in separating soil and canopy temperatures from dual angle composite radiometric temperature measurements has encouraged the development of two-source (soil and canopy) approaches to estimating surface energy fluxes given observations of component soil and canopy temperatures. A Simplified Two-Source Energy Balance (STSEB) model has been developed using a “patch” treatment of the surface flux sources, which does not allow interaction between the soil and vegetation canopy components. A simple algorithm to predict the net radiation partitioning between the soil and vegetation is introduced as part of the STSEB patch modelling scheme. The feasibility of the STSEB approach under a full range in fractional vegetation cover conditions is explored using data collected over a maize (corn) crop in Beltsville Maryland, USA during the 2004 summer growing season. Measurements of soil and canopy component temperatures as well as the effective composite temperature were collected over the course of the growing season from crop emergence to cob development. Comparison with tower flux measurements yielded root-mean-square-difference values between 15 and 50 W m − 2 for the retrieval of the net radiation, soil, sensible and latent heat fluxes. A detailed sensitivity analysis of the STSEB approach to typical uncertainties in the required inputs was also conducted indicating greatest model sensitivity to soil and canopy temperature uncertainties with relative errors reaching ∼ 30% in latent heat flux estimates. With algorithms proposed to infer component temperatures from bi-angular satellite observations, the STSEB model has the capability of being applied operationally.

Generation of Electrically Induced Stimuli for MEMS Self-Test

A major task for the implementation of Built-In-Self-Test (BIST) strategies for MEMS is the generation of the test stimuli. These devices can work in different energy domains and are thus designed to sense signals which are generally not electrical. In this work, we describe, for different types of MEMS, how the required non-electrical test stimuli can be induced on-chip by means of electrical signals. This provides the basis for adding BIST strategies for MEMS parts embedded in the coming generation of integrated systems. The on-chip test signal generation is illustrated for the case of MEMS transducers which exploit such physical principles as time-varying electrostatic capacitance, piezo-resistivity effect and Seebeck effect. These principles are used in devices such as accelerometers, infrared imagers, pressure sensors or tactile sensors. For implementation, we have used two major MEMS technologies including CMOS-compatible bulk micromachining and surface micromachining. We illustrate the ability to generate on-chip test stimuli and to implement a self-test strategy for the case of a complete application. This corresponds to an infrared imager that can be used in multiple applications such as overheating detection, night vision, and earth tracking for satellite positioning. The imager consists of an array of thermal pixels that sense an infrared radiation. Each pixel is implemented as a suspended membrane that contains several thermopiles along the different support arms. The on-chip test signal generation proposed requires only slight modifications and allows a production test of the imager with a standard test equipment, without the need of special infrared sources and the associated optical equipment. The test function can also be activated off-line in the field for validation and maintenance purposes.

Mass flux measurements at active lava lakes: Implications for magma recycling

Remotely sensed and field data can be used to estimate heat and mass fluxes at active lava lakes. Here we use a three thermal component pixel model with three bands of Landsat thematic mapper (TM) data to constrain the thermal structure of, and flux from, active lava lakes. Our approach considers that a subpixel lake is surrounded by ground at ambient temperatures and that the surface of the lake is composed of crusted and/or molten material. We then use TM band 6 (10.42–12.42 μm) with bands 3 (0.63–0.69 μm) or 4 (0.76–0.90 μm) and 5 (1.55–1.75 μm) or 7 (2.08–2.35 μm), along with field data (e.g., lava lake area), to place limits on the size and temperature of each thermal component. Previous attempts to achieve this have used two bands of TM data with a two‐component thermal model. Using our model results with further field data (e.g., petrological data) for lava lakes at Erebus, Erta 'Ale, and Pu'u 'O'o, we calculate combined radiative and convective fluxes of 11–20, 14–27 and 368–373 MW, respectively. These yield mass fluxes, of 30–76, 44–104 and 1553–2079 kg s−1, respectively. We also identify a hot volcanic feature at Nyiragongo during 1987 from which a combined radiative and convective flux of 0.2–0.6 MW implies a mass flux of 1–2 kg s−1. We use our mass flux estimates to constrain circulation rates in each reservoir‐conduit‐lake system and consider four models whereby circulation results in intrusion within or beneath the volcano (leading to endogenous or cryptic growth) and/or magma mixing in the reservoir (leading to recycling). We suggest that the presence of lava lakes does not necessarily imply endogenous or cryptic growth: lava lakes could be symptomatic of magma recycling in supraliquidus reservoirs.

Pixel block intensity modulation: Adding spatial detail to TM band 6 thermal imagery

A pixel block intensity modulation (PRIM) method has been developed to add spatial detail to Landsat Thematic Mapper (TM) thermal band TM6 images in regions with sufficient topography. The method uses 30 m resolution TM reflective spectral band images (TM1-5 and 7) to modulate the relevant TM6 image on the basis of its 120m resolution thermal pixel blocks. Topographic detail in each 120m resolution pixel block of the TM6 image is thus recovered, without altering the average thermal digital number level of the block, by the spatial information recorded in the reflective spectral bands at 30m resolution. Tests confirm that the PBIM can effectively integrate topographic spatial detail from reflective spectral bands with TM6 images while retaining the fidelity of the original thermal spectral information. PBIM is also applicable, as a general method, for data fusion of multispectral and panchromatic images with different spatial resolutions. Bearing in mind that, for space-borne remote sensing, the spatial resolution of the thermal band will continue to be lower than that of VISSWIR and panchromatic bands in the multispectral sensor systems of the next generation, such as Enhanced Thematic Mapper Plus, the PBIM method will remain a useful technique for enhancing thermal imagery data for some time.

Test vehicle for a wafer-scale thermal pixel scene simulator

A chip sized test vehicle has been created to experiment with the technology and design required for two wafer scale thermal pixel displays: a thermal pixel dynamic scene simulator and a visual-to-thermal converter. The 4.7/spl times/4.7 mm device contains a 2/spl times/4 array of micromachined thermal pixels, optical detectors, A/D converters, digital pixel control circuitry, and laser links for interconnection. Laser produced defect avoidance schemes allowed testing and harvesting for global redundancy of the control circuitry and local transducer substitution. Also the laser linked signal buses flexibility was used to arrange the chip in different thermal pixel display configurations. Two chips were laser interconnected, one as fabricated and another where anisotropic etching had created a suspended plate holding the polysilicon resistor thermal pixels. From the electrical and laser interconnection points of operation there was no difference between the etched and unetched circuits. >

Integrated CMOS transducers for thermal scene generation

In this paper we present a silicon black-body emitter array chip fabricated using the standard 3 μm CMOS technology available through the Canadian Microelectronic Corporation and a maskless post-fabrication micromachining process developed at Simon Fraser University. The transducer chip comprises of a 2 × 2 array of silicon black-body emitters (thermal pixels) integrated with on-chip control circuitry. The paper will discuss the transducer design, micromachining-related design constraints on the CMOS layout design, the micromachining technique and the circuit performance.