LWIR hyperspectral image classification based on a temperature-emissivity residual network and conditional random field model

Abstract

ABSTRACT Long-wave infrared (LWIR) hyperspectral images can provide temperature and emissivity information, and have unique advantages in classification tasks, especially in the fields of minerals, vegetation, and man-made materials. However, LWIR image acquisition is greatly affected by the atmosphere, which results in obvious spectral noise. Meanwhile, field-based and aerial thermal infrared hyperspectral images are faced with problems such as spatial heterogeneity and salt-and-pepper noise, which results in the traditional spectral-based and object-oriented classification methods performing poorly in the classification task. In this paper, using the emissivity spectra and land surface temperature inversed from LWIR hyperspectral images, a classification framework based on a temperature-emissivity residual network and conditional random field model (TERN-CRF) is proposed. In this method, the residual network is designed to extract and fuse in-depth spectral and local spatial features, and the conditional random field (CRF) model incorporates the spatial-contextual information by the use of the temperature information, to improve the problem of voids and isolated regions in the classification map. To validate the performance of the proposed method, three LWIR datasets acquired by the Hyper-Cam sensor were used in the experiments: an aerial dataset and two field-based datasets of minerals and leaves. The experimental classification results obtained using the LWIR datasets confirm the performance of the proposed TERN-CRF classification method, which is shown to be superior to the traditional hyperspectral image classification methods and a CNN-based method. Furthermore, the CRF post-processing part can effectively alleviate the isolated regions in the classification map.