Domain transfer and difference-aware band weighting for object tracking in hyperspectral videos

Abstract

ABSTRACT Object tracking plays an important role in computer vision. In recent years, hyperspectral object tracking has gained increasing attention because the material information contained in a large number of spectral bands of hyperspectral images (HSIs), which is critical in distinguishing the target from the background. However, owing to the high-dimensional characteristics of HSIs and complex real-world scenarios, hyperspectral object tracking remains a challenging task. In this paper, we propose a domain transfer and difference-aware band weighting (DT-DBW) tracker for hyperspectral object tracking. Firstly, a domain transfer module is designed to adjust the feature distribution of HSIs, so that the deep learning object tracker can be effectively applied to hyperspectral videos. To further improve the performance and accuracy of the tracker, a difference-aware band weighting module is implemented to exploit the spectral difference features between the target and the background to generate individual band weights for the hyperspectral videos. Through the band weighting operation, the spectral response value of HSIs is recalibrated to enhance the value of spectral information and suppress the background spectral information. Experimental results on hyperspectral datasets demonstrate that the Area-Under-Curve (AUC) and tracking speed of DT-DBW tracker are up to 0.647 and 48.6 FPS, outperforming existing hyperspectral object trackers.