HDINet: Hierarchical Dual-Sensor Interaction Network for RGBT Tracking

Abstract

RGBT tracking is a practical solution that combines RGB and thermal infrared modes to solve tracking failures in complex environments to achieve all-day and all-weather work, which makes it gradually applied in multifarious fields. The fundamental reason is that it could avoid the damage of tracking performance caused by the limitation of the imaging characteristics of a single sensor. The existing work aggregates features in different ways, without considering hierarchical complementary interactions and the value of the initial input that may affect subsequent aggregation. In this paper, a novel hierarchical dual-sensor interaction network is proposed, which is mainly composed of dual-sensor interaction, sensor-specific and instance learning. Specifically, our network mainly benefits from the design of two modules, called feature interaction module and data encoding module. The dominant information of the dual sensor is extracted and supplemented by the former based on attention. The latter encodes the raw data into the initial input of the first feature interaction module, whose quality has a key influence on the follow-up. We investigate the performance through extensive experiments compared with the recent state-of-the-art RGB and RGBT trackers on the GTOT and RGBT234 datasets, which verify that our network is effective in both quantitative and qualitative evaluation.