Feature mutual reinforcement learning and resampling for RGB-T tracking

Abstract

More and more researchers have paid attention to the tracking of visible-thermal infrared (RGB-T). How to fully exploit the complementary features of visible and thermal infrared images and fully integrate them is a key issue. After extracting image features, many researchers simply fuse the features by adding, connecting operations or designing fusion modules. However, these methods ignore the effects of different levels of fusion features on target modeling and specific feature extraction. In this work, we propose a RGB-T tracking network (MRLRNet) based on feature mutual reinforcement learning and resampling. Specifically, we design a feature mutual reinforcement learning module, which combines different layers of features to achieve progressive fusion. After each layer feature is extracted, the aggregation features are used to enhance specific modal features to achieve better specific feature representation and reduce noise and redundancy features. At the same time, we design a resampling module, which calculates the offset of two adjacent frames by phase correlation operation, and recalculates the Gaussian sample points to solve the problem of ground target loss caused by sudden camera movement. A large number of experiments on three RGB-T tracking datasets, GTOT, RGBT234 and LasHeR, demonstrate the effectiveness of this method.