Residual LSTM Attention Network for Object Tracking

Abstract

In this letter, we propose an attention network for object tracking. To construct the proposed attention network for sequential data, we combine long–short term memory (LSTM) and a residual framework into a residual LSTM (RLSTM). The LSTM, which learns temporal correlation, is used for a temporal learning of object tracking. In the proposed RLSTM method, the residual framework, which achieves the highest accuracy in ImageNet large scale visual recognition competition (ILSVRC) 2016, learns the variations of spatial inputs and thus achieves the spatio-temporal attention of the target object. Also, a rule-based RLSTM learning is used for robust attention. Experimental results on large tracking benchmark datasets object tracking benchmark (OTB)-2013, OTB-100, and OTB-50 show that the proposed RLSTM tracker achieves the highest performance among existing trackers including the Siamese trackers, attention trackers, and correlation trackers, and also has comparable performance with the state-of-the-art deep trackers.