View-invariant action recognition via Unsupervised AttentioN Transfer (UANT)

Abstract

With wide applications in surveillance and human-robot interaction, view-invariant human action recognition is critical, however, challenging, due to the action occlusion and information loss caused by view change. Current methods mainly seek for a common feature space for different views. However, such solutions become invalid when there exist few common features, e.g. large view change. To tackle the problem, we propose an Unsupervised AttentioN Transfer (UANT) approach for view-invariant action recognition. Other than transferring feature knowledge, UANT transfers attention from one selected reference view to arbitrary views, which correctly emphasizes crucial body joints and their relations for view-invariant representation. In addition, the attention calculation method taking into account both recognition contribution and reliability of skeleton joints generates effective attention. Experiments showed its effectiveness for correctly locating crucial body joints in action sequences. We exhaustively evaluate our approach on the UESTC and the NTU dataset, performing unsupervised view-invariant evaluations, i.e. X-view and Arbitrary-view recognition. Experiment results demonstrate its superiority in view-invariant representation and recognition.