Static Image Action Recognition with Hallucinated Fine-Grained Motion Information

Abstract

Static image action recognition is a challenging task due to the lack of motion information in a static image. Some previous works have attempted to hallucinate the motion information in a static image using a generator learnt from freely available unlabeled videos. However, their hallucinated motion information is either low-level or coarse-grained, which may contain lots of noise or lose motion details. In contrast, we propose to hallucinate fine-grained high-level motion information, which is more robust and detail-preserving. Specifically, we hallucinate motion feature map which encodes the motion details of human body parts. We also hallucinate motion attention map to focus on motion-related regions. Our hallucinated motion information can greatly facilitate static image action recognition, which is confirmed by the experiments on two static action image datasets and two video datasets.