Action Classification from Egocentric Videos Using Reinforcement Learning-based Pose Estimation

Abstract

We propose a novel action classification method using egocentric pose estimation based on reinforcement learning. In the field of egocentric video analysis, action classification and pose estimation have actively been studied. However, there have been few works that use pose estimation for action classification from egocentric videos. Since egocentric videos do not contain human pose information, it will be effective to introduce the estimated pose information into egocentric action classification methods. As the first step of this study, we train a pose estimation model using flow and humanoid information. Then an action classification model is trained by inputting the estimated pose sequence information into a Bidirectional Long Short-Term Memory network. Experimental results show that the proposed method is effective for the action classification from egocentric videos.