Recognizing Activities from Egocentric Images with Appearance and Motion Features

Abstract

With the development of wearable cameras, recognizing activities from egocentric images has attracted the interest of many researchers. The motion of the camera wearer is an important cue for the activity recognition, and is either explicitly used by optical flow for videos or implicitly used by fusing several images for images. In this paper, based on the observation that the two consecutive images captured by the wearable camera contain the motion information of the camera wearer, we propose to use the camera wearer's rotation and translation computed from the two consecutive images as the motion features. The motion features are combined with appearance features extracted by a CNN as the activity features, and the activity is classified by a random decision forest. We test our method on two egocentric image datasets. The experimental results show that by adding the motion information, the accuracy of activity recognition has been significantly improved.