Human daily activity recognition with joints plus body features representation using Kinect sensor

Abstract

Human activity recognition has been studied actively from decades using a sequence of 2D images/video. With the development of depth sensors, new opportunities arise to improve and advance this field. This study presents a depth imaging activity recognition system to monitor and recognize daily activities of the human without attaching optical markers or motion sensors. In this paper, we proposed a new feature representation and extraction method using a sequence of depth silhouettes. Particularly, we first extract the depth silhouette by removing background from noisy effects and then extract the joints plus body features as skin color detection from joint information and multi-view body shape from depth silhouettes (i.e., front and side views). We combine the joints plus body shape features to make feature vector. These features have two nice properties including invariant with respect to body shape or size and insensitive to small noise. Self-Organized Map (SOM) is then used to train and test the feature vectors. Experimental results regarding our proposed human activity dataset and publically available dataset demonstrate that our feature extraction method is more promising and outperforms the state-of-the-art feature extraction methods.