RGB-Depth feature for 3D human activity recognition

Abstract

We study the problem of human activity recognition from RGB-Depth (RGBD) sensors when the skeletons are not available. The skeleton tracking in Kinect SDK works well when the human subject is facing the camera and there are no occlusions. In surveillance or nursing home monitoring scenarios, however, the camera is usually mounted higher than human subjects, and there may be occlusions. The interest-point based approach is widely used in RGB based activity recognition, it can be used in both RGB and depth channels. Whether we should extract interest points independently of each channel or extract interest points from only one of the channels is discussed in this paper. The goal of this paper is to compare the performances of different methods of extracting interest points. In addition, we have developed a depth map-based descriptor and built an RGBD dataset, called RGBD-SAR, for senior activity recognition. We show that the best performance is achieved when we extract interest points solely from RGB channels, and combine the RGB-based descriptors with the depth map-based descriptors. We also present a baseline performance of the RGBD-SAR dataset.