Human Complex Activity Recognition With Sensor Data Using Multiple Features

Abstract

In order to achieve better results of human complex activity recognition, the optimal feature representation of human complex activities from sensor data are studied. Features from multiple sets that are obtained by the learned networks, the manual extraction, and the location information are fused to generate mixed features to achieve better results of human complex activity recognition. A method extracting multi-layer features from the hybrid CNN and BLSTM network is proposed. The output feature vector of the second layer of the CNN and that of the BLSTM are combined to generate multi-layer features. Meanwhile, a new feature selection method based on SFS and network weight analysis is proposed. The method is to select a series of features extracted from the segmented sensor data using manual methods as dominant features. The location information of complex activities after one-hot encoding is also used as feature, which is then fused with multi-layer features and manual dominant features to generate mixed features. To further improve the recognition performance, the sensor data collected at different body positions are separated and used independently to train the hybrid CNN and BLSTM network to obtain their respective state information. The PAMAP2 and UT-Data datasets are used in our experiments to verify the proposed method. The results show that the method based on multiple features proposed in this paper dramatically outperforms the existing state-of-the-art methods for human complex activity recognition using sensor data.