Multiresolution Fusion Convolutional Network for Open Set Human Activity Recognition

Abstract

In recent years, sensor-based human activity recognition (HAR) technology has been the focus of extensive research and has been successfully applied to many aspects of people’s lives, but there are still some deficiencies. Most studies only distinguish daily activities and have low accuracy for easy confusing activities. In addition, many deep learning models only consider closed set HAR, but the real world contains unknown class activities that cannot be foreseen, which makes it challenging to apply these models to practice. In view of the above problems, this paper proposes a multi-resolution fusion convolution network (MRFC-Net) to cover the shortcoming that confusing activities are difficult to correctly identify, thus improving the accuracy of recognition. Furthermore, a multi-resolution fusion convolution variational auto-encoder network (MRFC-VAE-Net) for open set HAR is proposed. According to the reconstruction loss of the network, the corresponding threshold is set to effectively classify the known and unknown class activities in the open set. At the same time, a rich data set named daily-abnormal activity of special groups (DAASG) is constructed, which can be applied to the daily monitoring of special groups such as prisoners and the elderly. Experiments and analyses are carried out on the wireless sensor data mining (WISDM), physical activity monitoring for aging people (PAMAP2) and DAASG data sets, to prove the effectiveness and superiority of the proposed networks.