Sensor-Based Open-Set Human Activity Recognition Using Representation Learning With Mixup Triplets

Abstract

The main objective of sensor-based human activity recognition (HAR) is to classify predefined human physical activities with multi-channel signals acquired from wearable sensors. In a real-world scenario, signal data is changing over time and undefined activities may occur. Thus, an open-set classifier for HAR is required to detect it as an unknown class rather than to assign it to the one of the known classes. However, open-set HAR is a challenging task because of the small variability of inter-activities and the large variability of intra-activities. To tackle this problem, we propose mixup triplet learning for Mahalanobis distance (MTMD) for open-set recognition. In MTMD, we introduce a new representation learning with stochastically linearly interpolated triplets (mixup triplets) that makes discriminative features of the known from the unknown. In the trained embedding space with the proposed representation learning, thresholds based on Mahalanobis distances from the centers of each known activity are defined for the proposed open-set action recognition. To demonstrate the effectiveness of MTMD, we conducted experiments on sensor-based HAR benchmark datasets. The experimental results show that the proposed MTMD outperforms the existing methods for various open-set experimental settings in terms of numerical results of F1, recall, precision, and accuracy.