VFAT: A Personalized HAR Scheme Through Exploiting Virtual Feature Adaptation Based on Transfer Learning

Abstract

Recently, on one hand, human activity recognition (HAR) has witnessed great application on portable smart devices (e.g., smart phones and wearables, etc.) as they are widely used around the world. On the other hand, HAR methods based on deep learning have attracted much attention, for they possess excellent performance due to their strength on extracting virtual features automatically and hierarchically. However, to establish a personalized deep learning based HAR scheme based on smart devices, insufficient records from target users and heavy computation cost on training from scratch are two challenges. Considering that, in transfer learning, the knowledge learnt in the source domain could be appropriately transferred to help accomplish tasks in the target domain, this paper proposes a personalized HAR scheme through exploiting virtual feature adaptation based on transfer learning (i.e., VFAT) to achieve high recognition accuracy with low computation time. VFAT is composed of pre-training phase on sufficient labeled records in source-domain, and adaption phase on target-domain that uses the few labeled records available. Specifically, VFAT scheme pre-trains the LSTM-based feature extraction component in the pre-training phase and then introduces domain loss in the adaptation phase to minimize the similarity between target-domain virtual features and source-domain activity patterns (i.e., virtual features averaged by activity labels). The HAR scheme applied to the MotionSense dataset and results demonstrate the effectiveness of our proposed VFAT scheme. Moreover, we also investigate the impact of domain division on the performance of transfer learning based HAR.