Exploration on the Negative Effects of Sensor Shifts in Photoplethysmography-Based Gesture Recognition and a Solution Based on Transfer Learning

Abstract

In order to promote the development of gesture interaction on wearable devices, this paper explores the negative effects of sensor shifts in photoplethysmography (PPG)-based gesture recognition technology and propose a solution based on transfer learning method. First, 10 batches of PPG data with sensor shifts are collected for 14 gestures and 10 participants. Then, the negative effects of sensor shifts is explored through experiments of feature visualization and gesture recognition based on single-batch data and multi-batch data. Experimental results show that the negative effect of sensor shifts can significantly change the feature distribution of gesture PPG signals, resulting in a sharp drop (59.24%) in gesture recognition accuracy. Finally, with the goal of reducing user training burden in the presence of sensor shifts, a long short-term memory (LSTM)-based transfer learning (TL) scheme is proposed and implemented. Compared to Non-TL strategy, TL strategy can improve the accuracy of gesture recognition to a certain degree, especially, the improvement effect is more significant when the amount of data involved in model calibration is small. Using the proposed TL scheme, only a small amount of data is needed to calibrate the classier in the practical application of PPG gesture interaction. This study lays a good foundation for the realization of PPG-based gesture interaction applications on wearable devices.