Exploring neural motion transfer for unsupervised remote physiological measurement: A practicality study

Abstract

Remote Photoplethysmography (rPPG) is a method to measure cardiac activity without the need for any contact-based sensors, garnering attention due to its non-invasive and convenient nature. Addressing the issue of substantial missing data in large-scale datasets, current research is inclined towards unsupervised learning methods. However, prevailing unsupervised learning methods often prioritize inter-sample comparative learning while neglecting individual sample features, severely impacting performance and generalizability. Meanwhile, body movement is one of the most important influencing factors when attempting to extract the rPPG signal from videos. Hence, we embarked on the inaugural exploration of integrating Neural Motion Transfer into unsupervised learning methods for remote physiological measurements, aiming to enrich the inherent motion characteristics and diversity of samples. Employing Neural Motion Transfer, we synthesized videos encompassing diverse movements as a means of data augmentation. Through an in-depth comparison with other synthetic video methods, we extensively analyzed the physiological information within motion-enhanced synthetic videos. We investigated the applicability of these motion-enhanced synthetic videos to unsupervised learning methods and various associated impacts. We conducted extensive experiments on four benchmark datasets. After utilizing motion-enhanced synthetic videos to aid unsupervised training, our approach achieved an improvement of approximately 52% in intra-dataset testing and up to 97% in cross-dataset testing compared to the baseline unsupervised method, achieving the state-of-the-art (SOTA) performance among current unsupervised methods. Our research underscores the importance of neural motion networks in enhancing unsupervised learning methods and highlights the significant potential of synthetic videos.