Exploring the feasibility of seamless remote heart rate measurement using multiple synchronized cameras

Abstract

Heart rate (HR) measurement and monitoring is of great importance to determine the physiological and mental status of individuals. Recently, it has been demonstrated that HR can be remotely retrieved from facial video-based photoplethysmographic signals captured using consumer-grade cameras. However, in existing studies, subjects are mostly required to keep their facial regions of interest (ROIs) within one single camera. To make this technique usable in a daily life situation where subjects move around freely, we launch a preliminary simulated study of seamless remote HR measurement using multiple synchronized cameras by combining ensemble empirical mode decomposition (EEMD) with time-delay canonical correlation analysis (TDCCA), termed as EEMD-TDCCA. At each time point, a target ROI with the largest area is first determined from all the ROIs provided by all the cameras. Then, the RGB time sequence is formed by taking average of all pixels within each target ROI. Afterwards, the green channel time sequence is decomposed into several intrinsic mode functions (IMFs) and only the IMF candidates, whose frequency corresponding to the maximum amplitude falling into the interested HR range will be further processed by TDCCA. Finally, the first pair of the canonical variables having the largest correlation coefficient is the HR source and the corresponding HR is derived by peak detection or frequency analysis. Thirty subjects were recruited and four state-of-the-art methods were employed for comparison. The best performance was achieved by using the proposed EEMD-TDCCA followed by frequency analysis, with the mean absolute error 4.11 bpm, mean percentage error 5.26%, root mean square error 5.37 bpm, the Pearson’s correlation coefficient 0.90 and the intra-class correlation coefficient 0.89, demonstrating the feasibility of our proposed seamless remote HR measurement framework. This study will provide a promising solution for practical and robust non-contact HR measurement applications.