Non-contact Heart Rate Estimation Research Articles

A robust non-contact heart rate estimation from facial video based on a non-parametric signal extraction model

Remote photoplethysmography (rPPG) is a non-contact method to extract heart rate (HR) pulse signals from facial videos. However, challenges persist due to head motion disturbances and ambient illumination variations. To address these issues, a novel non-parametric signal extraction model is proposed for HR estimation using the webcam camera. We first select the better-quality RGB signal from the divided face regions through the ranking of signal-to-noise ratio (SNR). Then, we use blind source separation and skin reflection modeling to eliminate the effect of ambient illumination. Finally, we employ a non-parametric signal extraction model named circulant singular spectrum analysis (CiSSA), with frequency-based decomposition and energy-driven reconstruction, to extract the HR-related pulse signal effectively. To evaluate the performance of our method, we conducted the validation experiments on three datasets, including two public datasets (UBFC and PURE dataset) and one self-collected dataset (ILVA dataset). Our method outperformed significantly on all three datasets, achieving Mean Absolute Deviation (MAD) of 2.42, 1.36, and 2.84 bpm, Root Mean Square Error (RMSE) of 1.89, 2.24, and 3.58 bpm, and Pearson Correlation Coefficients (PCC) of 0.98, 0.98, and 0.93, respectively. The experimental results demonstrate that our method achieves comparable performance, particularly in scenarios involving severe head motion disturbances and variations in ambient illumination.

MAE-Based Self-Supervised Pretraining Algorithm for Heart Rate Estimation of Radar Signals.

Noncontact heart rate monitoring techniques based on millimeter-wave radar have advantages in unique medical scenarios. However, the accuracy of the existing noncontact heart rate estimation methods is still limited by interference, such as DC offsets, respiratory harmonics, and environmental noise. Additionally, these methods still require longer observation times. Most deep learning methods related to heart rate estimation still need to collect more heart rate marker data for training. To address the above problems, this paper introduces a radar signal-based heart rate estimation network named the "masked phase autoencoders with a vision transformer network" (MVN). This network is grounded on masked autoencoders (MAEs) for self-supervised pretraining and a vision transformer (ViT) for transfer learning. During the phase preprocessing stage, phase differencing and interpolation smoothing are performed on the input phase signal. In the self-supervised pretraining step, masked self-supervised training is performed on the phase signal using the MAE network. In the transfer learning stage, the encoder segment of the MAE network is integrated with the ViT network to enable transfer learning using labeled heart rate data. The innovative MVN offers a dual advantage-it not only reduces the cost associated with heart rate data acquisition but also adeptly addresses the issue of respiratory harmonic interference, which is an improvement over conventional signal processing methods. The experimental results show that the process in this paper improves the accuracy of heart rate estimation while reducing the requisite observation time.

Non-Contact Heart Rate Measurement Based on Adaptive Notch Filter and Elimination of Respiration Harmonics

Continuous-wave Doppler radar is an established method for non-contact heart rate estimation. The output signal of the Doppler radar includes the fundamental wave of a heartbeat, heartbeat harmonics, the fundamental wave of respiration, respiration harmonics, and artifacts due to body movement. The respiration harmonics and artifacts degrade the accuracy of heart rate estimation. To this end, a novel heart rate estimation method is proposed that uses adaptive notch filters (ANFs). First, the proposed system estimates respiration frequency. Next, cascaded notch filters, the notch frequencies of which are controlled by the estimated respiration frequency, eliminate respiration harmonics; thus, the accuracy of heart rate estimation improves. In addition, the proposed method uses an ANF based on all-pass filter and an adaptive algorithm to estimate heart rate. The ANF is robust against disturbances, allowing the proposed method to improve the estimation accuracy in the presence of moving artifacts. The simulation results for human subjects show that MAPE, MAE, MSE, and RMSE of the proposed method are 5.24%, 4.00, 28.38, and 5.26, respectively; thus, they can be reduced up to fourth-order respiration harmonics. In an experiment on a dog not restrained in a cage, the proposed method, handling up to the fifth-order respiration harmonic, yields MAPE of 7.89%, MAE of 5.34, MSE of 96.11, and RMSE of 9.80. Thus, the proposed method shows better estimation accuracy than any other conventional method.

Non-contact heart rate estimation based on singular spectrum component reconstruction using low-rank matrix and autocorrelation.

The remote photoplethysmography (rPPG) based on cameras, a technology for extracting pulse wave from videos, has been proved to be an effective heart rate (HR) monitoring method and has great potential in many fields; such as health monitoring. However, the change of facial color intensity caused by cardiovascular activities is weak. Environmental illumination changes and subjects' facial movements will produce irregular noise in rPPG signals, resulting in distortion of heart rate pulse signals and affecting the accuracy of heart rate measurement. Given the irregular noises such as motion artifacts and illumination changes in rPPG signals, this paper proposed a new method named LA-SSA. It combines low-rank sparse matrix decomposition and autocorrelation function with singular spectrum analysis (SSA). The low-rank sparse matrix decomposition is employed to globally optimize the components of the rPPG signal obtained by SSA, and some irregular noise is removed. Then, the autocorrelation function is used to optimize the global optimization results locally. The periodic components related to the heartbeat signal are selected, and the denoised rPPG signal is obtained by weighted reconstruction with a singular value ratio. The experiment using UBFC-RPPG and PURE database is performed to assess the performance of the method proposed in this paper. The average absolute error was 1.37 bpm, the 95% confidence interval was -7.56 bpm to 6.45 bpm, and the Pearson correlation coefficient was 98%, superior to most existing video-based heart rate extraction methods. Experimental results show that the proposed method can estimate HR effectively.

RPPG Signal Measurement Based on Time-do-main Attention Network

The measurement of heart rate without touching the object being measured is referred to as “non-contact heart rate estimation.” Telemedicine, education, and a variety of other fields can all benefit from this approach, which is capable of achieving continuous heart rate measurement under certain conditions. The video segmentation network and the backbone network are cascaded in this paper’s cascaded heart rate estimation model. Adding temporal attention based on the existing 3D convolutional network model, the video segmentation network divides the video into multiple segments and fuses the relevant temporal features. It then sends the fusion results to the backbone network with the temporal attention mechanism. Mechanisms and modules like spatiotemporal convolution to make heart rate measurement more reliable and use a custom joint loss function to cut out other kinds of periodic signals’ influence. The trial results show the way that the proposed calculation can successfully lessen movement ancient rarity and light change clamor.Additionally, the dataset UBFC-rPPG demonstrates the model’s efficacy.

Combining Chrominance Features and Fast ICA for Noncontact Imaging Photoplethysmography

Video-based noncontact detection of heart rate has a wide range of applications in the field of medicine and health. However, this method is susceptible to noise interference, making it difficult to effectively extract blood volume pulse (BVP) signals. To overcome this problem, a new method of noncontact heart rate estimation that can suppress noise interference is proposed in this paper. First, the established data acquisition system conducts video collection, and the captured videos are divided into multiple small regions. Subsequently, the initial signals of BVP are extracted in accordance with the chrominance features extracted through multi-channel data fusion. The BVP signals are separated using the FastICA algorithm. The kurtosis value and signal-to-noise ratios of the power spectrum of the separated signals are analyzed to determine the effective separation component. Results show that this method can extract and process pulse signals, effectively suppressing non-periodic interference. The experiment also proves that the method has good consistency with the measurement of pulse oximeter and has good stability and accuracy in the detection of heart rate of the human body.

Remote Sensing for Vital Information Based on Spectral-Domain Harmonic Signatures

In this paper, we propose an efficient and robust non-contact heart rate estimation algorithm. The key idea is to recover the fundamental heartbeat frequency from its higher-order spectral features. Our contributions are threefold: 1) carefully review and examine possible approaches in vital signs detection with ultra-wide band (UWB) impulse radar; 2) numerically and experimentally show an important result that, in the spectral domain, the fundamental heartbeat signal is respiration-interference-limited, whereas its higher-order harmonics are noise-limited; and 3) implement an adaptive heart rate monitoring algorithm based on the proposed theory, which is feasible with continuous monitoring. To justify the proposed theory, we perform a spectral analysis of the harmonics of vital signs signal. We validate the proposed algorithm using a controlled vital sign simulator. We experimentally demonstrate the effectiveness of the harmonics-based heart rate estimation algorithm and compare it against existing methods. For completeness, we also provide a limitation analysis of the proposed algorithm.

MUSIC-based Non-contact Heart Rate Estimation with Adaptive Window Size Setting.

Continuous HR (Heart Rate) monitoring enables the stress estimation in daily life. A Doppler sensor could be a key device to facilitate the non-contact HR estimation. As one of the Doppler sensor-based HR estimation methods, we have previously proposed a MUSIC (MUltiple SIgnal Classification)-based HR estimation method. MUSIC is the algorithm widely used as a tool to estimate DOA (Direction of Arrival). In our previous method, MUSIC spectrum is calculated in each sliding window, and then HR is estimated by the maximum peak detection over the MUSIC spectrum. However, when HR changes largely within the window, several peaks due to heartbeats appear over the MUSIC spectrum, which might cause the incorrect peak detection. Hence, an adaptive window is required so that only one peak appears. In this paper, we propose a MUSIC-based HR estimation method with an adaptive window size setting. When several peaks due to heartbeats appear over the MUSIC spectrum, our proposed method shortens the time window and re-calculates the MUSIC spectrum, which is repeated until only one peak appears. The experimental results showed that our method outperformed not only our previous one but also the other existing MUSIC-based HR estimation one in terms of the estimation accuracy of the HR, the stress indexes CVI (Cardiac Vagal Index) and CSI (Cardiac Sympathetic Index).

A New Approach for Noncontact Imaging Photoplethysmography Using Chrominance Features and Low-Rank in the IoT Operating Room

Advancements in image-processing and medical imaging technologies and other research areas have resulted in the growing importance of surgical navigation systems in the field of minimally invasive surgery. However, sophisticated auxiliary equipment is a prerequisite for full heart-rate monitoring in operative systems. A system puts forward to address this issue with a wireless sensor network architecture comprising acquisition modules, router gateways, remote servers, and a medical monitoring center. Factors such as variations in ambient light and face shaking can also easily affect heart rate detection based on face videos, thus resulting in inaccurate estimations of heart rate from the blood volume pulse (BVP) signals. This study proposes to address this concern by employing a novel method for non-contact heart rate estimation to overcome noise interference. First, chrominance features are selected to extract BVP signals, and the low-rank and sparse matrix decomposition methods are applied to overcome the detrimental effects of noise and interference while ensuring that valuable data are preserved.Next, the data are relayed to the server via the gateway. Finally, users can log on to the health-related cloud platform and gain information regarding their health status in real time. Experimental results reveal the advantages of the proposed technique over conventional face-based heart-rate estimation methods, including the capability to decrease dependence on sophisticated auxiliary equipment, avoid direct skin contact that can cause discomfort to patients undergoing surgery, and improve the comfort of surgical operations. Moreover, the proposed heart-rate measurement technique can contribute to improve the construction of smart cities.

Self-adaptive signal separation for non-contact heart rate estimation from facial video in realistic environments

Objective: Recent research indicates that facial epidermis color varies with the rhythm of heat beats. It can be captured by consumer-level cameras and, astonishingly, be adopted to estimate heart rate (HR). The HR estimated remains not as precise as required in a practical environment where illumination interference, facial expressions, or motion artifacts are involved, although numerous methods have been proposed in the last few years. A novel algorithm is proposed to make non-contact HR estimation technique more robust. Approach: First, the face of the subject is detected and tracked to follow the head movement. The facial region then falls into several blocks, and the chrominance feature of each block is extracted to establish a raw HR sub-signal. Self-adaptive signal separation is performed to separate the noiseless HR sub-signals from raw sub-signals. On that basis, the noiseless sub-signals full of HR information are selected using a weight-based scheme to establish the holistic HR signal, from which the average HR is computed adopting wavelet transform and data filtering. Main results: Forty subjects took part in our experiments, whose facial videos were recorded by a normal webcam with the frame rate of 30 fps under ambient lighting conditions. The average HR estimated by our method correlates strongly with ground truth measurements, as indicated in experimental results measured in a static scenario with the Pearson correlation r = 0.980 and a dynamic scenario with the Pearson correlation r = 0.897. In addition, our method, compared to the newest method, decreases the error rate by 38.63% and increases the Pearson correlation by 15.59%. Significance: This work proposes a robust method for non-contact HR measurement in a realistic environment. Results of comparative experiments indicate that our method out-performs state-of-the-art non-contact HR estimation methods in realistic environments.

Bounded Kalman filter method for motion-robust, non-contact heart rate estimation.

The authors of this work present a real-time measurement of heart rate across different lighting conditions and motion categories. This is an advancement over existing remote Photo Plethysmography (rPPG) methods that require a static, controlled environment for heart rate detection, making them impractical for real-world scenarios wherein a patient may be in motion, or remotely connected to a healthcare provider through telehealth technologies. The algorithm aims to minimize motion artifacts such as blurring and noise due to head movements (uniform, random) by employing i) a blur identification and denoising algorithm for each frame and ii) a bounded Kalman filter technique for motion estimation and feature tracking. A case study is presented that demonstrates the feasibility of the algorithm in non-contact estimation of the pulse rate of subjects performing everyday head and body movements. The method in this paper outperforms state of the art rPPG methods in heart rate detection, as revealed by the benchmarked results.

Real-Time Robust Noncontact Heart Rate Monitoring With a Camera

Noncontact heart rate monitoring that enables the detection of cardiac activity cycles with a normal camera without cutaneous contact has broad application prospects. However, the state-of-the-art approaches lack robustness to the lower illuminance and the varying illuminance. This paper builds an optical noncontact heart rate monitoring framework (lower/vary illumiance PPG, LiPPG) with the aim of improving the robustness of these illuminance situations. In LiPPG, we use the YIQ color space to represent the PPG signal. By applying the Savitzky–Golay filter, we propose a mathematical model, named, multiscale variable-weight Savitsky–Golay combination, to extract signals in the time domain and reduce the disturbance from noise. We test the LiPPG on three scenarios: stationary, facial motion, and varying illuminance, with 18 video clips from 6 participants. The experimental results show that the noncontact heart rate estimation using the LiPPG reduced the errors compared with the gold-standard method of recording from a finger using a pulse oximeter and that it is superior to existing methods.

A Novel Non-contact Heart Rate Estimation Algorithm and System with User Identification

In these days, the wearable devices have been developed for measuring biological data effectively. However, wearable devices have tissue allege and noise problem. Also, it is impossible for a remote center to identify the person whose data are measured by wearable devices, which could trigger a communication problem over treatment. To solve these problems, biometric measurement based on a non-contact method, such as face image sequencing is necessary. This makes it possible to measure biometric data without any operation and side effects. This system can monitor the biological signals of people in real time without allege and noise and simultaneously identify them. In this paper, we propose an authentication process while measuring biometric data, through a non-contact method.

Robust Sensor Fusion of Unobtrusively Measured Heart Rate

Contactless vital sign measurement technologies often have the drawback of severe motion artifacts and periods in which no signal is available. However, using several identical or physically different sensors, redundancy can be used to decrease the error in noncontact heart rate estimation, while increasing the time period during which reliable data are available. In this paper, we show for the first time two major results in case of contactless heart rate measurements deduced from a capacitive ECG and optical pulse signals. First, an artifact detection is an essential preprocessing step to allow a reliable fusion. Second, the robust but computationally efficient median already provides good results; however, using a Bayesian approach, and a short time estimation of the variance, best results in terms of difference to reference heart rate and temporal coverage can be achieved. In this paper, six sensor signals were used and coverage increased from 0-90% to 80-94%, while the difference between the estimated heart rate and the gold standard was less than ±2 BPM.

Need for a Revision of the Normal Limits of Resting Heart Rate

The current definition of sinus tachycardia is a heart rate >100 beats per minute (bpm).1 This limit was set arbitrarily when heart rate was not yet regarded as a risk factor for cardiovascular disease, probably with the main purpose of distinguishing between a disease state (fever, thyrotoxicosis, anemia, congestive heart failure, etc) and a normal condition. In recent years, interest has been aroused by the awareness that fast heart rate is a potent precursor of hypertension, atherosclerosis, and their sequelae.2 3 4 5 6 7 8 9 10 In addition, many leading epidemiological studies have shown that tachycardia is associated with an increased risk of death from cardiovascular and noncardiovascular causes. This relationship has been found in general populations,3 4 5 6 7 in elderly individuals,9 and in hypertensive cohorts.10 In all of these studies, the heart rate value above that in which a significant increase in risk was observed was below the 100 bpm threshold (Table 1⇓). Only in the study by Levy et al3 was tachycardia defined as a heart rate >99 bpm, but in that study the cutoff between normal and high heart rate was chosen arbitrarily, and the highest heart rate value measured during the examination was taken to define the subject’s heart rate. In all the other studies, the threshold level between normal and fast heart rate was between 79 and 90 bpm. View this table: Table 1. Heart Rate Values Above Which a Significant Increase in Risk Was Found: Data From 9 Epidemiological Studies The normalcy limits of a clinical variable can be established according to different criteria. For many parameters, such as most biochemical indexes, the 95% confidence interval is calculated to identify the upper normal limit of the variable. This statistical approach does not appear suitable for those clinical variables …