iPPG Signal Research Articles

A Study of Data Processing Methods for Non-Contact Multispectral Method of Blood Oxygen Saturation.

Regular monitoring of blood oxygenation is important for disease prevention and treatment. Image photoplethysmography (IPPG) technology is a non-contact physiological parameter detection technology, which has been widely used in blood oxygenation detection. However, traditional imaging devices still have issues such as low detection accuracy, narrower receiving spectral range. In this paper, we proposed two improved detection methods based on the dual-wavelength measurement principle, that is, dual-band IPPG signal ratio method and dual-band IPPG signal AC/DC method. To verify the effectiveness of the two methods, we used different heartbeat period IPPG signals as sample data sets, and combined PLS and RF algorithms for model training, thus obtaining the best data processing method. The experimental results showed that the dual-band IPPG signal AC/DC method can effectively reduce the model training time. This method meets the strong demand for non-contact blood oxygen measurement and provides a new measurement idea.

TO-LAB model: Real time Touchless Lung Abnormality detection model using USRP based machine learning algorithm.

Due to the increasing prevalence of respiratory diseases and the importance of early diagnosis. The need for non-invasive and touchless medical diagnostic solutions has become increasingly crucial in modern healthcare to detect lung abnormalities. Existing methods for lung abnormality detection often rely on invasive and time-consuming procedures limiting their effectiveness in real-time diagnosis. This work introduces a novel Touchless Lung Abnormality (TO-LAB) detection model utilizing universal software radio peripherals (USRP) and machine learning algorithms. The TO-LAB model integrates a blood pressure meter and an RGB-D depth-sensing camera to gather individual data without physical contact. Heart rate (HR) is analyzed through image conversion to IPPG signals, while blood pressure (BP) is obtained via analog conversion from the blood pressure meter. This touchless imaging setup facilitates the extraction of essential signal features crucial for respiratory pattern analysis. Advanced computer vision algorithms like Mel-frequency cepstral coefficients (MFCC) and Principal Component Analysis (PCA) process the acquired data to focus on breathing abnormalities. These features are then combined and inputted into a machine learning-based Multi-class SVM for breathing activity analysis. The Multi-class SVM categorizes breathing abnormalities as normal, shallow, or elevated based on the fused features. The efficiency of this TO-LAB model is evaluated with the simulated and real-time data. According to the findings, the proposed TO-LAB model attains the maximum accuracy of 96.15% for real time data; however, the accuracy increases to 99.54% for simulated data for the efficient classification of breathing abnormalities. From this analysis, our model attains better results in simulated data but it declines the accuracy while processing with real-time data. Moreover, this work has a significant medical impact since it presents a solution to the problem of gathering enough data during the epidemic to create a realistic model with a large dataset.

Fair non-contact blood pressure estimation using imaging photoplethysmography.

Hypertension is typically manifested as a latent symptom that requires detection through specialized equipment. This poses an inconvenience for individuals who need to undergo long-term blood pressure monitoring in their daily lives. Therefore, there is a need for a portable, non-contact method for estimating blood pressure. However, current non-contact blood pressure estimation methods often rely on relatively narrow datasets, lacking a broad range of blood pressure distributions. Additionally, their applicability is confined to controlled experimental environments. This study proposes a non-contact blood pressure estimation method suitable for various life scenarios, encompassing multiple age groups, diverse ethnicities, and individuals with different skin tones. The aim is to enhance the practicality and accuracy of existing non-contact blood pressure estimation methods. The research extracts the imaging photoplethysmogram (IPPG) signal from facial videos and processes the signal through four layers of filtering operations to obtain an IPPG signal reflecting pulse wave variations. A CNN+BiLSTM+GRU network structure is constructed to improve the accuracy of current non-contact blood pressure estimation methods. In comparison to existing approaches, the mean absolute error (MAE) for systolic blood pressure (SBP) and diastolic blood pressure (DBP) is reduced by 13.6% and 16.4%, respectively.

Assessing heart rate and blood pressure estimation from image photoplethysmography using a digital blood pressure meter

This study presents a non-contact approach to measuring heart rate and blood pressure using an image photoplethysmography (iPPG) signal, and compares the results to those from an oscillometric blood pressure meter. Facial videos of 100 subjects were recorded via a webcam under ambient lighting conditions to extract iPPG signals. The results revealed a strong correlation between the heart rate derived from iPPG and that obtained from an oscillometric blood pressure meter. In addition, a continuous wavelet transform images with a 6-s duration were used as input for a custom convolutional neural network model, providing the most accurate blood pressure estimation. The proposed method received a grade A for diastolic and grade B for systolic blood pressure based on the British Hypertension Society's criteria. It also met the standards set by the Association for the Advancement of Medical Instrumentation. This non-contact framework shows promising potential for efficient screening purposes.

Anti-motion imaging photoplethysmography via self-adaptive multi-ROI tracking and selection

Objective. The imaging photoplethysmography (IPPG) technique allows people to measure heart rate (HR) from face videos. However, motion artifacts caused by rigid head movements and nonrigid facial muscular movements are one of the key challenges. Approach. This paper proposes a self-adaptive region of interest (ROI) pre-tracking and signal selection method to resist motion artifacts. Based on robust facial landmark detection, we split the whole facial skin (including the forehead, cheeks, and chin) symmetrically into small circular regions. And two symmetric sub-regions constitute a complete ROI. These ROIs are tracked and the motion state is simultaneously assessed to automatically determine the visibility of these ROIs. The obscured or invisible sub-regions will be discarded while the corresponding symmetric sub-regions will be retained as available ROIs to ensure the continuity of the IPPG signal. In addition, based on the frequency spectrum features of IPPG signals extracted from different ROIs, a self-adaptive selection module is constructed to select the optimum IPPG signal for HR calculation. All these operations are updated per frame dynamically for the real-time monitor. Results. Experimental results on the four public databases show that the IPPG signal derived by our proposed method exhibits higher quality for more accurate HR estimation. Compared with the previous method, metrics of the evaluated HR value on our approach demonstrates superior or comparable performance on PURE, VIPL-HR, UBFC-RPPG and MAHNOB-HCI datasets. For instance, the RMSEs on PURE, VIPL-HR, and UBFC-RPPG datasets decrease from 4.29, 7.62, and 3.80 to 4.15, 3.87, and 3.35, respectively. Significance. Our proposed method can help enhance the robustness of IPPG in real applications, especially given motion disturbances.

Detail-preserving arterial pulse wave measurement based Biorthogonal wavelet decomposition from remote RGB observations

Extracting arterial pulse wave from the video, also known as Imaging Photoplethysmography (IPPG), has emerged as an efficient health monitoring approach. The detail-preserving arterial pulse wave can significantly improve healthcare efficiency through the provision of physiological and pathological information. However, the accurate measurement of arterial pulse wave by IPPG is still challenged by the fact that the pulsation is weakly expressed at the skin surface and extremely vulnerable to interferences. This work proposes a novel approach to measure the detail-preserving arterial pulse wave via interference isolation from facial video. A skin reflection model is introduced to guide the design of the IPPG measurement system, where the IPPG signal is pre-filtered to obtain the variation related to the pulse wave, and a Biorthogonal wavelet is further employed to distinguish the unique pattern of arterial subtle variation from interferences. Experimental results validate the efficiency of proposed method with a correlation coefficient of 0.92.

Modified Allen Test Assessment via Imaging Photoplethysmography

Abstract The modified Allen test (MAT) is a widely used tool to examine the perfusion capacities of the arteries (ulnar or radial) of the hand. In clinical practice, MAT is utilized to ensure that in the event of occlusion, remaining vessels guarantee sufficient hand perfusion. This work shows that perfusion assessment via imaging photoplethysmography (iPPG) can be used to assess MAT. We captured the hands of seven subjects with a standard RGB camera while performing a MAT. Out of these captured video recordings, the iPPG signal was extracted via plane-orthogonal-to-skin transformation. Based on these extracted signals, we derive the heart rate, signal-to-noise ratio, DC parameter, and correlation with a reference iPPG signal to determine if an Allen test was pathological or inconspicuous. For all seven subjects, the results of our MAT assessment were identical to the medical expert’s assessment. This paper presents a procedure and initial thresholds for objectively evaluating and quantifying MAT.We show how globally and locally calculated parameters can be used to assess the perfusion of the investigated hand. This analysis makes it possible to quantify the exact point in time when the perfusion is again sufficient. If a reperfusion problem occurs, we are able to localize the spatial regions of occurrence. The presented hardware setup is affordable and can be easily installed and used in every doctor’s office.

Deep Learning-Based Non-Contact IPPG Signal Blood Pressure Measurement Research

In this paper, a multi-stage deep learning blood pressure prediction model based on imaging photoplethysmography (IPPG) signals is proposed to achieve accurate and convenient monitoring of human blood pressure. A camera-based non-contact human IPPG signal acquisition system is designed. The system can perform experimental acquisition under ambient light, effectively reducing the cost of non-contact pulse wave signal acquisition while simplifying the operation process. The first open-source dataset IPPG-BP for IPPG signal and blood pressure data is constructed by this system, and a multi-stage blood pressure estimation model combining a convolutional neural network and bidirectional gated recurrent neural network is designed. The results of the model conform to both BHS and AAMI international standards. Compared with other blood pressure estimation methods, the multi-stage model automatically extracts features through a deep learning network and combines different morphological features of diastolic and systolic waveforms, which reduces the workload while improving accuracy.

Hand-over-face occlusion and distance adaptive heart rate detection based on imaging photoplethysmography and pixel distance in online learning

In online learning, imaging photoplethysmography (iPPG) can extract biometric information from the students’ face video, which can be used for further analysis of learning status. However, the iPPG-based methods cannot detect the heart rate accurately when the face-camera distance changes or the subject's face is occluded by the hand (Hand-Over-Face, HOF). This paper proposes a distance-adaptive heart rate detection model based on chroma and pixel distance, which can detect heart rate accurately with face-camera distance changing in a certain range. The influence factors of distance are verified on the region of interest (ROI) of the image in the process of iPPG signal extraction, and the relationship model is established between the pixel distance characteristic parameters and the physiological parameters through the BP neural network. The experimental results show that the RMSE of the model proposed in this paper is 1.50 bpm, and the model determination coefficient is 0.9504, which can effectively correct the heart rate detection error caused by face-camera distance changes. This paper proposes a gesture segmentation method GROI combined with gesture recognition to obtain an effective face ROI when HOF occurs. The experimental results show that when the ROI is occluded severely by the gesture, the RMSE for heart rate detection is reduced by 1.11 bpm, which demonstrates the effectiveness of the algorithm.

Arterial pulsation modulates the optical attenuation coefficient of skin.

Photoplethysmographic (PPG) signals arise from the modulation of light reflectivity on the skin due to changes of physiological origin. Imaging plethysmography (iPPG) is a video-based PPG method that can remotely monitor vital signs in a non-invasive manner. iPPG signals result from skin reflectivity modulation. The origin of such reflectivity modulation is still a subject of debate. Here, we have used optical coherence tomography (OCT) imaging to find whether iPPG signals may result from skin optical properties being directly or indirectly modulated by arterial transmural pressure propagation. The light intensity across the tissue was modeled through a simple exponential decay (Beer-Lambert law) to analyze in vivo the modulation of the optical attenuation coefficient of the skin by arterial pulsation. The OCT transversal images were acquired from a forearm of three subjects in a pilot study. The results show that the optical attenuation coefficient of skin changes at the same frequency as the arterial pulsation due to transmural pressure propagation (local ballistographic effect), but we cannot discard the contribution of global ballistographic effects.

Hybrid D1DCnet Using Forehead iPPG for Continuous and Noncontact Blood Pressure Measurement

Camera-based image sequences can be used excitingly to provide noncontact and continuous blood pressure (BP) measurement. This study proposes an innovative deep-learning approach that achieves the performance of current state of the art for BP detection using forehead imaging photoplethysmography (iPPG). We studied 814 subjects with iPPG waveforms and utilize a hybrid deep 1-D convolutional neural network (Hybrid D1DCnet) to predict their systolic BP (SBP) and diastolic BP (DBP). Our model uses deep 1-D convolutions to extract high-level morphological information of waveforms, uses the attention-based long short-term memory (LSTM) unit to learn temporal-dependent features, and combines personal information through the multilayer perceptron (MLP) to perform regression estimation. This model is capable of continuously monitoring BP by detecting noncontact facial iPPG signal over a short-time period of 8 s. The mean absolute error (MAE) and the standard deviation (STD) of estimated SBP and DBP are 8.36 ± 6.22 and 5.69 ± 3.97 mmHg, respectively. Interpretability studies show that the middle part of the rising edge of the iPPG pulse waveform is of most interest. It is also verified that this model has interdevice generalizability. This scheme achieves the C/B grade (systolic/diastolic) according to the British Society of Hypertension grading criteria. In the current work, we demonstrate the feasibility and advantages of the Hybrid D1DCnet-based noncontact and continuous BP measurement using forehead iPPG waveforms. Such applications of monitoring circulatory health are expected to promote the prevention and intervention of cardiovascular diseases for the public.

A Non-contact Heart Rate Measurement Approach Based on Modified EEMD Algorithm with Updated White-noise Iterating

At present, Imaging Photoplethysmography (IPPG) has attracted extensive attention on heart rate measurement, because of its non-contact, non-invasive and safe characteristics. However, in reality, the IPPG signal is extremely feeble and susceptible to be submerged by ambient interference such as illumination. To this end, we propose a new IPPG signal processing approach to enhance the anti-interference capability of measuring heart rate through a modified EEMD algorithm with updated white-noise iterating. In the modified EEMD algorithm, based on the standard deviation of the input IPPG signal, zero-mean white noises are generated randomly for iterative ensemble averaging. Through this new method, the fault caused by the residual white noise of traditional EEMD is solved. Lastly, through the comparative experiments of our own acquisition statistics, the approach proposed by this paper has significantly improved the performance in terms of accuracy, timeliness and anti-interference under various scenarios. The mean absolute error reaches 1.4 beats per minute and the correlation coefficient r reaches 0.8089. In addition, the modified EEMD algorithm reduces the running time by more than 10 times compared to the original method, approximately 0.16s per time.

Estimation of blood pressure waveform from facial video using a deep U-shaped network and the wavelet representation of imaging photoplethysmographic signals

Background:The remote measurement of physiological signals from video has gained a particular attention over the last past years. Estimating cardiovascular parameters like oxygen saturation and arterial blood pressure (BP) is covered by a limited volume of studies and remain a very challenging issue. Recent attempts demonstrated that BP can be estimated from facial video but under very controlled scenarios or with moderate performances. The data used in these works have not been publicly released or were gathered in a clinical setting. Methods:We, in contrast, propose a framework for estimating BP from publicly available data in order to allow replication and to facilitate fair comparison. We developed and trained a deep U-shaped neural network to recover the blood pressure waveform from its imaging photoplethysmographic (iPPG) signal counterpart. The model predicts the continuous wavelet transform (CWT) representation of a BP signal from the CWT of an iPPG signal. Inverse CWT transform is ultimately computed to recover the BP time series. Results:The proposed framework has been evaluated on 57 participants using international standards developed by the AAMI and the BHS. Results exhibit close agreement with ground truth BP values. The method satisfies all standards in the estimation of mean and diastolic BP (grade A) and nearly all standards in the estimation of systolic BP (grade B). Conclusions:This is, to the best of our knowledge, the first demonstration of a deep learning-oriented framework that manages to predict the continuous blood pressure waveform from facial video analysis. Codes developed during the study are publicly available (https://github.com/frederic-bousefsaf/ippg2bp).

Non-Contact Abnormal Physiological Status Detection During Sport and Training

In sport and training, it is necessary to continue monitoring the physiological parameters of athletes to ensure that they can maintain a high level of competition. The previous monitoring physiological status methods mainly are contactable by sensors that are worn on the body. This paper adopts a non-contact physiological parameter monitoring method by using imaging photoplethysmography (iPPG). In order to eliminate the noises in iPPG signals, the correlation energy entropy threshold adaptive denoising and variance characterization series are introduced to resist the noises from external conditions. The noises are removed by a threshold which is estimated by noise energy entropy. The constructed signals after denoising are used to estimate physiological parameters, such as heart rate and respiratory rate. The experimental results demonstrate that it estimates the physiological parameters better by using iPPG-based physiological parameter monitoring method than previous methods.

Rational selection of RGB channels for disease classification based on IPPG technology.

The green channel is usually selected as the optimal channel for vital signs monitoring in image photoplethysmography (IPPG) technology. However, some controversies arising from the different penetrability of skin tissue in visible light remain unresolved, i.e., making the optical and physiological information carried by the IPPG signals of the RGB channels inconsistent. This study clarifies that the optimal channels for different diseases are different when IPPG technology is used for disease classification. We further verified this conclusion in the classification model of heart disease and diabetes mellitus based on the random forest classification algorithm. The experimental results indicate that the green channel has a considerably excellent performance in classifying heart disease patients and the healthy with an average Accuracy value of 88.43% and an average F1score value of 93.72%. The optimal channel for classifying diabetes mellitus patients and the healthy is the red channel with an average Accuracy value of 82.12% and the average F1score value of 89.31%. Due to the limited penetration depth of the blue channel into the skin tissue, the blue channel is not as effective as the green and red channels as a disease classification channel. This investigation is of great significance to the development of IPPG technology and its application in disease classification.

IPPG 2 cPPG: Reconstructing contact from imaging photoplethysmographic signals using U-Net architectures

Imaging photoplethysmography (iPPG) is an optical technique dedicated to the assessment of several vital functions using a simple camera. Significant efforts have been made to reliably estimate heart and respiratory rates. Currently, research is focusing on the remote estimation of oxygen saturation and blood pressure (BP). The limited number of publicly available data tends to restrict the advancements related to BP estimation. To overcome this limit, we propose to split the problem in a two-stage processing chain: (i) converting iPPG to contact PPG (cPPG) signals using available video dataset and (ii) estimate BP from converted cPPG signals by exploiting large existing databases (e.g. MIMIC). This article presents the first developments where a method for converting iPPG signals measured using a camera into cPPG signals measured by contact sensors is proposed. Real and imaginary parts of the continuous wavelet transform (CWT) of cPPG and iPPG signals are passed to various deep pre-trained U-shaped architectures. Conventional metrics and specific waveform estimators have been implemented to validate the relevance of the predictions. The results exhibit good agreements towards a large portion of metrics, showing that the neural architectures properly estimated cPPG from iPPG signals through their CWT representations. The performance indicates that BP estimation from iPPG signals converted to cPPG signals can now be envisaged. Consequently, future work will focus on the integration of models dedicated to BP estimation trained on MIMIC. This is the first demonstration of a method for accurate reconstruction of cPPG from iPPG signals satisfying pulse waveform criteria.

EFFECTS OF SKIN TONE AND AMBIENT LIGHTING ON ACCURACY OF CONTACTLESS MEASUREMENT OF HEART RATE AND PULSE TRANSIT TIME: AN INVESTIGATION TOWARD IMAGE-BASED BLOOD PRESSURE ESTIMATION

Abstract Objective: Image-based photoplethysmography (iPPG) using video of the skin offers potential for contactless blood pressure (BP) estimation from the pulse transit time (PTT). Optical properties of human skin and ambient lighting are likely to affect accuracy of image-based PTT (iPPT) and are investigated here. Design and method: Twelve healthy volunteers (6 female, 27 ± 6 years) with a range of skin tones (Fitzpatrick skin scores of 5 to 27) were recruited. For each subject, the experiment was performed in standard fluorescent room lighting (100 lux) and repeated at five other levels of light intensity (40–310 lux). Two-minute videos were acquired from each subject's face and palm of the left hand; simultaneous electrocardiogram (ECG) and non-invasive finger BP were also measured. In each frame, two regions of interest (ROIs) were selected from the subject's face and hand, and the average of green colour intensity for each ROI were calculated as iPPG. The iPPG signal was band-pass filtered between 0.66–3 Hz, (corresponding to a 40–180 bpm heart rate (HR) range). HR was calculated as the dominant peak in the Fourier spectrum of filtered iPPG. Beat-to-beat values of iPTT were calculated between the corresponding feet of the iPPG pulsations. Results: The absolute error between average HR (errHR) determined from ECG and that calculated from iPPG in both ROIs was significantly correlated with room lumen (RL) and skin score (SC) (errHR = – 0.08 × RL + 0.67 × SC + 12, p < 0.05). The minimum HR error was in standard room lighting (errHR = 3.9 ± 8.9 bpm). Using average iPTT in room lighting as the reference, the percentage error (iPTT%err) between reference iPTT and iPTT in other lighting conditions significantly decreased with lower lighting conditions but was not significantly changed with skin tone score (iPTT%err = -0.09 × RL + 55, p < 0.05). Conclusions: Skin tone and ambient lighting intensity affects accuracy of contactless HR measurement. Only light intensity affects accuracy of iPTT measurement. It is likely that errors in contactless BP estimation are minimised in standard room lighting conditions but increase in dimmer conditions.

HRVCam: robust camera-based measurement of heart rate variability.

.Significance: Non-contact, camera-based heart rate variability estimation is desirable in numerous applications, including medical, automotive, and entertainment. Unfortunately, camera-based HRV accuracy and reliability suffer due to two challenges: (a) darker skin tones result in lower SNR and (b) relative motion induces measurement artifacts.Aim: We propose an algorithm HRVCam that provides sufficient robustness to low SNR and motion-induced artifacts commonly present in imaging photoplethysmography (iPPG) signals.Approach: HRVCam computes camera-based HRV from the instantaneous frequency of the iPPG signal. HRVCam uses automatic adaptive bandwidth filtering along with discrete energy separation to estimate the instantaneous frequency. The parameters of HRVCam use the observed characteristics of HRV and iPPG signals.Results: We capture a new dataset containing 16 participants with diverse skin tones. We demonstrate that HRVCam reduces the error in camera-based HRV metrics significantly (more than 50% reduction) for videos with dark skin and face motion.Conclusion: HRVCam can be used on top of iPPG estimation algorithms to provide robust HRV measurements making camera-based HRV practical.

Non-Contact HR Monitoring via Smartphone and Webcam During Different Respiratory Maneuvers and Body Movements.

As a reliable indicator for individual's healthiness conditions, heart rate (HR) has been widely considered and used. Imaging photoplethysmography (iPPG) is recently highlighted as a promising HR measurement method, due to its non-contact characteristics, by extracting the HR from facial video recordings. In this study, we propose a camera-based HR monitoring technique that estimates HR information from iPPG signals extracted from a video sequence. Videos were recorded using a smartphone or a laptop camera. We adopted the plane-orthogonal-to-skin (POS) method to compute iPPG. The proposed method is evaluated by applying it to extract HR of 9 subjects at rest and during two motion conditions (lateral and frontal) while they were performing several respiratory maneuvers-spontaneous, metronome, and forced. Automatic face detection algorithms were implemented in the proposed method. Our experimental results show that mean values of HR have 0.56% error and 99.4% accuracy when compared to HR calculated from the gold-standard electrocardiography (ECG) reference in diverse conditions of motions and respiratory maneuvers.

Near-Infrared Imaging Photoplethysmography During Driving

Imaging photoplethysmography (iPPG) could greatly improve driver safety systems by enabling capabilities ranging from identifying driver fatigue to unobtrusive early heart failure detection. Unfortunately, the driving context poses unique challenges to iPPG, including illumination and motion. First, drastic illumination variations present during driving can overwhelm the small intensity-based iPPG signals. Second, significant driver head motion during driving, as well as camera motion (e.g., vibration) make it challenging to recover iPPG signals. To address these two challenges, we present two innovations. First, we demonstrate that we can reduce most outside light variations using narrow-band near-infrared (NIR) video recordings and obtain reliable heart rate estimates. Second, we present a novel optimization algorithm, which we call AutoSparsePPG, that leverages the quasi-periodicity of iPPG signals and achieves better performance than the state-of-the-art methods. In addition, we release the first publicly available driving dataset that contains both NIR and RGB video recordings of a passenger’s face with simultaneous ground truth pulse oximeter recordings.