Camera-based Photoplethysmography Research Articles

CbPPGGAN: A Generic Enhancement Framework for Unpaired Pulse Waveforms in Camera-Based Photoplethysmography.

Camera-based photoplethysmography (cbP PG) is a non-contact technique that measures cardiac-related blood volume alterations in skin surface vessels through the analysis of facial videos. While traditional approaches can estimate heart rate (HR) under different illuminations, their accuracy can be affected by motion artifacts, leading to poor waveform fidelity and hindering further analysis of heart rate variability (HRV); deep learning-based approaches reconstruct high-quality pulse waveform, yet their performance significantly degrades under illumination variations. In this work, we aim to leverage the strength of these two methods and propose a framework that possesses favorable generalization capabilities while maintaining waveform fidelity. For this purpose, we propose the cbPPGGAN, an enhancement framework for cbPPG that enables the flexible incorporation of both unpaired and paired data sources in the training process. Based on the waveforms extracted by traditional approaches, the cbPPGGAN reconstructs high-quality waveforms that enable accurate HR estimation and HRV analysis. In addition, to address the lack of paired training data problems in real-world applications, we propose a cycle consistency loss that guarantees the time-frequency consistency before/after mapping. The method enhances the waveform quality of traditional POS approaches in different illumination tests (BH-rPPG) and cross-datasets (UBFC-rPPG) with mean absolute error (MAE) values of 1.34 bpm and 1.65 bpm, and average beat-to-beat (AVBB) values of 27.46 ms and 45.28 ms, respectively. Experimental results demonstrate that the cbPPGGAN enhances cbPPG signal quality and outperforms the state-of-the-art approaches in HR estimation and HRV analysis. The proposed framework opens a new pathway toward accurate HR estimation in an unconstrained environment.

Vital signs estimation in elderly using camera-based photoplethysmography

In the context of Ambient Assisted Living, the demand for healthcare technologies development has been increased in the last two years by the current global health situation. The contagious nature of COVID-19 warrants inevitability to the thought of easing a continuous measurement of vital signs such as Heart Rate, Breath Rate, and SpO2 in a non-invasive manner, contributing of fundamental importance in the assessment of health status especially among frail and/or elderly individuals. A widely diffused method for contactless vital signs monitoring is remote photoplethysmography from facial video streams that, contrary to traditional contact measurement techniques, allows the measurement of vital parameters without the need for wearable sensors (generally considered uncomfortable, especially by the elderly), even with commercial and low-cost digital cameras. This paper proposes the design and implementation of a new pipeline for estimating Heart Rate, Breath Rate, and SpO2 values, and its integration on Raspberry Pi 4 as an elaboration unit. The pipeline provides specific algorithmic blocks to improve vital signs estimation in elderly subjects as it is made more difficult by the skin tone and the presence on the face of wrinkles, folds, and moles. Quantitative evaluations on our dataset containing the acquisition of only elderly older than 65 years of age demonstrate the validity of the proposed pipeline. For validation against the state of the art, tests were also conducted on three standard benchmark datasets containing video with subjects of varying ages. Again, the pipeline proved to be robust concerning the estimation of vital signs considered in the present work.

Benchmarking of Contactless Heart Rate Measurement Systems in ARM-Based Embedded Platforms

Heart rate monitoring is especially important for aging individuals because it is associated with longevity and cardiovascular risk. Typically, this vital parameter can be measured using wearable sensors, which are widely available commercially. However, wearable sensors have some disadvantages in terms of acceptability, especially when used by elderly people. Thus, contactless solutions have increasingly attracted the scientific community in recent years. Camera-based photoplethysmography (also known as remote photoplethysmography) is an emerging method of contactless heart rate monitoring that uses a camera and a processing unit on the hardware side, and appropriate image processing methodologies on the software side. This paper describes the design and implementation of a novel pipeline for heart rate estimation using a commercial and low-cost camera as the input device. The pipeline's performance was tested and compared on a desktop PC, a laptop, and three different ARM-based embedded platforms (Raspberry Pi 4, Odroid N2+, and Jetson Nano). The results showed that the designed and implemented pipeline achieved an average accuracy of about 96.7% for heart rate estimation, with very low variance (between 1.5% and 2.5%) across processing platforms, user distances from the camera, and frame resolutions. Furthermore, benchmark analysis showed that the Odroid N2+ platform was the most convenient in terms of CPU load, RAM usage, and average execution time of the algorithmic pipeline.

Fundamentals of Camera-PPG Based Magnetic Resonance Imaging.

In Magnetic Resonance Imaging (MRI), cardiac triggering that synchronizes data acquisition with cardiac contractions is an essential technique for acquiring high-quality images. Triggering is typically based on the Electrocardiogram (ECG) signal (e.g. R-peak). Since ECG acquisition involves extra workflow steps like electrode placement and ECG signals are usually disturbed by magnetic fields in high Magnetic Resonance (MR) systems, we explored camera-based photoplethysmography (PPG) as an alternative. We used the in-bore camera of a clinical MR system to investigate the feasibility and challenges of camera-based cardiac triggering. Data from ECG, finger oximeter and camera were synchronously collected. Compared to finger-PPG, camera-based PPG provides a higher availability of the signal and the PPG marker delay relative to the ECG R-peak is considerably less with a camera monitoring the forehead. The insights obtained in this study provide a basis for an envisioned system-design phase.

Contactless heart rate measurement in newborn infants using a multimodal 3D camera system.

Newborns and preterm infants require accurate and continuous monitoring of their vital parameters. Contact-based methods of monitoring have several disadvantages, thus, contactless systems have increasingly attracted the neonatal communities' attention. Camera-based photoplethysmography is an emerging method of contactless heart rate monitoring. We conducted a pilot study in 42 healthy newborn and near-term preterm infants for assessing the feasibility and accuracy of a multimodal 3D camera system on heart rates (HR) in beats per min (bpm) compared to conventional pulse oximetry. Simultaneously, we compared the accuracy of 2D and 3D vision on HR measurements. The mean difference in HR between pulse oximetry and 2D-technique added up to + 3.0 bpm [CI−3.7 – 9.7; p = 0.359, limits of agreement (LOA) ± 36.6]. In contrast, 3D-technique represented a mean difference in HR of + 8.6 bpm (CI 2.0–14.9; p = 0.010, LOA ± 44.7) compared to pulse oximetry HR. Both, intra- and interindividual variance of patient characteristics could be eliminated as a source for the results and the measuring accuracy achieved. Additionally, we proved the feasibility of this emerging method. Camera-based photoplethysmography seems to be a promising approach for HR measurement of newborns with adequate precision; however, further research is warranted.

Camera-based assessment of cutaneous perfusion strength in a clinical setting

Objective. After skin flap transplants, perfusion strength monitoring is essential for the early detection of tissue perfusion disorders and thus to ensure the survival of skin flaps. Camera-based photoplethysmography (cbPPG) is a non-contact measurement method, using video cameras and ambient light, which provides spatially resolved information about tissue perfusion. It has not been researched yet whether the measurement depth of cbPPG, which is limited by the penetration depth of ambient light, is sufficient to reach pulsatile vessels and thus to measure the perfusion strength in regions that are relevant for skin flap transplants. Approach. We applied constant negative pressure (compared to ambient pressure) to the anterior thighs of 40 healthy subjects. Seven measurements (two before and five up to 90 min after the intervention) were acquired using an RGB video camera and photospectrometry simultaneously. We investigated the performance of different algorithmic approaches for perfusion strength assessment, including the signal-to-noise ratio (SNR), its logarithmic components logS and logN, amplitude maps, and the amplitude height of alternating and direct signal components. Main results. We found strong correlations of up to r = 0.694 (p < 0.001) between photospectrometric measurements and all cbPPG parameters except SNR when using the green color channel. The transfer of cbPPG signals to POS, CHROM, and O3C did not lead to systematic improvements. However, for direct signal components, the transformation to O3C led to correlations of up to r = 0.744 (p < 0.001) with photospectrometric measurements. Significance. Our results indicate that a camera-based perfusion strength assessment in tissue with deep-seated pulsatile vessels is possible.

Camera-based Photoplethysmography (cbPPG) using smartphone rear and frontal cameras: an experimental study.

Non-expensive methods for measuring heart rate and oxygen saturation are of great importance in the scope of the COVID-19 outbreak to follow up on the symptoms and help to control the disease.Smartphones are widely available and their cameras can be used to acquire relevant physiological data, such as Photo-plethysmography (PPG) signals. Covering a light source and the camera sensor with a finger, it is possible to acquire the camera-based photoplethysmography (cbPPG) signal. Two methods were analyzed in this work, namely using the rear smartphone camera and the flash LED, and using the front camera and device display as a light source. The latter presents more advantages overall - in particular, greater control over the emitted light and finger detection - and better results were found when compared to a reference device.Clinical relevance- This technology allows the pervasive monitoring of the PPG signal using a standard smartphone, providing a tool to evaluate the subject's heart rate and its variability, respiration, blood oxygenation, etc.

Optimal color channel combination across skin tones for remote heart rate measurement in camera-based photoplethysmography

ObjectiveThe heart rate is an essential vital sign that can be measured remotely with camera-based photoplethysmography (cbPPG). Systems for cbPPG typically use cameras that deliver red, green, and blue (RGB) channels. The combination of these channels has been proven to increase signal-to-noise ratio (SNR) and heart rate measurement accuracy (ACC). However, many combinations remain untested, the comparison of proposed combinations on large datasets is insufficiently investigated, and the interplay with skin tone is rarely addressed. MethodsEight regions of interest and eight color spaces with a total of 25 color channels were compared in terms of ACC and SNR based on the Binghamton-Pittsburgh-RPI Multimodal Spontaneous Emotion Database (BP4D+). Additionally, two systematic grid searches were performed to evaluate ACC in the space of linear combinations of the RGB channels. ResultsGlabella and forehead regions of interest provided highest ACC (up to 74.1 %) and SNR (> -3 dB) with the hue channel H from HSV color space and the chrominance channel Q from NTSC color space. The grid searches revealed a global optimum of linear RGB combinations (ACC: 79.2 %). This optimum occurred for all skin tones, although ACC dropped for darker skin tones. ConclusionThrough systematic grid searches we were able to identify the skin tone independent optimal linear RGB color combination for measuring heart rate with cbPPG. Our results proved on a large dataset that the identified optimum outperformed conventionally used color channels. SignificanceThe presented findings provide useful evidence for future considerations of algorithmic approaches for cbPPG.

Intraoperative Imaging of Cortical Blood Flow by Camera-Based Photoplethysmography at Green Light

Intraoperative evaluation of blood perfusion in the brain cortex is an important but hitherto unresolved problem. Our aim was to demonstrate the feasibility of cerebral microcirculation assessment during open brain surgery by using camera-based photoplethysmography (cbPPG) synchronized with an electrocardiograph. Cortical blood flow was monitored in five patients with different diagnoses. Two cases (tumor resection and extra-intracranial bypass grafting) are presented in detail. Blood-flow parameters were visualized after processing cortex images recorded under green-light illumination before and after surgical intervention. In all cases, blood flow was successfully visualized in &gt;95% of open brain. Distributions of blood pulsation amplitude, a parameter related to cortical blood perfusion; pulse arrival time; and blood-pressure-pulse shape were calculated with high spatial resolution (in every pixel). Changes in cerebral blood supply caused by surgical intervention were clearly revealed. We have shown that the temporal spread of pulse arrival time and the spatiotemporal variability of pulse shape are very sensitive markers of brain circulatory disturbances. The green-light cbPPG system offers a new approach to objective assessment of blood-flow changes in the brain during surgical intervention. The proposed system allows for contactless monitoring of cortex blood flow in real time with high resolution, thus providing useful information for surgery optimization and minimization of brain tissue damage.

Remote Photoplethysmographic Assessment of the Peripheral Circulation in Critical Care Patients Recovering From Cardiac Surgery.

Camera-based photoplethysmography (cbPPG) remotely detects the volume pulse of cardiac ejection in the peripheral circulation. The cbPPG signal is sourced from the cutaneous microcirculation, yields a 2-dimensional intensity map, and is therefore an interesting monitoring technique. In this study, we investigated whether cbPPG is in general sufficiently sensitive to discern hemodynamic conditions. cbPPG recordings of 70 patients recovering from cardiac surgery were analyzed. Photoplethysmograms were processed offline and the optical pulse power (OPP) of cardiac ejection was calculated. Hemodynamic data, image intensity, and patient movements were recorded synchronously. The effects of hemodynamic parameters and measurement conditions on the patient's individual OPP variability and their actual OPP values were calculated in mixed-effects regression models. Mean arterial pressure (MAP), pulse pressure (PP), heart rate (HR), and central venous pressure (CVP) significantly explained the individual OPP variability. PP had the highest explanatory power (19.9%). Averaged OPP significantly increased with PP and MAP (P < 0.001, respectively) and decreased with higher HR (P = 0.024). CVP had a 2-directional, nonsignificant effect on averaged OPP. Image intensity and patient movements did significantly affect OPP. After adjustment for hemodynamic covariables and measurement conditions, the effect of PP and HR remained unchanged, whereas that of MAP vanished. cbPPG is sensitive to hemodynamic parameters in critical care patients. It is a potential application for monitoring the peripheral circulation. Its value in a clinical setting has to be determined.

Pulse decomposition analysis in camera-based photoplethysmography.

Imaging photoplethysmography (iPPG) is an interesting alternative to laser speckle contrast imaging for the analysis of spatio-temporal patterns in the cutaneous microcirculation. Recent years have witnessed the development of sophisticated techniques for the non-invasive extraction of vascular-related features. These techniques, referred to as pulse decomposition algorithms (PDA), most often involve the analysis of photoplethysmographic waves. This study validated the use of a multi-Gaussian (PDA) for the automatic mapping of iPPG pulse waveforms acquired with a standard camera. We show that iPPG-based PDA can reveal differences in skin perfusion in response to cold stimuli. The study thus proves the potential for morphological analyses of the iPPG pulse waveform.

Waveform Analysis for Camera-based Photoplethysmography Imaging.

With the advent of sensitive and affordable cameras, classical contact-based photoplethysmography (PPG) could be enhanced to the spatial domain. Cost-efficient cameras are available in everyday items such as smartphones or computer webcams. The PPG signal, blood volume changes in the vascularity, can be measured remotely by using the camera as a 2-D-PPG detector. However, the evaluation of the extracted signals has mostly been limited to the pulse rate and sometimes the systolic amplitude. In this work, we motivate to generate images and video sequences based on features from the PPG waveform commonly not extracted via cameras. This is achieved by calculating the features for timeseries extracted from an evenly spaced grid of virtual PPG sensors. We briefly discuss the adaption of conventional PPG algorithms to camera-based PPG imaging (PPGI). The extracted parameters are associated with vessel properties and thus, mapping these to images could lead to enhanced vascular diagnostics. In this work, we test the feasibility of the mapping approach: we present the preliminary results gathered from the analysis of two videos of lab experiments with healthy subjects.

Alterations of blood pulsations parameters in carotid basin due to body position change

The velocity of the pulse wave (PWV) propagating through the vascular tree is an essential parameter for diagnostic the state of the cardiovascular system especially when it is measured in the pool of carotid arteries. In this research, we showed for the first time that the time of the blood-pressure-wave propagation from the heart to the face is a function of the body position. Significant asymmetry and asynchronicity of blood pulsations in the facial area were found in a recumbent position. Parameters of blood pulsations were measured by an advanced camera-based photoplethysmography system in 73 apparently healthy subjects. Most likely, observed changes of the blood-pulsation parameters are caused by variations of the arterial blood pressure due to hydrostatic pressure changes, and secondary reaction of blood vessels in response to these variations. Demonstrated feasibility of PWV measurements in the pool of carotid arteries provides considerable advantages over other technologies. Moreover, possibilities of the method to estimate physiological regulation of the peripheral blood flow (particularly, as a response to the gravitational changes) have been demonstrated. The proposed concept allows development of non-invasive medical equipment capable of solving a wide range of scientific and practical problems related to vascular physiology.

Camera-based photoplethysmography in an intraoperative setting

BackgroundCamera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment.MethodsWe recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity.ResultsThe results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations.ConclusionsThe proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.

The value of polarization in camera-based photoplethysmography

Camera-based photoplethysmography (cbPPG) is a novel measuring technique that permits the remote acquisition of cardiovascular signals using video cameras. Research still lacks in fundamental studies to reach a deeper technical and physiological understanding. This work analyzes the employment of polarization filtration to (i) assess the gain for the signal quality and (ii) draw conclusions about the cbPPG signal's origin. We evaluated various forehead regions of 18 recordings with different color and filter settings. Our results prove that for an optimal illumination, the perpendicular filter setting provides a significant benefit. The outcome supports the theory that signals arise from blood volume changes. For lateral illumination, ballistocardiographic effects dominate the signal as polarization's impact vanishes.

A systematic review of novel technology for monitoring infant and newborn heart rate.

This systematic review identified seven new technologies, including camera-based photoplethysmography, reflectance pulse oximetry, laser Doppler methods, capacitive sensors, piezoelectric sensors, electromyography and a digital stethoscope. Clinicians should be aware of several of these, which may become available for clinical use in the near future.

Camera-Based, Non-Contact, Vital-Signs Monitoring Technology May Provide a Way for the Early Prevention of SIDS in Infants.

Sudden infant death syndrome (SIDS) is the unexplained death, usually during sleep, of a baby younger than 1-year-old. Even though researchers have discovered some factors that may put babies at extra risk, SIDS remains unpredictable up until now. One hypothesis is that impaired cardiovascular control may play a role in the underlying mechanism of SIDS. A reduction of heart rate variability (HRV) and progressive decrease in heart rate (HR) have been observed in infants who have later succumbed to SIDS. Many clues indicated the heart could be the final weakness in SIDS. Therefore, continuous monitoring of the dynamic changes within the heart may provide a possible preventive strategy of SIDS. Camera-based photoplethysmography was recently demonstrated as a contactless method to determine HR and HRV. This perspective presents a hypothesis that a camera-based, non-contact, vital-sign monitoring technology, which can indicate abnormal changes or a sudden loss of vital signs in a timely manner, may enable a crucial and low-cost means for the early prevention of SIDS in newborn infants.

Calibration of Contactless Pulse Oximetry

Contactless, camera-based photoplethysmography (PPG) interrogates shallower skin layers than conventional contact probes, either transmissive or reflective. This raises questions on the calibratability of camera-based pulse oximetry. We made video recordings of the foreheads of 41 healthy adults at 660 and 840 nm, and remote PPG signals were extracted. Subjects were in normoxic, hypoxic, and low temperature conditions. Ratio-of-ratios were compared to reference SpO2 from 4 contact probes. A calibration curve based on artifact-free data was determined for a population of 26 individuals. For an SpO2 range of approximately 83% to 100% and discarding short-term errors, a root mean square error of 1.15% was found with an upper 99% one-sided confidence limit of 1.65%. Under normoxic conditions, a decrease in ambient temperature from 23 to 7°C resulted in a calibration error of 0.1% (±1.3%, 99% confidence interval) based on measurements for 3 subjects. PPG signal strengths varied strongly among individuals from about 0.9 × 10 to 4.6 × 10 for the infrared wavelength. For healthy adults, the results present strong evidence that camera-based contactless pulse oximetry is fundamentally feasible because long-term (eg, 10 minutes) error stemming from variation among individuals expressed as A*rms is significantly lower (<1.65%) than that required by the International Organization for Standardization standard (<4%) with the notion that short-term errors should be added. A first illustration of such errors has been provided with A**rms = 2.54% for 40 individuals, including 6 with dark skin. Low signal strength and subject motion present critical challenges that will have to be addressed to make camera-based pulse oximetry practically feasible.

Camera-based photoplethysmography in critical care patients.

Camera-based photoplethysmography (cbPPG) is an optical measurement technique that reveals pulsatile blood flow in cutaneous microcirculation from a distance. cbPPG has been shown to reflect pivotal haemodynamic events like cardiac ejection in healthy subjects. In addition, it provides valuable insight into intrinsic microcirculatory regulation as it yields dynamic, two-dimensional perfusion maps. In this study, we evaluate the feasibility of a clinical cbPPG application in critical care patients. A mobile camera set-up to record faces of patients at the bed site was constructed. Videos were made during the immediate recovery after cardiac surgery under standard critical care conditions and were processed offline. Major motion artefacts were detected using an optical flow technique and suitable facial regions were manually annotated. cbPPG signals were highpass filtered and Fourier spectra out of consecutive 10s signal segments calculated for heart rate detection. Signal-to-noise ratios (SNR) of the Fourier spectra were derived as a quality measure. Reference data of vital parameters were synchronously acquired from the bed site monitoring system. Seventy patient videos of an average time of 28.6±2.8 min were analysed. Heart rate (HR) was detected within a±5 bpm range compared to reference in 83% of total recording time. Low SNR and HR detection failure were mostly, but not exclusively, attributed to non-physiological events like patient motion, interventions or sudden changes of illumination. SNR was reduced by low arterial blood pressure, whereas no impact of other perioperative or disease-related parameters was identified. Cardiac ejection is detectable by cbPPG under pathophysiologic conditions of cardiovascular disease and perioperative medicine. cbPPG measurements can be seamlessly integrated into the clinical work flow of critical care patients.

Vasomotor assessment by camera-based photoplethysmography

Abstract Camera-based photoplethysmography (cbPPG) is a novel technique that allows the contactless acquisition of cardio-respiratory signals. Previous works on cbPPG most often focused on heart rate extraction. This contribution is directed at the assessment of vasomotor activity by means of cameras. In an experimental study, we show that vasodilation and vasoconstriction both lead to significant changes in cbPPG signals. Our findings underline the potential of cbPPG to monitor vasomotor functions in real-life applications.