Measurement Of Physiological Signals Research Articles

Self-Supervised Representation Learning Framework for Remote Physiological Measurement Using Spatiotemporal Augmentation Loss

Recent advances in supervised deep learning methods are enabling remote measurements of photoplethysmography-based physiological signals using facial videos. The performance of these supervised methods, however, are dependent on the availability of large labelled data. Contrastive learning as a self-supervised method has recently achieved state-of-the-art performances in learning representative data features by maximising mutual information between different augmented views. However, existing data augmentation techniques for contrastive learning are not designed to learn physiological signals from videos and often fail when there are complicated noise and subtle and periodic colour/shape variations between video frames. To address these problems, we present a novel self-supervised spatiotemporal learning framework for remote physiological signal representation learning, where there is a lack of labelled training data. Firstly, we propose a landmark-based spatial augmentation that splits the face into several informative parts based on the Shafer’s dichromatic reflection model to characterise subtle skin colour fluctuations. We also formulate a sparsity-based temporal augmentation exploiting Nyquist–Shannon sampling theorem to effectively capture periodic temporal changes by modelling physiological signal features. Furthermore, we introduce a constrained spatiotemporal loss which generates pseudo-labels for augmented video clips. It is used to regulate the training process and handle complicated noise. We evaluated our framework on 3 public datasets and demonstrated superior performances than other self-supervised methods and achieved competitive accuracy compared to the state-of-the-art supervised methods. Code is available at https://github.com/Dylan-H-Wang/SLF-RPM.

Estimation of blood pressure waveform from facial video using a deep U-shaped network and the wavelet representation of imaging photoplethysmographic signals

Background:The remote measurement of physiological signals from video has gained a particular attention over the last past years. Estimating cardiovascular parameters like oxygen saturation and arterial blood pressure (BP) is covered by a limited volume of studies and remain a very challenging issue. Recent attempts demonstrated that BP can be estimated from facial video but under very controlled scenarios or with moderate performances. The data used in these works have not been publicly released or were gathered in a clinical setting. Methods:We, in contrast, propose a framework for estimating BP from publicly available data in order to allow replication and to facilitate fair comparison. We developed and trained a deep U-shaped neural network to recover the blood pressure waveform from its imaging photoplethysmographic (iPPG) signal counterpart. The model predicts the continuous wavelet transform (CWT) representation of a BP signal from the CWT of an iPPG signal. Inverse CWT transform is ultimately computed to recover the BP time series. Results:The proposed framework has been evaluated on 57 participants using international standards developed by the AAMI and the BHS. Results exhibit close agreement with ground truth BP values. The method satisfies all standards in the estimation of mean and diastolic BP (grade A) and nearly all standards in the estimation of systolic BP (grade B). Conclusions:This is, to the best of our knowledge, the first demonstration of a deep learning-oriented framework that manages to predict the continuous blood pressure waveform from facial video analysis. Codes developed during the study are publicly available (https://github.com/frederic-bousefsaf/ippg2bp).

Flexible Non-contact Electrodes for Wearable Biosensors System on Electrocardiogram Monitoring in Motion.

Electrocardiogram (ECG) is a critical physiological indicator that contains abundant information about human heart activities. However, it is a kind of weak low-frequency signal, which is easy to be interfered by various noises. Therefore, wearable biosensors (WBS) technique is introduced to overcome this challenge. A flexible non-contact electrode is proposed for wearable biosensors (WBS) system, which is made up of flexible printed circuits materials, and can monitor the ECG signals during exercise for a long time. It uses the principle of capacitive coupling to obtain high-quality signals, and reduces the impact of external noise through active shielding; The results showed that the proposed non-contact electrode was equivalent to a medical wet electrode. The correlation coefficient was as high as 99.70 ± 0.30% when the subject was resting, while it was as high as 97.53 ± 1.80% during exercise. High-quality ECG could still be collected at subjects walking at 7 km/h. This study suggested that the proposed flexible non-contact electrode would be a potential tool for wearable biosensors for medical application on long-term monitoring of patients’ health and provide athletes with physiological signal measurements.

Psychological and Emotional Recognition of Preschool Children Using Artificial Neural Network.

The artificial neural network (ANN) is employed to study children’s psychological emotion recognition to fully reflect the psychological status of preschool children and promote the healthy growth of preschool children. Specifically, the ANN model is used to construct the human physiological signal measurement platform and emotion recognition platform to measure the human physiological signals in different psychological and emotional states. Finally, the parameter values are analyzed on the emotion recognition platform to identify the children’s psychological and emotional states accurately. The experimental results demonstrate that the recognition ability of children aged 4–6 to recognize the three basic emotions of happiness, calm, and fear increases with age. Besides, there are significant age differences in children’s recognition of happiness, calm, and fear. In addition, the effect of 4-year-old children on the theory of mind tasks is less than that of 5- to 6-year-old children, which may be related to more complex cognitive processes. Preschool children are experiencing a stage of rapid emotional development. If children cannot be guided to reasonably identify and deal with emotions at this stage, their education level and social ability development will be significantly affected. Therefore, this study has significant reference value for preschool children’s emotional recognition and guidance and can promote children’s emotional processing and mental health.

Motion Artifacts Correction From EEG and fNIRS Signals Using Novel Multiresolution Analysis

Physiological signal measurement and processing are increasingly becoming popular in the ambulatory setting as the hospital-centric treatment is moving towards wearable and ubiquitous monitoring. Most of the physiological signals are highly susceptible to various types of noises, especially movement artifacts. The electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals are no exception to motion artifacts, which become prominent in the ambulatory setting. Since successful detection of various neurological disorders is greatly dependent upon clean EEG and fNIRS signals, it is a matter of utmost importance to remove motion artifacts from these two signal modalities using reliable and robust methods. This paper proposes three novel multiresolution analysis techniques: i) Variational mode decomposition (VMD), ii) VMD in combination with principal component analysis (VMD-PCA), and iii) VMD in combination with canonical correlation analysis (VMD-CCA), for motion artifact correction from single-channel EEG and fNIRS signals. The efficacy of these novel techniques is validated by computing the difference in the signal to noise ratio (<inline-formula> <tex-math notation="LaTeX">$\Delta SNR$ </tex-math></inline-formula>) and percentage reduction in motion artifacts (<inline-formula> <tex-math notation="LaTeX">$\eta$ </tex-math></inline-formula>). Among the three proposed novel methods, VMD-CCA decomposed with 15 intrinsic mode functions (IMFs) has shown the best denoising performance for EEG signals producing an average <inline-formula> <tex-math notation="LaTeX">$\Delta SNR$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\eta $ </tex-math></inline-formula> values of 23.81 dB and 57.01&#x0025;, respectively for all 23 EEG recordings. On the other hand, for the available 16 fNIRS recordings, VMD-CCA decomposed with 10 IMFs produced an average <inline-formula> <tex-math notation="LaTeX">$\Delta SNR$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\eta $ </tex-math></inline-formula> values of 15.97 dB and 39.01&#x0025;, respectively. The results reported using the proposed methods outperform most of the existing state-of-the-art techniques.

Facial-Video-Based Physiological Signal Measurement: Recent advances and affective applications

Monitoring physiological changes [e.g., heart rate (HR), respiration, and HR variability (HRV)] is important for measuring human emotions. Physiological responses are more reliable and harder to alter compared to explicit behaviors (such as facial expressions and speech), but they require special contact sensors to obtain. Research in the last decade has shown that photoplethysmography (PPG) signals can be remotely measured (rPPG) from facial videos under ambient light, from which physiological changes can be extracted. This promising finding has attracted much interest from researchers, and the field of rPPG measurement has been growing fast. In this article, we review current progress on intelligent signal processing approaches for rPPG measurement, including earlier works on unsupervised approaches and recently proposed supervised models, benchmark data sets, and performance evaluation. We also review studies on rPPG-based affective applications and compare them with other affective computing modalities. We conclude this article by emphasizing the current main challenges and highlighting future directions.

Measurement of multimodal physiological signals for stimulation detection by wearable devices

The presence of stimuli and the consequent reactions undoubtedly reflect in experience-related changes of physiological parameters, which can be monitored by wearable devices. Generally, reactions related to the sympathetic nervous system activity are assessed through heart rate variability analysis. However, the exploitation of multimodal physiological signals provides a broader fingerprint. This study aims to identify the elicitation of acoustic stimulation through a wearable device; physiological signals, including electrodermal activity and skin temperature, were measured on a test population wearing a wrist-worn medical device. Eight machine learning algorithms were evaluated in a binary classification (presence/absence of stimuli), using 22 meaningful metrics from the collected data. The experimental results showed that Linear Regression (LR) algorithm, followed by Support Vector Machine (SVM), performed satisfactorily across all the evaluation metrics, achieving 75.00% and 72.62% of accuracy rate, respectively. Finally, the trained LR and SVM algorithms have been validated on a publicly available dataset (WESAD).

Soft Bio-Integrated Multifunctional Devices Using an Intrinsically Stretchable Conducting Nanomembrane

Soft bioelectronic systems with a unique mechanical property, namely modulus matching between human skin (or tissue) and the device, have gained widespread attention. This is because of their closed-loop strain-insensitive electrical performance ranging from application in the long-term stable measurements of physiological signals and feedback modulation to human skin (or organs). Various materials and integration/fabrication strategies such as buckled, rigid islands, and wavy designs addressed for soft bioelectronic systems require complex device fabrication with time-consuming packaging and integration processes. In this study, we developed a soft bio-integrated multifunctional device (SBMD) fabricated through the simple thermal evaporation and transfer processes. The intrinsically stretchable Au–SEBS film composed of thermally evaporated gold (Au) nanomembranes and an elastomeric substrate was applied to various functional modules that are capable of sensing the strain (up to ~300%), temperature (with a thermal sensitivity of ~0.6 Ω/°C), chemicals (at a concentration of NaCl of ~0.5 wt%), and even electrophysiological cardiac/muscle signals and showing thermal actuations (80 °C at 9 V). Specifically, such multifunctions of the SBMD were stably performed even on skin. Thus, we believe the SBMD would be a promising candidate for realizing soft bioelectronic systems.

Stretchable, Multifunctional Epidermal Sensor Patch for Surface Electromyography and Strain Measurements

The concurrent collection of surface electromyography (sEMG) and strain signals is important for many applications, such as human–machine interaction, sign language recognition, and clinical evaluation of muscle function. Nevertheless, the conventional sensor systems made of rigid, bulky components cannot provide a reliable, conformal interface for accurate, continuous measurements of the epidermal physiological signals. Herein, a skin‐interfaced, multifunctional epidermal sensor patch with characteristics of mechanical softness, large stretchability, and wearable conformability for multimodal measurements of sEMG signals and associated skin deformations from various muscle activities and joint motions is reported. The sensor patch features two pairs of stretchable sEMG electrodes and two thin, miniaturized strain sensors, which are connected by stretchable filamentary serpentine interconnects in an open‐meshed structure. Experimental and computational studies reveal the design and operation of the sensor patch, which exhibit stable and repetitive performance even under a 30% stretching strain level. Demonstrations of the sensor patch on the wrist for simple sign language recognition and on the lower back for the flexion‐relaxation phenomenon illustrate its potential for the comprehensive assessment of the muscle activities and related motions of muscle joints.

Enabling the Unconstrained Epidermal Pulse Wave Monitoring via Finger‐Touching

AbstractUnconstrained measurement of physiological signals including electrocardiograph, respiration, and temperature by sensors through incorporation into commonly used objects has sparked a notable revolution in healthcare monitoring. However, unconstrained precision epidermal pulse wave monitoring is rarely reported. Although the current flexible skin‐mounted sensors can capture pulse waves, they lack the capability to perceive tiny pulse pressure in an unconstrained manner. Herein, utilizing thin‐film materials and multilevel microstructure design, an ultrathin and flexible sensor (UFS) with the features of high flexibility, shape‐adaptability, and ultra‐broad‐range high pressure sensitivity is proposed for unconstrained precision pulse wave sensing. Given these compelling features, the UFS is mounted to the surfaces of commonly used objects and successfully detects the fingertip pulse wave even under an ultra‐broad‐range finger‐touching force. Key cardiovascular parameters are also extracted from the acquired fingertip pulse wave accurately. Furthermore, a proof‐of‐concept healthcare system, by combining the UFS and flexible devices (for example, flexible phones or E‐newspapers) is demonstrated, offering a great advancement in developing an all‐in‐one system for IoT‐based bio‐health monitoring at all times and places.

All-Fabric Piezoionic Sensor for Simultaneous Sensing of Static and Dynamic Pressures

The development of flexible and textile-based wearable pressure sensors has provided the opportunity of continuous and real time measurement of human physiological and biomechanical signals during daily activities. Pressure sensors are transducers that convert an exerted compression stress into a detectable electrical signal. Different transduction mechanisms have been introduced so far including triboelectricity, transistivity, capacitance, piezoelectricity, and piezoresistivity. Piezoresistive pressure sensors are the most widely used type due to the simplicity of their structure, and the wide range of materials that can be selected along with low-cost fabrication methods, and easy read-out system required for signal extraction.A vast majority of piezoresistive sensors developed so far are on-skin sensors developed to detect subtle pressures (1 Pa-10 kPa) for touchpads and electronic skin applications. However, to sense physiological signals such as pulse, respiration, and phonation the sensor range of detection must fall within medium range of 10 kPa to 100 kPa. As expected, for larger-scale human motion detection such as sleep posture and footwear evaluation, the sensor must be able to sense compression stresses larger than 100 kPa. This wide range of detection required by the piezoresistive pressure sensors is one of the important challenges in designing these sensors.Many of the piezoresistive sensors function based on employing the composite of conductive additives in an elastomer as an active layer. The functionality and sensitivity of these sensors are highly limited by the poor bulk mechanical properties of the elastomer in addition to unbreathability and the complications arising from the skin-sensor interface. Textile-based sensors overcome the issues regarding the elastomer sensors to a good extend. These sensors are mainly developed through coating fibers by conductive inks or intrinsically conductive polymers (ICPs). However, these sensors suffer from major drawbacks. First, the high conductivity of the conductive coatings leads to shortening in signals upon the application of a small amount of pressure. These sensors can respond either to static or dynamic pressures and once being pressed by a pressure, completely lose their sensitivity to further pressure exertions which resembles a connection/disconnection mode of performance. Second, the sensors need to be used in tight-fitting clothing to be able to capture signals which makes it quite uncomfortable and hinders the widespread adoption of the device in society.Here, we introduce an all-fabric piezoionic pressure sensor, called “PressION”, that successfully overcomes the mentioned drawbacks. We gained significant wide range of pressure detection (several kPa to larger than 100 kPa) through tunning the conductivity of the active layer by coating the fabric with an ion-conductive polysiloxane polymer. This optimal conductivity of the fabric makes the sensor capbale of simultaneously responding to both static and dynamic pressures. To overcome challenge of tight-fitting clothing, we took advantage of the fact that even with loose-fitting garments, there are still some parts of the garment which are pressed due to the exerted pressure by the body limbs over the torso or against an external surface such as bed/chair or even a subtle pressure exerted by a blanket over the body. The success of PressION was confirmed by data acquisition from different body locations (Fig1a-h). All these characteristics makes PressION an ideal candidate to be embedded in daily garments for future biomedical and psychological studies. -Figure Caption- A variety of physiological signals can be extracted by gently placing PressION on different locations of the body: (a) artery pulse from the face; (b) phonation from throat; (c) joint motion at the elbow or knee; (d) heartbeat from the chest while lying face down on a bed; (e) steps from the sensor used as an insole in footwear; and respiration from the sensor placed on the (f) back, (g) side or (h) front of the body. Figure 1

Contactless and low-burden measurements of physiological signals and comparison of those

Contactless and low-burden measurements of physiological signals and comparison of those

Safety envelope of pedestrians upon motor vehicle conflicts identified via active avoidance behaviour

Human reaction plays a key role in improved protection upon emergent traffic situations with motor vehicles. Understanding the underlying behaviour mechanisms can combine active sensing system on feature caption and passive devices on injury mitigation for automated vehicles. The study aims to identify the distance-based safety boundary (“safety envelope”) of vehicle–pedestrian conflicts via pedestrian active avoidance behaviour recorded in well-controlled, immersive virtual reality-based emergent traffic scenarios. Via physiological signal measurement and kinematics reconstruction of the complete sequence, we discovered the general perception-decision-action mechanisms under given external stimulus, and the resultant certain level of natural harm-avoidance action. Using vision as the main information source, 70% pedestrians managed to avoid the collision by adapting walking speeds and directions, consuming overall less “decision” time (0.17–0.24 s vs. 0.41 s) than the collision cases, after that, pedestrians need enough “execution” time (1.52–1.84 s) to take avoidance action. Safety envelopes were generated by combining the simultaneous interactions between the pedestrian and the vehicle. The present investigation on emergent reaction dynamics clears a way for realistic modelling of biomechanical behaviour, and preliminarily demonstrates the feasibility of incorporating in vivo pedestrian behaviour into engineering design which can facilitate improved, interactive on-board devices towards global optimal safety.

Human Mental Condition Monitoring through Measurement of Physiological Signals

Human Mental Condition Monitoring through Measurement of Physiological Signals

Early Detection of Hypotension Using a Multivariate Machine Learning Approach.

The ability to accurately detect hypotension in trauma patients at the earliest possible time is important in improving trauma outcomes. The earlier an accurate detection can be made, the more time is available to take corrective action. Currently, there is limited research on combining multiple physiological signals for an early detection of hemorrhagic shock. We studied the viability of early detection of hypotension based on multiple physiologic signals and machine learning methods. We explored proof of concept with a small (5 minutes) prediction window for application of machine learning tools and multiple physiologic signals to detecting hypotension. Multivariate physiological signals from a preexisting dataset generated by an experimental hemorrhage model were employed. These experiments were conducted previously by another research group and the data made available publicly through a web portal. This dataset is among the few publicly available which incorporate measurement of multiple physiological signals from large animals during experimental hemorrhage. The data included two hemorrhage studies involving eight sheep. Supervised machine learning experiments were conducted in order to develop deep learning (viz., long short-term memory or LSTM), ensemble learning (viz., random forest), and classical learning (viz., support vector machine or SVM) models for the identification of physiological signals that can detect whether or not overall blood loss exceeds a predefined threshold 5 minutes ahead of time. To evaluate the performance of the machine learning technologies, 3-fold cross-validation was conducted and precision (also called positive predictive value) and recall (also called sensitivity) values were compared. As a first step in this development process, 5 minutes prediction windows were utilized. The results showed that SVM and random forest outperform LSTM neural networks, likely because LSTM tends to overfit the data on small sized datasets. Random forest has the highest recall (84%) with 56% precision while SVM has 62% recall with 82% precision. Upon analyzing the feature importance, it was observed that electrocardiogram has the highest significance while arterial blood pressure has the least importance among all other signals. In this research, we explored the viability of early detection of hypotension based on multiple signals in a preexisting animal hemorrhage dataset. The results show that a multivariate approach might be more effective than univariate approaches for this detection task.

Emotion Recognition Using Explainable Genetically Optimized Fuzzy ART Ensembles

There is a growing demand for explainability in complex artificial intelligence solutions to support critical applications’ decision-making processes. Barriers to explainable processes include black-box classifiers, such as deep learning, and noisy datasets. Affect recognition involving neural networks attempts to map complex human emotions onto Arousal and Valence scales based on physiological signal measurements. Datasets collected for this purpose are inherently noisy and may contain outliers and imbalanced classes, hindering accurate classification. In our approach, these issues are addressed using Fuzzy ART (FA) for clustering data samples into more condensed memory templates, introducing stochastic resonant noise to amplify signal-to-noise ratio, and SMOTE sampling to generate synthetic minority samples. A genetic algorithm is developed for FA optimization and ensemble model selection. Clusters obtained from the resulting ensembles are then used to train an ensemble of boosted decision trees for classification and to visualize the decision-making processes. Individual features such as heart rate variability and EEG band power, as well as feature interactions between pairs of features, may contain critical information as human affect indicators. Contributions of individual features and feature interactions toward describing human affect are quantified and interpreted using Shapley additive explanation values. Three established affect recognition datasets were considered for mapping physiological features onto a binary classification of Low/High Arousal and Positive/Negative Valence. Our framework was able to achieve good generalization for both classification tasks as well as provide detailed insights into the contributions of physiological features towards describing Arousal and Valence affects.

Noncontact Physiological Measurement Using a Camera: A Technical Review and Future Directions.

Using a camera as an optical sensor to monitor physiological parameters has garnered considerable research interest in biomedical engineering in recent decades. Researchers have explored the use of a camera for monitoring a variety of physiological waveforms, together with the vital signs carried by these waveforms. Most of the obtained waveforms are related to the human respiratory and cardiovascular systems, and in addition of being indicative of overall health, they can also detect early signs of certain diseases. While using a camera for noncontact physiological signal monitoring offers the advantages of low cost and operational ease, it also has the disadvantages such as vulnerability to motion and lack of burden-free calibration solutions in some use cases. This study presents an overview of the existing camera-based methods that have been reported in recent years. It introduces the physiological principles behind these methods, signal acquisition approaches, various types of acquired signals, data processing algorithms, and application scenarios of these methods. It also discusses the technological gaps between the camera-based methods and traditional medical techniques, which are mostly contact-based. Furthermore, we present the manner in which noncontact physiological signal monitoring use has been extended, particularly over the recent years, to more day-to-day aspects of individuals' lives, so as to go beyond the more conventional use case scenarios. We also report on the development of novel approaches that facilitate easier measurement of less often monitored and recorded physiological signals. These have the potential of ushering a host of new medical and lifestyle applications. We hope this study can provide useful information to the researchers in the noncontact physiological signal measurement community.

Wearables for arrhythmia care: Challenges and future prospects

Wearables for arrhythmia care: Challenges and future prospects

Multimodal Smart Eyewear for Longitudinal Eye Movement Tracking

Summary Contextual measurements of multiple physiological signals using familiar garments or accessories can provide insights into health and behavior. Recording these signals on or near the face is challenging because most subjects are sensitive to implements placed on the face or head. Here, fabric electrodes and garment design strategies are used synergistically to afford an unobtrusive platform, Chesma, for longitudinal acquisition of physiological signals from a user's face. Chesma contains two novel fabric-based electrodes: a hydrogel electrode (tAgTrode) that retains its ionic character after multiple uses/washes, does not cause skin irritation, and can be continuously operated for upward of 8 h without losing signal integrity. Second is an ion-conductive fabric pressure sensor (Press-ION) that captures pulse waveforms. When sewn onto a lightweight eye mask, these electrodes simultaneously gather complementary electro-oculography and cardiac data, which can enable sleep quality and psychological studies and improve the accuracy and usability of virtual reality headsets.

A Rigid-Flex Wearable Health Monitoring Sensor Patch for IoT-Connected Healthcare Applications

The Internet of Things (IoT) is a new communication paradigm that can connect elements from various fields through the Internet. One of the most attractive IoT applications is in the modern healthcare area, as the traditional healthcare system has an increasing demand for social resources, including doctors, nurses, hospital beds, and health monitoring devices. In this article, the design of a compact wearable sensor patch is presented for measurements of different physiological signals, such as the electrocardiogram (ECG), photoplethysmography (PPG), and body temperature. As ECG and PPG sensors are integrated with the same device, the proposed sensor patch can be used to estimate blood pressure (BP) continuously based on the pulse arrival time (PAT) without extra wires and devices. The sensor patch consists of a center board for signal acquisition and processing, a power board for energy supply and charging batteries, and three sensors for vital signs monitoring. All the components are designed in a rigid-flex structure, which can be easily attached to the human body for remote health monitoring applications. The sensors can be detached from the center board for customized measurements of a certain physiological signal (e.g., ECG) to reduce power consumption. Experiments are conducted to validate the performance of the proposed sensor patch by comparison with a commercial reference device. With the integration of a miniaturized Bluetooth low-energy (BLE) module, the proposed sensor system can transmit physiological measurements wirelessly to a gateway. Data encryption is applied on both the sensor patch and gateways to protect data for privacy and security purposes during transmission. Both a mobile gateway (based on smartphones) and a fixed gateway (based on portable computers) are designed as the bridge between the wearable sensor system and the Internet cloud, where health data can be stored and further analyzed. The experimental results demonstrate the feasibility of the overall platform for IoT-connected healthcare applications.