Multiple Physiological Signals Research Articles

LDSG-Net: an efficient lightweight convolutional neural network for acute hypotensive episode prediction during ICU hospitalization.

Acute hypotension episode (AHE) is one of the most critical complications in intensive care unit (ICU). A timely and precise AHE prediction system can provide clinicians with sufficient time to respond with proper therapeutic measures, playing a crucial role in saving patients' lives. Recent studies have focused on utilizing more complex models to improve predictive performance. However, these models are not suitable for clinical application due to limited computing resources for bedside monitors. To address this challenge, we propose an efficient lightweight dilated shuffle group network (LDSG-Net). It effectively incorporates shuffling operations into grouped convolutions on the channel and dilated convolutions on the temporal dimension, enhancing global and local feature extraction while reducing computational load. Our benchmarking experiments on the MIMIC-III and VitalDB datasets, comprising 6,036 samples from 1,304 patients and 2,958 samples from 1,047 patients, respectively, demonstrate that our model outperforms other state-of-the-art lightweight CNNs in terms of balancing parameters and computational complexity. Additionally, we discovered that the utilization of multiple physiological signals significantly improves the performance of AHE prediction. External validation on the MIMIC-IV dataset confirmed our findings, with prediction accuracy for AHE 5 minutes prior reaching 93.04% and 92.04% on the MIMIC-III and VitalDB datasets, respectively, and 89.47% in external verification. Our study demonstrates the potential of lightweight CNN architectures in clinical applications, providing a promising solution for real-time AHE prediction under resource constraints in ICU settings, thereby marking a significant step forward in improving patient care.

An Overview of Approaches and Methods for the Cognitive Workload Estimation in Human–Machine Interaction Scenarios through Wearables Sensors

Background: Human-Machine Interaction (HMI) has been an important field of research in recent years, since machines will continue to be embedded in many human actvities in several contexts, such as industry and healthcare. Monitoring in an ecological mannerthe cognitive workload (CW) of users, who interact with machines, is crucial to assess their level of engagement in activities and the required effort, with the goal of preventing stressful circumstances. This study provides a comprehensive analysis of the assessment of CW using wearable sensors in HMI. Methods: this narrative review explores several techniques and procedures for collecting physiological data through wearable sensors with the possibility to integrate these multiple physiological signals, providing a multimodal monitoring of the individuals’CW. Finally, it focuses on the impact of artificial intelligence methods in the physiological signals data analysis to provide models of the CW to be exploited in HMI. Results: the review provided a comprehensive evaluation of the wearables, physiological signals, and methods of data analysis for CW evaluation in HMI. Conclusion: the literature highlighted the feasibility of employing wearable sensors to collect physiological signals for an ecological CW monitoring in HMI scenarios. However, challenges remain in standardizing these measures across different populations and contexts.

Intrinsically stretchable jellyfish-like gold nanowires film as multifunctional wearable chemical and physical sensors

We report on an intrinsically stretchable jellyfish-like gold nanowires film as a multifunctional wearable platform that can detect both chemical and physical biometrics. The film inherently exhibits multifunctionality, serving directly as an electrochemical electrode, strain sensor, temperature sensor, pressure sensor, and noncontact humidity sensor without the need of any additional materials. Our results highlight the excellent electrochemical properties, evidenced by a large electrochemical active surface area (EASA) of approximately 22.39 cm2 and the ability to withstand significant strain values up to 70 %. Leveraging these attributes, a stretchable electrochemical glucose biosensor was further designed with high sensitivity (45.40μA·mM−1·cm−2) and low limit of detection (10 μM). In addition, its sensing applications in strain, temperature, pressure and noncontact humidity were systematically explored. Impressively, the gold film is capable of monitoring multiple human motions and physiological signals, including joints bending, pulse motions, throat motions (e.g., drinking, speaking, swallowing and pronunciation recognition) and breathing. Furthermore, the film exhibited great versatility by integrating with 3D printing hydrogels, enabling customized patterning. The results demonstrate that our gold nanowires coating approach represent a new route to design multimodal soft wearable platforms for applications in personalized medicine and next-generation healthcare systems.

HybMED: A Hybrid Neural Network Training Processor with Multi-Sparsity Exploitation for Internet of Medical Things.

Cloud-based training and edge-based inference modes for Artificial Intelligence of Medical Things (AIoMT) applications suffer from accuracy degradation due to physiological signal variations among patients. On-chip learning can overcome this issue by online adaptation of neural network parameters for user-specific tasks. However, existing on-chip learning processors have limitations in terms of versatility, resource utilization, and energy efficiency. We propose HybMED, which is a novel neural signal processor that supports on-chip hybrid neural network training using a composite direct feedback alignment-based paradigm. HybMED is suitable for general-purpose health monitoring AIoMT devices. It improves resource utilization and area efficiency by the reconfigurable homogeneous core with heterogeneous data flow and enhances energy efficiency by exploiting sparsity at different granularities. The chip was fabricated by TSMC 40nm process and tested in multiple physiological signal processing tasks, demonstrating an average improvement in accuracy of 41.16% following online few-shot learning. The chip demonstrates an area efficiency of 1.17 GOPS/mm2 and an energy efficiency of 1.58 TOPS/W. Compared to the previous state-of-the-art physiological signal processors with on-chip learning, the chip achieves a 65× improvement in area efficiency and 1.48× improvement in energy efficiency, respectively.

Assessment of dynamic cognitive performance under different thermal conditions by multiple physiological signals

A method to evaluate cognitive performance under different thermal conditions by multiple physiological signals is proposed. Heart rate (HR), heart rate variability (RMSSD), skin conductance level (SCL), skin conductance response (SCR) magnitude and SCR total frequency were measured by a multichannel electrophysiological signal monitor, which were interpreted as thermal stress indicators that can characterize cognitive performance of subjects during cognitive stimulation tasks. In the experiment, the indoor air temperature is the independent variable, ranging from 18 °C to 33 °C, with a step length of three degrees. Other indoor environmental quality (IEQ) parameters are controlled variables. In each experiment, subjects subjectively voted on thermal sensation and cognitive load, and performed cognitive stimulation tasks to evaluate cognitive performance. The study found that different physiological indicators have different “thermosensitivity” zones, indicating variations in thermal attributes. Moreover, the influence mechanism of thermal conditions on the effectiveness levels of three different cognitive functions, namely identification, decision and action, were found to be diverse. Additionally, this paper utilized machine learning algorithms to construct fatigue loss function and residual interference function for cognitive performance, effectively correcting cognitive performance errors caused by fatigue and task superimposition. The feasibility of predicting cognitive performance and thermal comfort by multiple physiological indicators is validated in this study.

AFLEMP: Attention-based Federated Learning for Emotion recognition using Multi-modal Physiological data

Automated emotion recognition systems utilizing physiological signals are essential for affective computing and intelligent interaction. Combining the multiple physiological signals is more precise and effective in accurately assessing a person’s emotional state. These automated emotion recognition systems using conventional machine learning techniques require complete access to the physiological data for emotion state classification, compromising sensitive data privacy. Federated Learning (FL) resolves this issue by preserving the user’s privacy and sensitive physiological data while recognizing emotions. However, existing FL methods have limitations in handling data heterogeneity in the physiological data and do not measure communication efficiency and scalability. In response to these challenges, this paper proposes a unique novel framework called AFLEMP (Attention-based Federated Learning for Emotion recognition using Multi-modal Physiological data) integrating attention mechanism-based Transformer with an Artificial Neural Network (ANN) model. The framework reduces two types of data heterogeneity: (1) Variation Heterogeneity (VH) in multi-modal EEG, GSR, and ECG physiological signal data using attention mechanisms and (2) Imbalanced Data Heterogeneity (IDH) in the FL environment using scaled weighted federated averaging. This paper validates the proposed AFLEMP framework on two publicly available emotion datasets, AMIGOS and DREAMER, achieving an average accuracy of 88.30% and 84.10%, respectively. The proposed AFLEMP framework proves robust, scalable, and efficient in communication. AFLEMP is the first FL framework to propose for emotion recognition using multi-modal physiological signals while reducing data heterogeneity and outperforming existing FL methods.

ECG-based emotion recognition using random convolutional kernel method

The recognition of human emotions based on physiological signals has garnered significant attention from various fields such as human–computer interaction, cognitive-behavioral science, and the treatment of emotion-related diseases. Using the easily accessible ECG signal for emotion recognition presents a viable approach. However, method relying on hand-crafted features necessitate task-specific domain knowledge and consume substantial time to design suitable features, and studies employing deep learning technologies incur greater computational complexity and extended training durations. To address these problems, we proposed an emotion recognition method that employs random convolutional kernels on ECG signals. Following ECG signal preprocessing, a large number of random convolutional kernels were used to transform the signals, and the resulting features are utilized to classify the various emotion states. In addition to accurately and automatically extracting the multi-scale features from ECG signals, combining the random convolutional kernels with ECG signal alone also reduces the computational complexity and training time compared to methods using multiple physiological signals or deep neural networks. Validation on three publicly available datasets revealed that the proposed method achieved recognition accuracies of 91.5%, 96% and 92.5% for valence, arousal and dominance in AMIGOS dataset, and 95.1%, 98.8% and 90.2% for same labels in DREAMER dataset. The accuracy for valence, arousal and affective states in WESAD dataset are 94.5%, 91.8% and 91.8%. The results show that the proposed method outperforms previous researches in multi-class emotion recognition, thus offering a new and promising approach for swift and precise emotion recognition.

Securing Pacemakers using Runtime Monitors over Physiological Signals

Wearable and implantable medical devices (IMDs) are increasingly deployed to diagnose, monitor, and provide therapy for critical medical conditions. Such medical devices are safety-critical cyber-physical systems (CPSs). These systems support wireless features introducing potential security vulnerabilities. Although these devices undergo rigorous safety certification processes, runtime security attacks are inevitable. Based on published literature, IMDs such as pacemakers and insulin infusion systems can be remotely controlled to inject deadly electric shocks and excess insulin, posing a threat to a patient’s life. While prior works based on formal methods have been proposed to detect potential attack vectors using different forms of static analysis, these have limitations in preventing attacks at runtime. This paper discusses a formal framework for detecting cyber-physical attacks on a pacemaker by monitoring its security policies at runtime. We propose a wearable device that senses the Electrocardiogram (ECG) and Photoplethysmogram (PPG) of the body to detect attacks in a pacemaker. To facilitate the design of this device, we map the security policies of a pacemaker w.r.t ECG and PPG, paving the way for designing formal verification monitors for pacemakers for the first time using multiple physiological signals. The proposed monitoring framework allows the synthesis of parallel monitors from a given set of desired security policies, where all the monitors execute concurrently and generate an alarm to the user in the case of policy violation. Our implementation and the performance evaluation results demonstrate the technical feasibility of designing such a wearable device for attack detection in pacemakers. This device is separate from the pacemaker, ensuring no need for re-certification of pacemakers. Our approach is amenable to the application of security patches when new attack vectors are detected, making the approach ideal for runtime monitoring of medical CPSs.

PainMeter: Automatic Assessment of Pain Intensity Levels From Multiple Physiological Signals Using Machine Learning

PainMeter: Automatic Assessment of Pain Intensity Levels From Multiple Physiological Signals Using Machine Learning

Quantifying the Suitability of Biosignals Acquired During Surgery for Multimodal Analysis.

Goal: Recently, large datasets of biosignals acquired during surgery became available. As they offer multiple physiological signals measured in parallel, multimodal analysis - which involves their joint analysis - can be conducted and could provide deeper insights than unimodal analysis based on a single signal. However, it is unclear what percentage of intraoperatively acquired data is suitable for multimodal analysis. Due to the large amount of data, manual inspection and labelling into suitable and unsuitable segments are not feasible. Nevertheless, multimodal analysis is performed successfully in sleep studies since many years as their signals have proven suitable. Hence, this study evaluates the suitability to perform multimodal analysis on a surgery dataset (VitalDB) using a multi-center sleep dataset (SIESTA) as reference. Methods: We applied widely known algorithms entitled "signal quality indicators" to the common biosignals in both datasets, namely electrocardiography, electroencephalography, and respiratory signals split in segments of 10 s duration. As there are no multimodal methods available, we used only unimodal signal quality indicators. In case, all three signals were determined as being adequate by the indicators, we assumed that the whole signal segment was suitable for multimodal analysis. Results: 82% of SIESTA and 72% of VitalDB are suitable for multimodal analysis. Unsuitable signal segments exhibit constant or physiologically unreasonable values. Histogram examination indicated similar signal quality distributions between the datasets, albeit with potential statistical biases due to different measurement setups. Conclusions: The majority of data within VitalDB is suitable for multimodal analysis.

Electrospinning fibrous membrane matrix ionic capacitive sensor for pressure value and distribution detection

Considering the practical applications of capacitive sensors, electrical sensing fibrous membranes should own high mechanical strength, elasticity, sensitivity, thermal diffusivity and biocompatibility, listing the basic design and preparation demands via electrostatic spinning. Polyvinylidene fluoride (PVDF) fibrous membrane matrix pressure ionic capacitive sensors were prepared via electrostatic spinning. The addition and dissolution of polyvinylpyrrolidone (PVP) particles realized porous structures of PVDF fibrous membrane, providing silver particles (Ag) dependence matrix for conductive Ag-porous PVDF fibrous membrane of electrode layer. The PVDF matrix capacitive sensor was composed of sandwiching dielectric layer of PVDF fibrous membrane with ionic liquid (IL-PVDF) between two electrode layers. The construction of electrical double layers at the interfaces between electrode layer and dielectric layer significantly enhanced capacitance sensing properties. After encapsulating by pure PVDF fibrous membranes, PVDF fibrous membrane matrix ionic capacitive sensor exhibited excellent stability, sensitivity and durability in pressure loading range of 0–8 kPa for 5000 cyclic experiments，which was suitable for real-time detection of multiple physiological bending signals. Based on pressure sensing array construction, the values and distribution of static and dynamic loads were reflected accurately and intuitively, providing feasible methods for practical applications of functional fiber matrix sensors in fields of healthcare and human-machine interaction.

Emotion recognition from multiple physiological signals using intra- and inter-modality attention fusion network

Recently, many studies have shown that integrating multiple modalities can more accurately and robustly identify human emotions compared with a single modality. However, how to fully utilize the heterogeneity and correlation of multiple modalities to improve emotion recognition performance remains a challenge. Given this, we propose a novel multimodal fusion method that considers both heterogeneity and correlation simultaneously, and realizes an end-to-end multimodal emotion recognition model using intra- and inter-modality attention fusion network. Firstly, the dual-stream feature extractor is designed to extract emotional features from raw EEG signals and peripheral physiological signals (PPS) separately. Then, the inter-modality fusion module is released to capture the correlation and complementarity of the two features. Meanwhile, the intra-modality encoding module is added to preserve the heterogeneity information of each feature. Finally, their joint loss function is applied to train the model. The proposed model has been extensively validated on DEAP and DREAMER multimodal datasets, with an average accuracy of 97.97%/98.02% on valence/arousal dimension for DEAP dataset and 99.47%/99.47% on valence/arousal dimension for DREAMER dataset, which outperforms the state-of-the-art multimodal methods. Additionally, we also explore the best combination of EEG and each peripheral physiological signal (e.g., EOG, EMG signals), which can assist in the development of a low-cost and more effective multimodal emotion analysis system. The proposed method can provide new insights into multimodal fusion research for emotion recognition.

A comprehensive study on the efficacy of a wearable sleep aid device featuring closed-loop real-time acoustic stimulation

Difficulty falling asleep is one of the typical insomnia symptoms. However, intervention therapies available nowadays, ranging from pharmaceutical to hi-tech tailored solutions, remain ineffective due to their lack of precise real-time sleep tracking, in-time feedback on the therapies, and an ability to keep people asleep during the night. This paper aims to enhance the efficacy of such an intervention by proposing a novel sleep aid system that can sense multiple physiological signals continuously and simultaneously control auditory stimulation to evoke appropriate brain responses for fast sleep promotion. The system, a lightweight, comfortable, and user-friendly headband, employs a comprehensive set of algorithms and dedicated own-designed audio stimuli. Compared to the gold-standard device in 883 sleep studies on 377 subjects, the proposed system achieves (1) a strong correlation (0.89 ± 0.03) between the physiological signals acquired by ours and those from the gold-standard PSG, (2) an 87.8% agreement on automatic sleep scoring with the consensus scored by sleep technicians, and (3) a successful non-pharmacological real-time stimulation to shorten the duration of sleep falling by 24.1 min. Conclusively, our solution exceeds existing ones in promoting fast falling asleep, tracking sleep state accurately, and achieving high social acceptance through a reliable large-scale evaluation.

Multivariate phase space reconstruction and Riemannian manifold for sleep stage classification

Background and ObjectiveSleep was highly imperative in human daily life. However, an increasing number of people were undergoing sleep deprivation and sleep disorders. Sleep stage classification became a highly essential process in sleep scoring. Nevertheless, the visual scoring of arousals during sleep routinely performed by sleep specialists was a challenging exercise. MethodsThis paper introduced a novel approach for sleep stage classification using covariance feature matrix architecture with multivariate phase space reconstruction (MPSR). The goal was to capture the geometric properties and the hidden dynamic characteristics of multiple physiological signals. The covariance matrices constructed through the MPSR approach were considered as symmetric and positive definite (SPD) matrices, forming a Riemannian manifold space. These SPD matrices in the Riemannian manifold were mapped to the matrices in the tangent space, allowing them to be vectorized and treated as feature vectors in Euclidean space. Finally, an ensemble learning classifier was applied to perform various sleep stage tasks. ResultsOur proposed method was evaluated on three benchmark datasets to assess its effectiveness and robustness. For the tasks of both Five class and Six class sleep stages, the proposed approach in ten-fold cross validation achieved high accuracy of 93.57% and 92.56% for the Sleep EDF dataset, 86.36% and 84.18% for the DREAMS Subjects dataset, as well as 88.93% and 88.42% for the Sleep EDF Expanded dataset (95 subjects), respectively. In leave-one-subject-out cross validation, our proposed approach for the tasks of both Five class and Six class sleep stages yielded an accuracy of 84.46% and 80.73% for the Sleep EDF dataset, 82.50% and 79.51% for the DREAMS Subjects dataset, as well as 93.25% and 92.06% for the Sleep EDF Expanded-20 dataset (20 subjects), respectively. ConclusionCompared to the traditional sample covariance matrix (SCM), the covariance feature matrix using the MPSR method successfully captured the distinction of spatial information among various sleep stages. Moreover, our proposed method obtained good performance without requiring computationally large artifact suppression or a long signal decomposition process.

(Invited) smart Wearable Electronics for Chronic Disease Management

60% of Americans live with at least one chronic disease. These diseases and associated comorbidities are now the leading causes of death in the United States. Effective management of complex chronic diseases requires body-wide, long-term, accurate, and continuous monitoring of multiple physiological signals from wearable devices to precisely determine the pathological state. These wearable physiological signal monitoring can dramatically reduce the demand of physician visiting and increase patients’ engagement and treatment adherence rate. Although wearable bioelectronic devices have shown huge potential in chronic disease management, existing technology still mostly relies on rigid chip components, where the interface between chips and skin/tissue has been a bottleneck that limits the system performance (noise, robustness etc). To address these challenges, my research has been involved in the exploration of rational system design concepts, material and device fabrication innovation, and tailored algorithms to enable smart wearable electronics that targets next-generation chronic disease management.Here I would like to discuss two of my developed technology platforms to elaborate the concept of smart wearables. First, I will describe a body area sensor network technology platform. In the system design concept side, this technology utilizes a novel wireless hybridization strategy and an unconventional detuned RFID working regime that provide an ideal skin interface. In the material/device side, this system integrates several skin-conforming polymer-based soft devices (ring oscillators, RF diodes, antennas, transistors etc.) as an integrated soft multimode sensing sticker. (1,2) Next, I will describe a wireless closed-loop smart bandage that can accelerate and monitor the wound healing process. (3) Such smart bandage also integrates new design concepts (wireless, battery-free design that combines both sensing and stimulation), novel materials (low-impedance, high-toughness, and tunable adhesion hydrogel as skin interface) and closed-loop control algorithms. Overall, the developed technology platform could assess multiple health outcomes and treatment responses to various chronic diseases. Ultimately, this technology will help to reduce chronic disease burden, lower medical costs, and provide a better quality of life for patients.

MetaFERA: a meta-framework for creating emotion recognition frameworks for physiological signals

Recognizing emotions from physiological signals has proven to be important in various scenarios. To assist in developing emotion recognizers, software frameworks and toolboxes have emerged, offering ready-to-use components. However,these have limitations regarding the type of physiological signals supported, the recognition steps covered, or the acquisition of multiple physiological signals. This paper presents metaFERA, an architectural meta-framework for creating software frameworks for end-to-end emotion recognition from physiological signals. The modularity and flexibility of the meta-framework and the resulting frameworks allow the fast prototyping of emotion recognition systems and experiments to test and validate new algorithms. To that end, metaFERA offers: (i) a set of pre-configured blocks to which we can add behavior to create framework components; (ii) an easy way to add behavior to the pre-configured blocks; (iii) a channel-based communication mechanism that transparently and efficiently supports the exchange of information between components; (iv) a simple and easy way to use and link components from a resulting framework to create applications. Additionally, we provide a set of Web services, already configured, to make the resulting recognition systems available as a service. To validate metaFERA, we created a framework for Electrodermal Activity, an emotion recognizer to identify high/low arousal using the aforementioned framework, and a layer to offer the recognizer as a service.

A Feature Fusion Method for Driving Fatigue of Shield Machine Drivers Based on Multiple Physiological Signals and Auto-Encoder

The driving fatigue state of shield machine drivers directly affects the safe operation and tunneling efficiency of shield machines during metro construction. To cope with the problem that it is challenging to simulate the working conditions and operation process of shield machine drivers using driving simulation platforms and that the existing fatigue feature fusion methods usually show low recognition accuracy, shield machine drivers at Shenyang metro line 4 in China were taken as the research subjects, and a multi-modal physiological feature fusion method based on an L2-regularized stacked auto-encoder was designed. First, the ErgoLAB cloud platform was used to extract the combined energy feature (E), the reaction time, the HRV (heart rate variability) time-domain SDNN (standard deviation of normal-to-normal intervals) index, the HRV frequency-domain LF/HF (energy ratio of low frequency to high frequency) index and the pupil diameter index from EEG (electroencephalogram) signals, skin signals, pulse signals and eye movement data, respectively. Second, the physiological signal characteristics were extracted based on the WPT (wavelet packet transform) method and time–frequency analysis. Then, a method for driving fatigue feature fusion based on an auto-encoder was designed aiming at the characteristics of the L2-regularization method to solve the over-fitting problem of small sample data sets in the process of model training. The optimal hyper-parameters of the model were verified with the experimental method of the control variable, which reduces the loss of multi-modal feature data in compression fusion and the information loss rate of the fused index. The results show that the method proposed outperforms its competitors in recognition accuracy and can effectively reduce the loss rate of deep features in existing decision-making-level fusion.

0537 Incident Hypertension Prediction in Obstructive Sleep Apnea using Machine Learning

Abstract Introduction Obstructive sleep apnea (OSA) is associated with hypertension due to intermittent hypoxia and sleep fragmentation. Due to the complex pathogenesis of hypertension, it is difficult to predict incident hypertension associated with OSA. A Machine Learning (ML) model to predict incident hypertension identified up to five years after the diagnosis of OSA by polysomnography developed. Methods Polysomnography provides time-series data on multiple physiological signals. We used the sleep heart health study (SHHS) cohort, where 4,797 participants had OSA. After excluding participants with pre-existing hypertension at baseline, the sample size was 2,652. 1,814 participants with follow-up data at 5 years were included (911/1,814, 50% with incident hypertension). In addition to clinical data (i.e. age and race), features extracted from polysomnography (heart rate variability, HRV calculated based on the electrocardiography R-R interval), electroencephalography delta power, statistical information (i.e., mean and standard deviation of signals), and heart rate periodicity functions fed to support vector machine (SVM) ML model to train and validate. The polysomnography features were calculated over the 30-second epochs identified based on respiratory events and EEG arousal and respiratory events annotation, and their corresponding parts in other signals based on sampling frequency. Technical artifacts in oxygen saturation and ECG were reconstructed with the interpolation method and removed from the signal respectively. The SVM is a robust ML method trained in an iterative fashion to find the global optimum. In comparison to the Deep Neural Network (DNN) approaches, SVMs results are interpretable. Each polysomnography signal and its corresponding features were trained on a separate SVM, followed by a fusion of the SVM results. The final results were fused by voting of individual SVM results. Results The SVM ML model thus far has achieved a test accuracy (area under the curve, AUC) of 66.06%, sensitivity 63.21%, and specificity 68.9%. Conclusion This proof-of-concept study suggests that supervised ML models, such as the SVM, may be useful in predicting incident hypertension associated with OSA. Further research is required regarding optimal input features to boost the accuracy, followed by external validation of the model in additional OSA cohorts. Support (if any) Research support 1R56HL157182, NIH/NHLBI

Emotion recognition based on multiple physiological signals

Physiological signals can more realistically reflect human emotional states. To overcome the limitations imposed in single-modal emotion recognition, emotion recognition of multimodal physiological signals has received increasingly widespread attention. However, the original fusion models usually ignore the different distributions of multiple signals and how to capture complementary features from multimodal information effectively. This paper proposes an effective classification model for multimodal physiological signals to address the above issues based on modeling the heterogeneity and correlation between multimodal signals. First, differential entropy features are extracted from Electroencephalography (EEG) signals and peripheral physiological signals (PPS) such as Electrocardiographic (ECG) signals, Electromyographic (EMG) signals, and other physiological signals. Then, according to the different distributions and frequency characteristics of the acquired signals, the EEG signal features are made into a three-dimensional feature map and input to the neural network to extract the frequency spatial dimension features. Further temporal features are extracted from the peripheral physiological signals using a long and short-term memory network. Finally, the EEG and peripheral physiological signal features were fused and input to a multimodal long and short-term memory network to extract the association between different modalities and perform classification. The experiments were conducted on the benchmark DEAP dataset, and the results showed that the classification accuracy of the proposed model in this paper was 95.89% and 94.99% in the arousal dimension and the valence dimension, respectively, which were 2.77% and 3.11% higher compared to the unimodal EEG model, respectively. This paper also analyzed the effects of different peripheral physiological signals on emotion recognition.

A Multifunctional Bioelectronic Device with Switchable Rigidity and Reconfigurable Shapes for Comprehensive Diagnosis

AbstractSoft bioelectronic devices have potential in medical applications, especially comprehensive diagnoses that rely on immovable equipment to acquire multiple physiological signals. Challenges for bioelectronic devices lie in the opposite modulus demand for multifunctional integration and biocompatibility, and shape compatibility for reducing implantation trauma and covering target tissue. Here, this work reports a multifunctional bioelectronic device endowed with switchable rigidity and reconfigurable shapes by a fast thermal response shape memory polymer substrate. The customizable substrate has a melting temperature around body temperature (≈38 °C) and an ≈100 times modulus drop (from ≈100 MPa to ≈700 kPa). The switchable rigidity realizes stacked layers fabrication in the rigid state and self‐adaptive contact in the soft state. Shape reconfiguration allows the device to be implanted through small incisions and recover in limited spaces to envelop biosurfaces. Multiple physical, biochemical, and electrophysiological signal sensing functions are integrated in the device, realized by temperature and pH electrodes with linear, stable, and fast responses and a low‐impedance potential electrode. For some epilepsy and pericardial effusion types that a single physiological parameter fails to distinguish, animal experiments demonstrate efficiency in their comprehensive diagnosis with electrocorticogram assisted by brain temperature and electrocardiogram assisted by pericardial fluid pH, respectively.