Unseen Subjects Research Articles

Unrevealing Reliable Cortical Parcellation of Individual Brains Using Resting-State Functional Magnetic Resonance Imaging and Masked Graph Convolutions.

Brain parcellation helps to understand the structural and functional organization of the cerebral cortex. Resting-state functional magnetic resonance imaging (fMRI) and connectivity analysis provide useful information to delineate individual brain parcels in vivo. We proposed an individualized cortical parcellation based on graph neural networks (GNN) to learn the reliable functional characteristics of each brain parcel on a large fMRI dataset and to infer the areal probability of each vertex on unseen subjects. A subject-specific confidence mask was implemented in the GNN model to account for the tradeoff between the topographic alignment across subjects and functional homogeneity of brain parcels on individual brains. The individualized brain parcellation achieved better functional homogeneity at rest and during cognitive tasks compared with the group-registered atlas (p-values < 0.05). In addition, highly reliable and replicable parcellation maps were generated on multiple sessions of the same subject (intrasubject similarity = 0.89), while notable variations in the topographic organization were captured across subjects (intersubject similarity = 0.81). Moreover, the intersubject variability of brain parcellation indicated large variations in the association cortices while keeping a stable parcellation on the primary cortex. Such topographic variability was strongly associated with the functional connectivity variability, significantly predicted cognitive behaviors, and generally followed the myelination, cytoarchitecture, and functional organization of the human brain. This study provides new avenues to the precise individualized mapping of the cortical areas through deep learning and shows high potentials in the personalized localization diagnosis and treatment of neurological disorders.

Automated Assessment of Cardiovascular Sufficiency Using Non-Invasive Physiological Data.

For fluid resuscitation of critically ill individuals to be effective, it must be well calibrated in terms of timing and dosages of treatments. In current practice, the cardiovascular sufficiency of patients during fluid resuscitation is determined using primarily invasively measured vital signs, including Arterial Pressure and Mixed Venous Oxygen Saturation (SvO2), which may not be available in outside-of-hospital settings, particularly in the field when treating subjects injured in traffic accidents or wounded in combat where only non-invasive monitoring is available to drive care. In this paper, we propose (1) a Machine Learning (ML) approach to estimate the sufficiency utilizing features extracted from non-invasive vital signs and (2) a novel framework to address the detrimental impact of inter-patient diversity on the ability of ML models to generalize well to unseen subjects. Through comprehensive evaluation on the physiological data collected in laboratory animal experiments, we demonstrate that the proposed approaches can achieve competitive performance on new patients using only non-invasive measurements. These characteristics enable effective monitoring of fluid resuscitation in real-world acute settings with limited monitoring resources and can help facilitate broader adoption of ML in this important subfield of healthcare.

A View Independent Classification Framework for Yoga Postures.

Yoga is a globally acclaimed and widely recommended practice for a healthy living. Maintaining correct posture while performing a Yogasana is of utmost importance. In this work, we employ transfer learning from human pose estimation models for extracting 136 key-points spread all over the body to train a random forest classifier which is used for estimation of the Yogasanas. The results are evaluated on an in-house collected extensive yoga video database of 51 subjects recorded from four different camera angles. We use a three step scheme for evaluating the generalizability of a Yoga classifier by testing it on (1) unseen frames, (2) unseen subjects, and (3) unseen camera angles. We argue that for most of the applications, validation accuracies on unseen subjects and unseen camera angles would be most important. We empirically analyze over three public datasets, the advantage of transfer learning and the possibilities of target leakage. We further demonstrate that the classification accuracies critically depend on the cross validation method employed and can often be misleading. To promote further research, we have made key-points dataset and code publicly available.Supplementary InformationThe online version contains supplementary material available at 10.1007/s42979-022-01376-7.

Risk prediction of prodromal Alzheimer’s disease using deep learning of MRI scans

AbstractBackgroundDeep learning approaches for classification have the advantage of automatically learning the relevant features but require large amount of data for training. We test the efficacy of deep learning approach for prediction of risk factor for prodromal Alzheimer’s disease from whole brain MRI scans segmented into grey/white matter and CSF images.MethodLongitudinal MPRAGE images were acquired from 146 participants in six month intervals over a period of two years as part of PharmaCog Consortium. Whole brain structural scans were segmented using FreeSurfer pipeline into grey matter (GM), white matter (WM) and cerebrospinal fluid (CSF) images. We used a convolutional neural network architecture with 11 layers to classify individuals with high and low risk based on 3‐dimensional MRI data. The model consisted of 3 convolution layers that performed slice wise convolutions in 2‐dimensions, 3 batch normalization layers, 1 max pooling layer, 1 flatten layer, 2 dense layers and 1 rectified linear unit (ReLU) activation layer. Hyperparameter tuning for the model was done using various combinations of kernel size ([2,3,4]), strides ([1,2,3]), batch size ([5,10,15,20]), epochs ([10,15,20]) and number of neurons in the dense layer ([16,32,64]) to identify the optimized parameters for the model.ResultOut of the 146 subjects available, we chose 97 subjects randomly. The cross‐sectional MRI data at various timestamps was considered for training the model. The rest of the data was considered as the test set. After this separation, we had training size and test size of 436 and 168 images respectively. The training and test sets did not have any common subjects and thus provided relevant results for subject wise classification for unseen subjects.We trained and tested our model with all possible combinations of segmented images separately (GM, WM, CSF, GM+WM, WM+CSF, GM+CSF, GM+WM+CSF). Out of all the combinations, we attained best results using Grey mask (GM) data, with training and test accuracies of 93.62% and 72.02% respectively.ConclusionWe presented a deep learning framework for risk prediction of cross‐sectional whole brain MRI data for unseen subjects. This work combined with classification approaches on longitudinal data can prove effective in early diagnosis of Alzheimer’s disease.

Predicting speech intelligibility from EEG in a non-linear classification paradigm * *Submitted for review on 7 June 2021. The research conducted in this paper is funded by KU Leuven Special Research Fund C24/18/099 (C2 project to Tom Francart and Hugo Van hamme), by a PhD Grant (1S89620N) of the Research Foundation Flanders (FWO) and from the European Research Council (ERC)

Objective. Currently, only behavioral speech understanding tests are available, which require active participation of the person being tested. As this is infeasible for certain populations, an objective measure of speech intelligibility is required. Recently, brain imaging data has been used to establish a relationship between stimulus and brain response. Linear models have been successfully linked to speech intelligibility but require per-subject training. We present a deep-learning-based model incorporating dilated convolutions that operates in a match/mismatch paradigm. The accuracy of the model’s match/mismatch predictions can be used as a proxy for speech intelligibility without subject-specific (re)training. Approach. We evaluated the performance of the model as a function of input segment length, electroencephalography (EEG) frequency band and receptive field size while comparing it to multiple baseline models. Next, we evaluated performance on held-out data and finetuning. Finally, we established a link between the accuracy of our model and the state-of-the-art behavioral MATRIX test. Main results. The dilated convolutional model significantly outperformed the baseline models for every input segment length, for all EEG frequency bands except the delta and theta band, and receptive field sizes between 250 and 500 ms. Additionally, finetuning significantly increased the accuracy on a held-out dataset. Finally, a significant correlation (r = 0.59, p = 0.0154) was found between the speech reception threshold (SRT) estimated using the behavioral MATRIX test and our objective method. Significance. Our method is the first to predict the SRT from EEG for unseen subjects, contributing to objective measures of speech intelligibility.

Calibration free meta learning based approach for subject independent EEG emotion recognition

Brain Computer Interfaces (BCI) detect changes in the electrical activity of brain which could be applied in use-cases like environmental control, neuro-rehabilitation etc. Prior to the actual usage, the subject has to undergo a lengthy calibration phase and hence prohibits an optimal plug-and-play experience. To quantify the minimum number of samples required for calibration, we propose a few-shot adaptation to the task of recognizing emotion from Electroencephalography (EEG) signals, without requiring any fine-tuning of the pre-trained classification models for every user. Our experiments illustrate the usefulness of various sampling strategies based on the presence or absence of subject dependent and subject independent reference samples during training. In comparison with the existing state-of-the-art model, which is trained in a supervised manner, our approach with only 20 reference samples from subjects under consideration (unseen during training) shows an absolute improvement of 8.56% and 7.53% in accuracy on emotion classification in valence and arousal space, respectively, without any re-training using the samples from the unseen subjects. Moreover, when tested in a zero calibration setup (when reference samples are taken from subjects other than the subject under consideration), our system improves the accuracy over the supervised model by 2.02% and 0.61% for emotion classification in valence and arousal space, respectively.

CARL: a running recognition algorithm for free-living accelerometer data

Wearable accelerometers hold great promise for physical activity epidemiology and sports biomechanics. However, identifying and extracting data from specific physical activities, such as running, remains challenging. Objective. To develop and validate an algorithm to identify bouts of running in raw, free-living accelerometer data from devices worn at the wrist or torso (waist, hip, chest). Approach. The CARL (continuous amplitude running logistic) classifier identifies acceleration data with amplitude and frequency characteristics consistent with running. The CARL classifier was trained on data from 31 adults wearing accelerometers on the waist and wrist, then validated on free-living data from 30 new, unseen subjects plus 166 subjects from previously-published datasets using different devices, wear locations, and sample frequencies. Main results. On free-living data, the CARL classifier achieved mean accuracy (F 1 score) of 0.984 (95% confidence interval 0.962–0.996) for data from the waist and 0.994 (95% CI 0.991–0.996) for data from the wrist. In previously-published datasets, the CARL classifier identified running with mean accuracy (F 1 score) of 0.861 (95% CI 0.836–0.884) for data from the chest, 0.911 (95% CI 0.884–0.937) for data from the hip, 0.916 (95% CI 0.877–0.948) for data from the waist, and 0.870 (95% CI 0.834–0.903) for data from the wrist. Misclassification primarily occurred during activities with similar torso acceleration profiles to running, such as rope jumping and elliptical machine use. Significance. The CARL classifier can accurately identify bouts of running as short as three seconds in free-living accelerometry data. An open-source implementation of the CARL classifier is available at github.com/johnjdavisiv/carl.

Rheumatic Heart Disease Screening Based on Phonocardiogram.

Rheumatic heart disease (RHD) is one of the most common causes of cardiovascular complications in developing countries. It is a heart valve disease that typically affects children. Impaired heart valves stop functioning properly, resulting in a turbulent blood flow within the heart known as a murmur. This murmur can be detected by cardiac auscultation. However, the specificity and sensitivity of manual auscultation were reported to be low. The other alternative is echocardiography, which is costly and requires a highly qualified physician. Given the disease’s current high prevalence rate (the latest reported rate in the study area (Ethiopia) was 5.65%), there is a pressing need for early detection of the disease through mass screening programs. This paper proposes an automated RHD screening approach using machine learning that can be used by non-medically trained persons outside of a clinical setting. Heart sound data was collected from 124 persons with RHD (PwRHD) and 46 healthy controls (HC) in Ethiopia with an additional 81 HC records from an open-access dataset. Thirty-one distinct features were extracted to correctly represent RHD. A support vector machine (SVM) classifier was evaluated using two nested cross-validation approaches to quantitatively assess the generalization of the system to previously unseen subjects. For regular nested 10-fold cross-validation, an f1-score of 96.0 ± 0.9%, recall 95.8 ± 1.5%, precision 96.2 ± 0.6% and a specificity of 96.0 ± 0.6% were achieved. In the imbalanced nested cross-validation at a prevalence rate of 5%, it achieved an f1-score of 72.2 ± 0.8%, recall 92.3 ± 0.4%, precision 59.2 ± 3.6%, and a specificity of 94.8 ± 0.6%. In screening tasks where the prevalence of the disease is small, recall is more important than precision. The findings are encouraging, and the proposed screening tool can be inexpensive, easy to deploy, and has an excellent detection rate. As a result, it has the potential for mass screening and early detection of RHD in developing countries.

Reliable sleep staging of unseen subjects with fusion of multiple EEG features and RUSBoost

Extensive experiments have been carried out in this study to classify sleep EEG from three different standard databases – Sleep EDF, DREAMS and Expanded sleep EDF databases. Both two-class (sleep-awake) and multiclass classifications have been performed using a fusion of various EEG features and an ensemble classifier called random undersampling with boosting technique (RUSBoost). The results achieved using a single channel EEG are comparable or better than the state-of-the-art methods in the literature for both types of classification, on all the databases. Two-class classification is useful to determine the preferred timings for sensory stimulation of patients with disorders of consciousness. 10-fold cross-validation accuracies of 92.6% and 97.9% have been obtained on Sleep EDF database for 6-class and 2-class problems, respectively. Using Expanded Sleep-EDF dataset, the accuracies improved to 96.3% for 6-state and 99.8% for 2-state classification. For DREAMS dataset, we achieved an accuracy of 96.6% for 2-state classification. Unlike most research in the literature where performance on unseen subjects is not considered, we report classification results on the data from unseen test subjects using both 50%-holdout and leave-one-out cross-validation approaches. Similar results were achieved using both validation techniques for different datasets emphasizing the reliability of our method. These results are very crucial for the method to be applicable for clinical use.

Calibration-free pTx of the human heart at 7T via 3D universal pulses.

MRI at ultra-high fields in the human body is highly challenging and requires lengthy calibration times to compensate for spatially heterogeneous profiles. This study investigates the feasibility of using pre-computed universal pulses for calibration-free homogeneous 3D flip angle distribution in the human heart at 7T. Twenty-two channel-wise 3D data sets were acquired under free-breathing in 19 subjects to generate a library for an offline universal pulse (UP) design (group 1: 12 males [M] and 7 females [F], 21-66 years, 19.8-28.3 kg/m2 ). Three of these subjects (2M/1F, 21-33 years, 20.8-23.6 kg/m2 ) were re-scanned on different days. A 4kT-points UP optimized for the 22 channel-wise 3D data sets in group 1 (UP22-4kT) is proposed and applied at 7T in 9 new and unseen subjects (group 2: 4M/5F, 25-56 years, 19.5-35.3 kg/m2 ). Multiple tailored and universal static and dynamic parallel-transmit (pTx) pulses were designed and evaluated for different permutations of the data sets in group 1 and 2. The proposed UP22-4kT provides low variation in all subjects, seen and unseen, without severe signal drops. Experimental data at 7T acquired with UP22-4kT shows comparable image quality as data acquired with tailored-4kT pulses and demonstrates successful calibration-free pTx of the human heart. UP22-4kT allows for calibration-free homogeneous flip angle distributions across the human heart at 7T. Large inter-subject variations because of sex, age, and body mass index are well tolerated. The proposed universal pulse removes the need for lengthy (10-15 min) calibration scans and therefore has the potential to bring body imaging at 7T closer to the clinical application.

Prediction of response to repetitive transcranial magnetic stimulation in phantom sounds based on individual brain anatomy

Non-invasive brain stimulation can reduce the severity of tinnitus phantom sounds beyond the time of stimulation by inducing regional neuroplastic changes. However, there are no good clinical predictors for treatment outcome. We used machine learning to investigate whether brain anatomy can predict therapeutic outcome. Sixty-one chronic tinnitus patients received repetitive transcranial magnetic stimulation of left dorsolateral prefrontal and temporal cortex. Before repetitive transcranial magnetic stimulation, a structural magnetic resonance image was obtained from all patients. To predict individual treatment response in new subjects, we employed a support vector machine ensemble for individual out-of-sample prediction. In the cross-validation, the support vector machine ensemble based on stratified sub-sampling and feature selection yielded an area under the curve of 0.87 for prediction of therapy success in new, previously unseen subjects. This corresponded to a balanced accuracy of 83.5%, sensitivity of 77.2% and specificity of 87.2%. Investigating the most selected features showed the involvement of the auditory cortex but also revealed a network of non-auditory brain areas. These findings suggest that idiosyncratic brain patterns accurately predict individual responses to repetitive transcranial magnetic stimulation treatment for tinnitus. Our findings may hence pave the way for future investigations into the precision treatment of tinnitus, involving automatic identification of the appropriate treatment method for the individual patient.

Deep Learning-Based ECG-Free Cardiac Navigation for Multi-Dimensional and Motion-Resolved Continuous Magnetic Resonance Imaging.

For the clinical assessment of cardiac vitality, time-continuous tomographic imaging of the heart is used. To further detect e.g., pathological tissue, multiple imaging contrasts enable a thorough diagnosis using magnetic resonance imaging (MRI). For this purpose, time-continous and multi-contrast imaging protocols were proposed. The acquired signals are binned using navigation approaches for a motion-resolved reconstruction. Mostly, external sensors such as electrocardiograms (ECG) are used for navigation, leading to additional workflow efforts. Recent sensor-free approaches are based on pipelines requiring prior knowledge, e.g., typical heart rates. We present a sensor-free, deep learning-based navigation that diminishes the need for manual feature engineering or the necessity of prior knowledge compared to previous works. A classifier is trained to estimate the R-wave timepoints in the scan directly from the imaging data. Our approach is evaluated on 3-D protocols for continuous cardiac MRI, acquired in-vivo and free-breathing with single or multiple imaging contrasts. We achieve an accuracy of > 98% on previously unseen subjects, and a well comparable image quality with the state-of-the-art ECG-based reconstruction. Our method enables an ECG-free workflow for continuous cardiac scans with simultaneous anatomic and functional imaging with multiple contrasts. It can be potentially integrated without adapting the sampling scheme to other continuous sequences by using the imaging data for navigation and reconstruction.

Comprehensive verbal fluency features predict executive function performance

Semantic verbal fluency (sVF) tasks are commonly used in clinical diagnostic batteries as well as in a research context. When performing sVF tasks to assess executive functions (EFs) the sum of correctly produced words is the main measure. Although previous research indicates potentially better insights into EF performance by the use of finer grained sVF information, this has not yet been objectively evaluated. To investigate the potential of employing a finer grained sVF feature set to predict EF performance, healthy monolingual German speaking participants (n = 230) were tested with a comprehensive EF test battery and sVF tasks, from which features including sum scores, error types, speech breaks and semantic relatedness were extracted. A machine learning method was applied to predict EF scores from sVF features in previously unseen subjects. To investigate the predictive power of the advanced sVF feature set, we compared it to the commonly used sum score analysis. Results revealed that 8 / 14 EF tests were predicted significantly using the comprehensive sVF feature set, which outperformed sum scores particularly in predicting cognitive flexibility and inhibitory processes. These findings highlight the predictive potential of a comprehensive evaluation of sVF tasks which might be used as diagnostic screening of EFs.

A Comparative Study of Window Size and Channel Arrangement on EEG-Emotion Recognition Using Deep CNN.

Emotion recognition based on electroencephalograms has become an active research area. Yet, identifying emotions using only brainwaves is still very challenging, especially the subject-independent task. Numerous studies have tried to propose methods to recognize emotions, including machine learning techniques like convolutional neural network (CNN). Since CNN has shown its potential in generalization to unseen subjects, manipulating CNN hyperparameters like the window size and electrode order might be beneficial. To our knowledge, this is the first work that extensively observed the parameter selection effect on the CNN. The temporal information in distinct window sizes was found to significantly affect the recognition performance, and CNN was found to be more responsive to changing window sizes than the support vector machine. Classifying the arousal achieved the best performance with a window size of ten seconds, obtaining 56.85% accuracy and a Matthews correlation coefficient (MCC) of 0.1369. Valence recognition had the best performance with a window length of eight seconds at 73.34% accuracy and an MCC value of 0.4669. Spatial information from varying the electrode orders had a small effect on the classification. Overall, valence results had a much more superior performance than arousal results, which were, perhaps, influenced by features related to brain activity asymmetry between the left and right hemispheres.

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables.

Human Locomotion Mode Recognition (LMR) has the potential to be used as a control mechanism for lower-limb active prostheses. Active prostheses can assist and restore a more natural gait for amputees, but as a medical device it must minimize user risks, such as falls and trips. As such, any control system must have high accuracy and robustness, with a detailed understanding of its internal operation. Long Short-Term Memory (LSTM) machine-learning networks can perform LMR with high accuracy levels. However, the internal behavior during classification is unknown, and they struggle to generalize when presented with novel users. The target problem addressed in this paper is understanding the LSTM classification behavior for LMR. A dataset of six locomotive activities (walking, stopped, stairs and ramps) from 22 non-amputee subjects is collected, capturing both steady-state and transitions between activities in natural environments. Non-amputees are used as a substitute for amputees to provide a larger dataset. The dataset is used to analyze the internal behavior of a reduced complexity LSTM network. This analysis identifies that the model primarily classifies activity type based on data around early stance. Evaluation of generalization for unseen subjects reveals low sensitivity to hyper-parameters and over-fitting to individuals’ gait traits. Investigating the differences between individual subjects showed that gait variations between users primarily occur in early stance, potentially explaining the poor generalization. Adjustment of hyper-parameters alone could not solve this, demonstrating the need for individual personalization of models. The main achievements of the paper are (i) the better understanding of LSTM for LMR, (ii) demonstration of its low sensitivity to learning hyper-parameters when evaluating novel user generalization, and (iii) demonstration of the need for personalization of ML models to achieve acceptable accuracy.

Reward Processing in Novelty Seekers: A Transdiagnostic Psychiatric Imaging Biomarker

BackgroundDysfunctional reward processing is implicated in multiple mental disorders. Novelty seeking (NS) assesses preference for seeking novel experiences, which is linked to sensitivity to reward environmental cues. MethodsA subset of 14-year-old adolescents (IMAGEN) with the top 20% ranked high-NS scores was used to identify high-NS–associated multimodal components by supervised fusion. These features were then used to longitudinally predict five different risk scales for the same and unseen subjects (an independent dataset of subjects at 19 years of age that was not used in predictive modeling training at 14 years of age) (within IMAGEN, n ≈1100) and even for the corresponding symptom scores of five types of patient cohorts (non-IMAGEN), including drinking (n = 313), smoking (n = 104), attention-deficit/hyperactivity disorder (n = 320), major depressive disorder (n = 81), and schizophrenia (n = 147), as well as to classify different patient groups with diagnostic labels. ResultsMultimodal biomarkers, including the prefrontal cortex, striatum, amygdala, and hippocampus, associated with high NS in 14-year-old adolescents were identified. The prediction models built on these features are able to longitudinally predict five different risk scales, including alcohol drinking, smoking, hyperactivity, depression, and psychosis for the same and unseen 19-year-old adolescents and even predict the corresponding symptom scores of five types of patient cohorts. Furthermore, the identified reward-related multimodal features can classify among attention-deficit/hyperactivity disorder, major depressive disorder, and schizophrenia with an accuracy of 87.2%. ConclusionsAdolescents with higher NS scores can be used to reveal brain alterations in the reward-related system, implicating potential higher risk for subsequent development of multiple disorders. The identified high-NS–associated multimodal reward-related signatures may serve as a transdiagnostic neuroimaging biomarker to predict disease risks or severity.

On the Generalization and Reliability of Single Radar-Based Human Activity Recognition

Identifying human activities using short-range and low-power radars has attracted much attention among the researchers and consumer electronics industry. This paper considers human activity recognition in the context of a single Frequency Modulated Continuous Wave (FMCW) radar as the measurement tool. A classification pipeline is proposed to handle the data pre-processing and feature extraction and a machine-learning based solution is devised to undertake the activity classification. The performance of the proposed architecture is evaluated under both unseen subjects and new room layouts. We show how the accuracy of the activity classification will be affected by situations such as poor aspect-angle and occlusions created by furniture that normally arise in realistic scenarios where an unseen layout is considered. A two-stage classifier will be then proposed to enhance the generalization of the model, especially, to unseen rooms. Besides, an extensive feature exploration will be conducted and the importance of features in the generalization will be studied. The results in this paper will conclude a machine learning pipeline that will generalize well to unseen subjects and new room layouts, which are two main difficulties that arise in most radar-based activity classification tasks.

A framework for automatic hand range of motion evaluation of rheumatoid arthritis patients

We propose a framework for evaluation of finger movement patterns on Rheumatoid Arthritis patients: flexion, extension, abduction and adduction. The framework uses a state-of-the-art 3D hand pose estimation method that runs in real-time, allowing users to visualize 3D skeleton tracking results at the same time as the depth images are acquired. We compute flexion and abduction angles from the obtained skeleton pose parameters. We performed data acquisition from a cohort of patients and a control set and compared the angles from those two sets of people. An analysis using time series similarity with frequency domain descriptors is adopted to characterize the movement patterns for flexion/extension. We performed classification experiments using these descriptors, thus distinguishing movement sequences of hands with rheumatoid arthritis from healthy hands. The descriptors used in the classification experiment were effective and reached average results of 89 % in scenarios of unseen subjects, and an average of 82 % in experiments with sample synthesis that allow a more robust statistical performance evaluation. Our framework allows the characterization of the current state of the disorder in each patient, with minimal intervention and reduced evaluation time.

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations. However, it is still a very challenging task as it needs full understanding of the input face and very depending on the facial appearance. In this paper, we present an end-to-end generative adversarial network for facial expression synthesis. Given the facial landmarks and the expression label of a target image, our method automatically generates a corresponding expression facial image with the identity information and facial details well preserved. Both qualitative and quantitative experiments are conducted on the CK+ and Oulu-CASIA datasets. Experimental results show that our method has the compelling perceptual results even there exist large differences in facial shapes for unseen subjects.

MACRO: Multi-Attention Convolutional Recurrent Model for Subject-Independent ERP Detection

Due to the low signal-to-noise ratio, limited training samples, and large inter-subject variabilities in electroencephalogram (EEG) signals, developing a subject-independent brain-computer interface (BCI) system used for new users without any calibration is still challenging. In this letter, we propose a novel Multi-Attention Convolutional Recurrent mOdel (MACRO) for EEG-based event-related potential (ERP) detection in the subject-independent scenario. Specifically, the convolutional recurrent network is designed to capture the spatial-temporal features, while the multi-attention mechanism is integrated to focus on the most discriminative channels and temporal periods of EEG signals. Comprehensive experiments conducted on a benchmark dataset for RSVP-based BCIs show that our method achieves the best performance compared with the five state-of-the-art baseline methods. This result indicates that our method is able to extract the underlying subject-invariant EEG features and generalize to unseen subjects. Finally, the ablation studies verify the effectiveness of the designed multi-attention mechanism in MACRO for EEG-based ERP detection.