Discriminative Latent Features Research Articles

Modelling appearance variations in expressive and neutral face image for automatic facial expression recognition

AbstractIn automatic facial expression recognition (AFER) systems, modelling the spatio‐temporal feature information in a specific manner, coalescing, and its effective utilization is challenging. The state‐of‐the‐art studies have examined integrating multiple features to enhance the recognition rate of AFER systems. However, the feature variations between expressive and neutral face images are not fully explored to identify the expression class. The proposed research presents an innovative approach to AFER by modelling appearance variations in both expressive and neutral face images. The prominent contributions of the work are developing a novel and hybrid feature space by integrating the discriminative feature distribution derived from expressive and neutral face images; preserving the highly discriminative latent feature distribution using autoencoders. Local binary pattern (LBP) and histogram of oriented gradients (HOG) are the feature descriptors employed to derive the discriminative texture and shape information, respectively. The component‐based approach is employed, wherein the features are derived from the salient facial regions instead of the whole face. The three‐stage stacked deep convolutional autoencoder (SDCA) and multi‐class support vector machine (MSVM) are employed to address dimensionality reduction and classification, respectively. The efficacy of the proposed model is substantiated by empirical findings, which establish its superiority in terms of accuracy in AFER tasks on widely recognized benchmark datasets.

A unified framework for visual domain adaptation with covariance matching

In the field of Machine Learning, it is widely acknowledged that training and test data should ideally stem from the same source and distribution. However, in the real world, this is not always feasible. Domain Adaptation (DA) techniques address this issue by adapting a classifier trained on annotated source domain data on unannotated target domain data while minimizing the impact of domain shift. Many recent DA approaches concentrate on learning a latent feature space that remains invariant to domain shift by mitigating various statistical and geometrical divergences. Although these methods have demonstrated effectiveness, they frequently overlook a crucial aspect of learning a domain-invariant discriminative latent feature space across diverse domains. To address this issue, we propose a novel framework called Unified Framework for Visual Domain Adaptation with Covariance Matching (UDACM) that aims to learn a domain invariant and class discriminative latent feature space by incorporating multiple objectives such as maximizing target domain variance, minimizing distribution and subspace divergence, performing manifold learning, preserving discriminative information of both the source and target domains, and measuring and matching covariance simultaneously. In our proposed framework, the measuring and matching of covariance play a crucial role in ensuring the learning of discriminative latent feature space by aligning the within-class and between-class covariance matrices of the source and target domains simultaneously. Detailed experiments demonstrate the outperformance of UDACM on baseline datasets such as CMU-PIE, Office+Caltech10, USPS+MNIST, VLCS, and Office–Home, compared to various established primitive, shallow, and deep domain adaptation methods on several image classification tasks.

Clustering of Multilayer Networks Using Joint Learning Algorithm With Orthogonality and Specificity of Features.

Complex systems in nature and society consist of various types of interactions, where each type of interaction belongs to a layer, resulting in the so-called multilayer networks. Identifying specific modules for each layer is of great significance for revealing the structure-function relations in multilayer networks. However, the available approaches are criticized undesirable because they fail to explicitly the specificity of modules, and balance the specificity and connectivity of modules. To overcome these drawbacks, we propose an accurate and flexible algorithm by joint learning matrix factorization and sparse representation (jMFSR) for specific modules in multilayer networks, where matrix factorization extracts features of vertices and sparse representation discovers specific modules. To exploit the discriminative latent features of vertices in multilayer networks, jMFSR incorporates linear discriminant analysis (LDA) into non-negative matrix factorization (NMF) to learn features of vertices that distinguish the categories. To explicitly measure the specificity of features, jMFSR decomposes features of vertices into common and specific parts, thereby enhancing the quality of features. Then, jMFSR jointly learns feature extraction, common-specific feature factorization, and clustering of multilayer networks. The experiments on 11 datasets indicate that jMFSR significantly outperforms state-of-the-art baselines in terms of various measurements.

MTLFuseNet: A novel emotion recognition model based on deep latent feature fusion of EEG signals and multi-task learning

How to extract discriminative latent feature representations from electroencephalography (EEG) signals and build a generalized model is a topic in EEG-based emotion recognition research. This study proposed a novel emotion recognition model based on deep latent feature fusion of EEG signals and multi-task learning, referred to as MTLFuseNet. MTLFuseNet learned spatio-temporal latent features of EEG in an unsupervised manner by a variational autoencoder (VAE) and learned the spatio-spectral features of EEG in a supervised manner by a graph convolutional network (GCN) and gated recurrent unit (GRU) network. Afterward, the two latent features were fused to form more complementary and discriminative spatio-temporal–spectral fusion features for EEG signal representation. In addition, MTLFuseNet was constructed based on multi-task learning. The focal loss was introduced to solve the problem of unbalanced sample classes in an emotional dataset, and the triplet-center loss was introduced to make the fused latent feature vectors more discriminative. Finally, a subject-independent leave-one-subject-out cross-validation strategy was used to validate extensively on two public datasets, DEAP and DREAMER. On the DEAP dataset, the average accuracies of valence and arousal are 71.33% and 73.28%, respectively. On the DREAMER dataset, the average accuracies of valence and arousal are 80.43% and 83.33%, respectively. The experimental results show that the proposed MTLFuseNet model achieves excellent recognition performance, outperforming the state-of-the-art methods.

Graph regularized spatial–spectral subspace clustering for hyperspectral band selection

Hyperspectral band selection, which aims to select a small number of bands to reduce data redundancy and noisy bands, has attracted widespread attention in recent years. Many effective clustering-based band selection methods have been proposed to accomplish the band selection task and have achieved satisfying performance. However, most of the previous methods reshape the original hyperspectral images (HSIs) into a set of stretched band vectors, which ignore the spatial information of HSIs and the difference between diverse regions. To address these issues, a graph regularized spatial–spectral subspace clustering method for hyperspectral band selection is proposed in this paper, referred to as GRSC. Specifically, the proposed method adopts superpixel segmentation to preserve the spatial information of HSIs by segmenting their first principal component into diverse homogeneous regions. Then the discriminative latent features are generated from each segmented region to represent the whole band, which can mitigate the effect of noise on the band selection. Finally, a self-representation subspace clustering model and an l2,1-norm regularization are utilized to explore the spectral correlation among all bands. In addition, a similarity graph between region-aware latent features is adaptively learned to preserve the spatial structure of HSIs in the latent representation space. Extensive classification experimental results on three public datasets verify the effectiveness of GRSC over several state-of-the-art methods. The demo code of this work is publicly available at https://github.com/WangJun2023/GRSC.

GAN-MVAE: A discriminative latent feature generation framework for generalized zero-shot learning

Generalized zero-shot learning (GZSL) is a challenging task that aims to recognize both seen and unseen classes. It is achieved by transferring knowledge from seen classes to unseen classes via a shared semantic space (e.g. attribute space). Recently, Generative adversarial network (GAN) have gained considerable attention in GZSL. GAN can generate missing unseen classes samples from class-specific semantic embedding for training, thereby transforming GZSL into a traditional classification task and achieving impressive results. However, due to the instability during training and the complexity of data distribution, a simple GAN framework cannot capture the real data distribution perfectly, and there is still a large gap between the generated and real sample distributions, which severely limits the performance of GZSL. Therefore, the proposed GAN-MVAE further aligns the real and generated samples by mapping them into the latent space of multi-modal reconstruction variational autoencoder (MVAE), while preserving discriminative semantic information through cross-modal reconstruction. GAN-MVAE provides some inspiration for the study of multi-modal alignment and asymmetry VAE. Extensive experiments on four GZSL benchmark datasets show that GAN-MVAE significantly outperforms the state of the arts.

Deep Representation-Based Domain Adaptation for Nonstationary EEG Classification.

In the context of motor imagery, electroencephalography (EEG) data vary from subject to subject such that the performance of a classifier trained on data of multiple subjects from a specific domain typically degrades when applied to a different subject. While collecting enough samples from each subject would address this issue, it is often too time-consuming and impractical. To tackle this problem, we propose a novel end-to-end deep domain adaptation method to improve the classification performance on a single subject (target domain) by taking the useful information from multiple subjects (source domain) into consideration. Especially, the proposed method jointly optimizes three modules, including a feature extractor, a classifier, and a domain discriminator. The feature extractor learns the discriminative latent features by mapping the raw EEG signals into a deep representation space. A center loss is further employed to constrain an invariant feature space and reduce the intrasubject nonstationarity. Furthermore, the domain discriminator matches the feature distribution shift between source and target domains by an adversarial learning strategy. Finally, based on the consistent deep features from both domains, the classifier is able to leverage the information from the source domain and accurately predict the label in the target domain at the test time. To evaluate our method, we have conducted extensive experiments on two real public EEG data sets, data set IIa, and data set IIb of brain-computer interface (BCI) Competition IV. The experimental results validate the efficacy of our method. Therefore, our method is promising to reduce the calibration time for the use of BCI and promote the development of BCI.

One-Shot Learning for Long-Tail Visual Relation Detection

The aim of visual relation detection is to provide a comprehensive understanding of an image by describing all the objects within the scene, and how they relate to each other, in < object-predicate-object > form; for example, < person-lean on-wall > . This ability is vital for image captioning, visual question answering, and many other applications. However, visual relationships have long-tailed distributions and, thus, the limited availability of training samples is hampering the practicability of conventional detection approaches. With this in mind, we designed a novel model for visual relation detection that works in one-shot settings. The embeddings of objects and predicates are extracted through a network that includes a feature-level attention mechanism. Attention alleviates some of the problems with feature sparsity, and the resulting representations capture more discriminative latent features. The core of our model is a dual graph neural network that passes and aggregates the context information of predicates and objects in an episodic training scheme to improve recognition of the one-shot predicates and then generate the triplets. To the best of our knowledge, we are the first to center on the viability of one-shot learning for visual relation detection. Extensive experiments on two newly-constructed datasets show that our model significantly improved the performance of two tasks PredCls and SGCls from 2.8% to 12.2% compared with state-of-the-art baselines.

Interpretable, highly accurate brain decoding of subtly distinct brain states from functional MRI using intrinsic functional networks and long short-term memory recurrent neural networks

Decoding brain functional states underlying cognitive processes from functional MRI (fMRI) data using multivariate pattern analysis (MVPA) techniques has achieved promising performance for characterizing brain activation patterns and providing neurofeedback signals. However, it remains challenging to decode subtly distinct brain states for individual fMRI data points due to varying temporal durations and dependency among different cognitive processes. In this study, we develop a deep learning based framework for brain decoding by leveraging recent advances in intrinsic functional network modeling and sequence modeling using long short-term memory (LSTM) recurrent neural networks (RNNs). Particularly, subject-specific intrinsic functional networks (FNs) are computed from resting-state fMRI data and are used to characterize functional signals of task fMRI data with a compact representation for building brain decoding models, and LSTM RNNs are adopted to learn brain decoding mappings between functional profiles and brain states. Validation results on fMRI data from the HCP dataset have demonstrated that brain decoding models built on training data using the proposed method could learn discriminative latent feature representations and effectively distinguish subtly distinct working memory tasks of different subjects with significantly higher accuracy than conventional decoding models. Informative FNs of the brain decoding models identified as brain activation patterns of working memory tasks were largely consistent with the literature. The method also obtained promising decoding performance on motor and social cognition tasks. Our results suggest that LSTM RNNs in conjunction with FNs could build interpretable, highly accurate brain decoding models.

Learning robust latent representation for discriminative regression

In this paper, we present a generic effective formulation, dubbed robust latent representation learning (RLRL), for discriminative linear regression. We formulate the RLRL optimization problem as a joint learning framework of discriminative latent feature selection and robust linear regression. Specifically, instead of directly projecting the original high-dimensional features onto a target space, RLRL learns discriminative latent representation by concurrently suppressing the redundant information from original features and constructing a robust latent subspace. To improve the effectiveness of the regression task, a capped lp-norm regression model is formulated for robust linear regression. Furthermore, RLRL incorporates learning latent representation and building marginalized regressing prediction into one framework for reducing the classification error of the regression model. Consequently, a general framework for simultaneous feature selection and robust regression is discussed and explicitly presented. An effective alternating optimization algorithm with efficient convergence is developed to solve the resulting optimization problem. Extensive experimental results conducted on diverse databases validate the effectiveness of the proposed RLRL method in comparison with state-of-the-art regression methods.