Latent Feature Space Research Articles

Abnormal event detection using adversarial predictive coding for motion and appearance

In this paper, we propose adversarial predictive coding (APC), a novel method for detecting abnormal events. Abnormal event detection (AED) is to identify unobserved events from a given training dataset consisting of normal events, and it is considered as one of the most important objectives in developing intelligent surveillance systems. Given videos and motion flows of normal events, the APC derives a normal event model by applying an adversarial prediction approach on the jointly learnt latent feature space from the videos and motion flows. Since latent space requires more abstracted and noise-free information than the raw data space, the APC can derive a more discriminative model for normal events compared with other deep learning-based AED methods which directly apply uni-modal losses such as mean square error and cross-entropy to low-level data such as video frames. We demonstrate the effectiveness of our method in detecting abnormal events using UCSD-Ped, Avenue, and UCF-Crime datasets. The experimental results show that the APC surpass the existing state-of-the-art AED methods by deriving a more discriminative model for normal events.

Towards Semantically-Rich Spatial Network Representation Learning via Automated Feature Topic Pairing.

Automated characterization of spatial data is a kind of critical geographical intelligence. As an emerging technique for characterization, spatial Representation Learning (SRL) uses deep neural networks (DNNs) to learn non-linear embedded features of spatial data for characterization. However, SRL extracts features by internal layers of DNNs, and thus suffers from lacking semantic labels. Texts of spatial entities, on the other hand, provide semantic understanding of latent feature labels, but is insensible to deep SRL models. How can we teach a SRL model to discover appropriate topic labels in texts and pair learned features with the labels? This paper formulates a new problem: feature-topic pairing, and proposes a novel Particle Swarm Optimization (PSO) based deep learning framework. Specifically, we formulate the feature-topic pairing problem into an automated alignment task between 1) a latent embedding feature space and 2) a textual semantic topic space. We decompose the alignment of the two spaces into: 1) point-wise alignment, denoting the correlation between a topic distribution and an embedding vector; 2) pair-wise alignment, denoting the consistency between a feature-feature similarity matrix and a topic-topic similarity matrix. We design a PSO based solver to simultaneously select an optimal set of topics and learn corresponding features based on the selected topics. We develop a closed loop algorithm to iterate between 1) minimizing losses of representation reconstruction and feature-topic alignment and 2) searching the best topics. Finally, we present extensive experiments to demonstrate the enhanced performance of our method.

Collaborative Reflection-Augmented Autoencoder Network for Recommender Systems

As the deep learning techniques have expanded to real-world recommendation tasks, many deep neural network based Collaborative Filtering (CF) models have been developed to project user-item interactions into latent feature space, based on various neural architectures, such as multi-layer perceptron, autoencoder, and graph neural networks. However, the majority of existing collaborative filtering systems are not well designed to handle missing data. Particularly, in order to inject the negative signals in the training phase, these solutions largely rely on negative sampling from unobserved user-item interactions and simply treating them as negative instances, which brings the recommendation performance degradation. To address the issues, we develop a C ollaborative R eflection-Augmented A utoencoder N etwork (CRANet), that is capable of exploring transferable knowledge from observed and unobserved user-item interactions. The network architecture of CRANet is formed of an integrative structure with a reflective receptor network and an information fusion autoencoder module, which endows our recommendation framework with the ability of encoding implicit user’s pairwise preference on both interacted and non-interacted items. Additionally, a parametric regularization-based tied-weight scheme is designed to perform robust joint training of the two-stage CRANetmodel. We finally experimentally validate CRANeton four diverse benchmark datasets corresponding to two recommendation tasks, to show that debiasing the negative signals of user-item interactions improves the performance as compared to various state-of-the-art recommendation techniques. Our source code is available at https://github.com/akaxlh/CRANet.

Few-shot contrastive learning for image classification and its application to insulator identification.

This paper presents a novel discriminative Few-shot learning architecture based on batch compact loss. Currently, Convolutional Neural Network (CNN) has achieved reasonably good performance in image recognition. Most existing CNN methods facilitate classifiers to learn discriminating patterns to identify existing categories trained with large samples. However, learning to recognize novel categories from a few examples is a challenging task. To address this, we propose the Residual Compact Network to train a deep neural network to learn hierarchical nonlinear transformations to project image pairs into the same latent feature space, under which the distance of each positive pair is reduced. To better use the commonality of class-level features for category recognition, we develop a batch compact loss to form robust feature representations relevant to a category. The proposed methods are evaluated on several datasets. Experimental evaluations show that our proposed method achieves acceptable results in Few-shot learning.

Robust subspace clustering network with dual-domain regularization

The field of deep subspace clustering has advanced rapidly in recent years. Ideas such as self-expression and self-supervision have led to innovative network design and improved clustering performance. However, it is observed that the nonlinear low-dimensional manifold constraint is valid in not only the ambient but also the latent feature spaces. Meanwhile, the issue of robustness has been largely overlooked in the literature of deep subspace clustering despite previous studies on robust model-based clustering. Based on these two observations, we present a robust subspace clustering network (RSCN) based on a novel hybrid loss function with dual-domain regularization. On the one hand, we propose to replace the existing L2 loss by a robust hybrid function inspired by half-quadratic minimization; on the other hand, we come up with a novel strategy of sparsity regularization in the dual domain (both ambient and feature space). To the best of our knowledge, this is the first attempt to incorporate dual manifold constraints into deep subspace clustering. Experimental results show that our new network outperforms the existing state-of-the-art on several widely-studied datasets such as Extended Yale B, COIL20, and COIL100. The performance gain of our RSCN over several other competing approaches improves dramatically in the presence of noise contamination.

Shape Estimation of 3D Human Body Model Based on the Geometric Generation Model

In this paper, we propose a geometric generation model for the shape estimation of 3D human body. The model decouples the two processes: (1) Encoding/decoding process maps the samples between the image space X and the latent (feature) space Z. This step achieves the dimension deduction. (2) The process of the Probability measure transformation transforms a fixed distribution ζ G P(Z) to any given distribution µ G P(Z). Before the shape estimation, U-V transform is done with Densepose, in order to transform the 2D human body image to 3D shape pose image.

A novel radar target recognition method for open and imbalanced high-resolution range profile

High-resolution range profile (HRRP) data often has imbalanced and open-ended distribution in realistic radar target recognition task, requiring the recognition system classify targets among majority and minority classes and detect unseen targets accurately. In this paper, we propose a novel algorithm for open and imbalanced HRPP recognition tasks, by learning from realistic data distribution and optimizing the accuracy over the majority, minority and open classes. This method maps the HRRP data to the latent feature space, enhances the feature by sharing knowledge from majority to minority class, and provides a scalar indicating the familiarity to known classes. Dual-attention is developed to provide strong discriminative feature representation. An angular penalty is employed in the loss function to optimize the intra-class similarity and inter-class variability. Experiments on measured data prove that the proposed algorithm outperforms other existing methods under both close and open settings with accuracy increased significantly, and more discriminative representation. This study provides a promising and effective approach for open and imbalanced HRRP target recognition.

Unsupervised Subspace Extraction via Deep Kernelized Clustering

Feature extraction has been widely studied to find informative latent features and reduce the dimensionality of data. In particular, due to the difficulty in obtaining labeled data, unsupervised feature extraction has received much attention in data mining. However, widely used unsupervised feature extraction methods require side information about data or rigid assumptions on the latent feature space. Furthermore, most feature extraction methods require predefined dimensionality of the latent feature space,which should be manually tuned as a hyperparameter. In this article, we propose a new unsupervised feature extraction method called Unsupervised Subspace Extractor ( USE ), which does not require any side information and rigid assumptions on data. Furthermore, USE can find a subspace generated by a nonlinear combination of the input feature and automatically determine the optimal dimensionality of the subspace for the given nonlinear combination. The feature extraction process of USE is well justified mathematically, and we also empirically demonstrate the effectiveness of USE for several benchmark datasets.

Self-spectral learning with GAN based spectral–spatial target detection for hyperspectral image

To alleviate the shortcomings of target detection in only one aspect and reduce redundant information among adjacent bands, we propose a spectral–spatial target detection (SSTD) framework in deep latent space based on self-spectral learning (SSL) with a spectral generative adversarial network (GAN). The concept of SSL is introduced into hyperspectral feature extraction in an unsupervised fashion with the purpose of background suppression and target saliency. In particular, a novel structure-to-structure selection rule that takes full account of the structure, contrast, and luminance similarity is established to interpret the mapping relationship between the latent spectral feature space and the original spectral band space, to generate the optimal spectral band subset without any prior knowledge. Finally, the comprehensive result is achieved by nonlinearly combining the spatial detection on the fused latent features with the spectral detection on the selected band subset and the corresponding selected target signature. This paper paves a novel self-spectral learning way for hyperspectral target detection and identifies sensitive bands for specific targets in practice. Comparative analyses demonstrate that the proposed SSTD method presents superior detection performance compared with CSCR, ACE, CEM, hCEM, and ECEM.

Attentive Edgemap Fusion for Sketch-Based Image Retrieval

<p indent=0mm>Sketch-based image retrieval (SBIR) aims to return a collection of corresponding images based on an input sketch. Different from traditional content-based image retrieval, unique difficulties exist due to the large domain gap between sketches and natural images. Based on the similarity between edgemaps and sketches, a novel SBIR model named spatial attentive edgemap fusion is presented which combines both image and edgemap features. Images and the corresponding edgemaps are first encoded to their own latent feature space, and then fused by a learned spatial attention map. Experiment results on two widely-used SBIR datasets, Sketchy and Flickr15K, show the promising performance of the proposed model.

Informative Feature Disentanglement for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. The strategy of aligning the two domains in latent feature space via metric discrepancy or adversarial learning has achieved considerable progress. However, these existing approaches mainly focus on adapting the entire image and ignore the bottleneck that occurs when forced adaptation of uninformative domain-specific variations undermines the effectiveness of learned features. To address this problem, we propose a novel component called Informative Feature Disentanglement (IFD), which is equipped with the adversarial network or the metric discrepancy model, respectively. Accordingly, the new network architectures, named IFDAN and IFDMN, enable informative feature refinement before the adaptation. The proposed IFD is designed to disentangle informative features from the uninformative domain-specific variations, which are produced by a Variational Autoencoder (VAE) with lateral connections from the encoder to the decoder. We cooperatively apply the IFD to conduct supervised disentanglement for the source domain and unsupervised disentanglement for the target domain. In this way, informative features are disentangled from the domain-specific details before the adaptation. Extensive experimental results on three gold-standard domain adaptation datasets, e.g., Office31, Office-Home and VisDA-C, demonstrate the effectiveness of the proposed IFDAN and IFDMN models for UDA.

Latent Independent Excitation for Generalizable Sensor-based Cross-Person Activity Recognition

In wearable-sensor-based activity recognition, it is often assumed that the training and test samples follow the same data distribution. This assumption neglects practical scenarios where the activity patterns inevitably vary from person to person. To solve this problem, transfer learning and domain adaptation approaches are often leveraged to reduce the gaps between different participants. Nevertheless, these approaches require additional information (i.e., labeled or unlabeled data, meta-information) from the target domain during the training stage. In this paper, we introduce a novel method named Generalizable Independent Latent Excitation (GILE) for human activity recognition, which greatly enhances the cross-person generalization capability of the model. Our proposed method is superior to existing methods in the sense that it does not require any access to the target domain information. Besides, this novel model can be directly applied to various target domains without re-training or fine-tuning. Specifically, the proposed model learns to automatically disentangle domain-agnostic and domain-specific features, the former of which are expected to be invariant across various persons. To further remove correlations between the two types of features, a novel Independent Excitation mechanism is incorporated in the latent feature space. Comprehensive experimental evaluations are conducted on three benchmark datasets to demonstrate the superiority of the proposed method over the state-of-the-art solutions.

DeepCollaboration: Collaborative Generative and Discriminative Models for Class Incremental Learning

An important challenge for neural networks is to learn incrementally, i.e., learn new classes without catastrophic forgetting. To overcome this problem, generative replay technique has been suggested, which can generate samples belonging to learned classes while learning new ones. However, such generative models usually suffer from increased distribution mismatch between the generated and original samples along the learning process. In this work, we propose DeepCollaboration (D-Collab), a collaborative framework of deep generative and discriminative models to solve this problem effectively. We develop a discriminative learning model to incrementally update the latent feature space for continual classification. At the same time, a generative model is introduced to achieve conditional generation using the latent feature distribution produced by the discriminative model. Importantly, the generative and discriminative models are connected through bidirectional training to enforce cycle-consistency of mappings between feature and image domains. Furthermore, a domain alignment module is used to eliminate the divergence between the feature distributions of generated images and real ones. This module together with the discriminative model can perform effective sample mining to facilitate incremental learning. Extensive experiments on several visual recognition datasets show that our system can achieve state-of-the-art performance.

Auxiliary task guided mean and covariance alignment network for adversarial domain adaptation

Adversarial domain adaptation (ADA) learns representations with strong transferability by eliminating the discrepancy between the probability distributions of the source and the target domains. Conventional ADA methods usually employ the Wasserstein distance as a discrepancy measure and train the classifier only from the source domain data. We show that such methods actually only consider first-order statistics in the latent feature space and the discriminability of the learned representations is not fully explored. In this paper, we propose a novel method called auxiliary task guided mean and covariance alignment network (AT-MCAN) for unsupervised domain adaptation. To take the second-order statistics differences into consideration, AT-MCAN introduces a covariance-aware divergence metric to align the distributions of two domains. To enhance the discriminability of the features, AT-MCAN introduces an auxiliary clustering task to the target domain so that the classifier can employ the data from both domains. We provide both theoretical analysis on the generalization bound and empirical evaluations on standard benchmarks to show the effectiveness of our proposed AT-MCAN.

Combine and conquer: event reconstruction with Bayesian Ensemble Neural Networks

Ensemble learning is a technique where multiple component learners are combined through a protocol. We propose an Ensemble Neural Network (ENN) that uses the combined latent-feature space of multiple neural network classifiers to improve the representation of the network hypothesis. We apply this approach to construct an ENN from Convolutional and Recurrent Neural Networks to discriminate top-quark jets from QCD jets. Such ENN provides the flexibility to improve the classification beyond simple prediction combining methods by linking different sources of error correlations, hence improving the representation between data and hypothesis. In combination with Bayesian techniques, we show that it can reduce epistemic uncertainties and the entropy of the hypothesis by simultaneously exploiting various kinematic correlations of the system, which also makes the network less susceptible to a limitation in training sample size.

Manifold adversarial training for supervised and semi-supervised learning

We propose a new regularization method for deep learning based on the manifold adversarial training (MAT). Unlike previous regularization and adversarial training methods, MAT further considers the local manifold of latent representations. Specifically, MAT manages to build an adversarial framework based on how the worst perturbation could affect the statistical manifold in the latent space rather than the output space. Particularly, a latent feature space with the Gaussian Mixture Model (GMM) is first derived in a deep neural network. We then define the smoothness by the largest variation of Gaussian mixtures when a local perturbation is given around the input data point. On one hand, the perturbations are added in the way that would rough the statistical manifold of the latent space the worst. On the other hand, the model is trained to promote the manifold smoothness the most in the latent space. Importantly, since the latent space is more informative than the output space, the proposed MAT can learn a more robust and compact data representation, leading to further performance improvement. The proposed MAT is important in that it can be considered as a superset of one recently-proposed discriminative feature learning approach called center loss. We conduct a series of experiments in both supervised and semi-supervised learning on four benchmark data sets, showing that the proposed MAT can achieve remarkable performance, much better than those of the state-of-the-art approaches. In addition, we present a series of visualization which could generate further understanding or explanation on adversarial examples.

CuRTAIL: ChaRacterizing and Thwarting AdversarIal Deep Learning

Recent advances in adversarial Deep Learning (DL) have opened up a new and largely unexplored surface for malicious attacks jeopardizing the integrity of autonomous DL systems. This article introduces CuRTAIL, a novel end-to-end computing framework to characterize and thwart potential adversarial attacks and significantly improve the reliability (safety) of a victim DL model. We formalize the goal of preventing adversarial attacks as an optimization problem to minimize the rarely observed regions in the latent feature space spanned by a DL network. To solve the aforementioned minimization problem, a set of complementary but disjoint modular redundancies are trained to validate the legitimacy of the input samples. The proposed countermeasure is unsupervised, meaning that no adversarial sample is leveraged to train modular redundancies. This, in turn, ensures the effectiveness of the defense in the face of generic attacks. We evaluate the robustness of our proposed methodology against the state-of-the-art adaptive attacks in a white-box setting considering that the adversary knows everything about the victim model and its defenders. Extensive evaluations for analyzing MNIST, CIFAR10, and ImageNet data corroborate the effectiveness of CuRTAIL framework against adversarial samples. The computations in each modular redundancy can be performed independently of the other redundancy modules. As such, CuRTAIL detection algorithm can be completely parallelized among multiple hardware settings to achieve maximum throughput. We further provide an open-source Application Programming Interface (API) to facilitate the adoption of the proposed framework for various applications.

Personal‐specific gait recognition based on latent orthogonal feature space

Exoskeleton has been applied in the field of medical rehabilitation and assistance. However, there are still some problems in the interaction between human and exoskeleton, such as time delay, the existence of certain constraints on the human body, and the movement in time is hard to follow. A human motion pattern recognition model based on the long short-term memory (LSTM) is proposed, which can recognise the state of the human body. Meanwhile, the orthogonalisation method is integrated to make personal-specific disentangling, and it can effectively improve the generalisation ability of different groups of people, so as to improve the effective follower ability of the exoskeleton. Compared with some other traditional methods, this model has better performance and stronger generalisation ability, which has certain significance in the field of exoskeleton algorithm.

Feature Space Transformation for Fault Diagnosis of Rotating Machinery under Different Working Conditions.

In recent years, various deep learning models have been developed for the fault diagnosis of rotating machines. However, in practical applications related to fault diagnosis, it is difficult to immediately implement a trained model because the distribution of source data and target domain data have different distributions. Additionally, collecting failure data for various operating conditions is time consuming and expensive. In this paper, we introduce a new transformation method for the latent space between domains using the source domain and normal data of the target domain that can be easily collected. Inspired by semantic transformations in an embedded space in the field of word embedding, discrepancies between the distribution of the source and target domains are minimized by transforming the latent representation space in which fault attributes are preserved. To match the feature area and distribution, spatial attention is applied to learn the latent feature spaces, and the 1D CNN LSTM architecture is implemented to maximize the intra-class classification. The proposed model was validated for two types of rotating machines such as a dataset of rolling bearings as CWRU and a gearbox dataset of heavy machinery. Experimental results show the proposed method has higher cross-domain diagnostic accuracy than others, therefore showing reliable generalization performance in rotating machines operating under various conditions.

OPLS-SR: A novel face super-resolution learning method using orthonormalized coherent features

Face super-resolution (FSR) is an effective way to deal with low-resolution (LR) face images, which can infer the latent high-resolution (HR) face images from the LR inputs. In contrast with traditional FSR methods such as interpolation, learning-based methods generate more realistic HR images of LR faces by exploiting the relationship between HR and LR images. In this paper, we propose a novel FSR learning approach based on orthonormalized partial least squares referred to as OPLS-SR. It first learns a latent coherent feature space of low-dimensional HR and LR face embeddings via a recursive optimization, and then super-resolves the LR face images through global face reconstruction and facial detail compensation. Experimental results on the CAS-PEAL-R1 and FERET face databases have demonstrated the effectiveness of the proposed OPLS-SR method in terms of quantitative and qualitative evaluations.