Latent Feature Space Research Articles

Adversarial Deep Domain Adaptation for Multi-Band SAR Images Classification

Deep convolutional neural networks (CNNs) have made a breakthrough on supervised SAR images classification. However, SAR imaging is considerably affected by the frequency band. That means a neural network trained on a SAR image set of one band is not suitable for the classification of another band images. As manually labeling the training samples of each band is always time-consuming, we propose an unsupervised multi-level domain adaptation method based on adversarial learning to solve the problem of multi-band SAR images classification. First, we train a discriminative CNN using samples of one frequency band data set that contains labels to map the data to a latent feature space. Then, we adjust the trained CNN to map the unlabeled samples of another frequency band data set to the same feature space through alternately optimizing two adversarial loss functions. Thus, the features of these two band images are fused and can be classified by the same classifier. We checked the performance of our method using both simulated data and measured data. Our method made a breakthrough in the classification of multi-band images with accuracies of 99% on both data sets. The results are even very close to the supervised CNN trained using a large number of labeled samples.

Adversarial MACE Prediction After Acute Coronary Syndrome Using Electronic Health Records.

Acute coronary syndrome (ACS), as an emergent and severe syndrome due to decreased blood flow in the coronary arteries, is a leading cause of death and serious long-term disability globally. ACS is usually caused by one of three problems: ST elevation myocardial infarction, non-ST elevation myocardial infarction, or unstable angina. Major adverse cardiac event (MACE) prediction, as a critical tool to estimate the likelihood an individual is at risk of ACS, has been widely adopted in the early prevention and intervention of ACS. Although valuable, existing MACE prediction models are designed to predict the overall probability of MACE occurrence for ACS patients, and lack the ability to look for insight into the disease to distinguish the different subtypes of ACS in a fine-grained manner. It is interesting to exploit the different subtypes of ACS and mine their private and shared underlying knowledge to improve the performance of MACE prediction. In this study, we propose utilizing a large volume of heterogeneous electronic health records for the application of MACE prediction. In detail, we address the multi-subtype-oriented MACE prediction for ACS as a multi-task learning (MTL) problem, present a MTL-based model to predict MACE of ACS patients with the different subtypes, and incorporate adversarial learning into the model to alleviate both the shared and private latent feature spaces of each subtype of ACS from interfering with each other. A real clinical dataset containing 2,863 ACS patient samples is collected from a Chinese hospital to validate the proposed model. Experimental results demonstrate that the prediction performance of our proposed model obtains a significant improvement, compared to single-subtype-oriented MACE prediction models.

Tensor-based Low-dimensional Representation Learning for Multi-view Clustering.

With the development of data collection techniques, multi-view clustering becomes an emerging research direction to improve the clustering performance. Existing work has shown that leveraging multi-view information is able to provide a rich and comprehensive description. One of the core problems is how to sufficiently represent multi-view data in the analysis. In this paper, we introduce a tensor-based Representation Learning method for Multi-view Clustering (tRLMvC) that can unify heterogenous and high-dimensional multi-view feature spaces to a low-dimensional shared latent feature space and improve multi-view clustering performance. To sufficiently capture plenty multi-view information, tRLMvC represents multi-view data as a third-order tensor, expresses each tensorial data point as a sparse t-linear combination of all data points with t-product, and constructs a self-expressive tensor through reconstruction coefficients. The low-dimensional multi-view data representation in the shared latent feature space can be obtained via Tucker decomposition on the self-expressive tensor. These two parts are iteratively performed so that the interaction between selfexpressive tensor learning and its factorization can be enhanced and the new representation can be effectively generated for clustering purpose. We conduct extensive experiments on eight multi-view data sets and compare the proposed model with the state-of-the-art methods. Experimental results have shown that tRLMvC outperforms the baselines in terms of various evaluation metrics.

Software defect prediction based on kernel PCA and weighted extreme learning machine

ContextSoftware defect prediction strives to detect defect-prone software modules by mining the historical data. Effective prediction enables reasonable testing resource allocation, which eventually leads to a more reliable software. ObjectiveThe complex structures and the imbalanced class distribution in software defect data make it challenging to obtain suitable data features and learn an effective defect prediction model. In this paper, we propose a method to address these two challenges. MethodWe propose a defect prediction framework called KPWE that combines two techniques, i.e., Kernel Principal Component Analysis (KPCA) and Weighted Extreme Learning Machine (WELM). Our framework consists of two major stages. In the first stage, KPWE aims to extract representative data features. It leverages the KPCA technique to project the original data into a latent feature space by nonlinear mapping. In the second stage, KPWE aims to alleviate the class imbalance. It exploits the WELM technique to learn an effective defect prediction model with a weighting-based scheme. ResultsWe have conducted extensive experiments on 34 projects from the PROMISE dataset and 10 projects from the NASA dataset. The experimental results show that KPWE achieves promising performance compared with 41 baseline methods, including seven basic classifiers with KPCA, five variants of KPWE, eight representative feature selection methods with WELM, 21 imbalanced learning methods. ConclusionIn this paper, we propose KPWE, a new software defect prediction framework that considers the feature extraction and class imbalance issues. The empirical study on 44 software projects indicate that KPWE is superior to the baseline methods in most cases.

Domain Invariant and Class Discriminative Feature Learning for Visual Domain Adaptation.

Domain adaptation manages to build an effective target classifier or regression model for unlabeled target data by utilizing the well-labeled source data but lying different distributions. Intuitively, to address domain shift problem, it is crucial to learn domain invariant features across domains, and most existing approaches have concentrated on it. However, they often do not directly constrain the learned features to be class discriminative for both source and target data, which is of vital importance for the final classification. Therefore, in this paper, we put forward a novel feature learning method for domain adaptation to construct both domain invariant and class discriminative representations, referred to as DICD. Specifically, DICD is to learn a latent feature space with important data properties preserved, which reduces the domain difference by jointly matching the marginal and class-conditional distributions of both domains, and simultaneously maximizes the inter-class dispersion and minimizes the intra-class scatter as much as possible. Experiments in this paper have demonstrated that the class discriminative properties will dramatically alleviate the cross-domain distribution inconsistency, which further boosts the classification performance. Moreover, we show that exploring both domain invariance and class discriminativeness of the learned representations can be integrated into one optimization framework, and the optimal solution can be derived effectively by solving a generalized eigen-decomposition problem. Comprehensive experiments on several visual cross-domain classification tasks verify that DICD can outperform the competitors significantly.

Multi-view predictive latent space learning

In unsupervised circumstances, multi-view learning seeks a shared latent representation by taking the consensus and complementary principles into account. However, most existing multi-view unsupervised learning approaches do not explicitly lay stress on the predictability of the latent space. In this paper, we propose a novel multi-view predictive latent space learning (MVP) model and apply it to multi-view clustering and unsupervised dimension reduction. The latent space is forced to be predictive by maximizing the correlation between the latent space and feature space of each view. By learning a multi-view graph with adaptive view-weight learning, MVP effectively combines the complementary information from multi-view data. Experimental results on benchmark datasets show that MVP outperforms the state-of-the-art multi-view clustering and unsupervised dimension reduction algorithms.

Scalable non-negative matrix tri-factorization

BackgroundMatrix factorization is a well established pattern discovery tool that has seen numerous applications in biomedical data analytics, such as gene expression co-clustering, patient stratification, and gene-disease association mining. Matrix factorization learns a latent data model that takes a data matrix and transforms it into a latent feature space enabling generalization, noise removal and feature discovery. However, factorization algorithms are numerically intensive, and hence there is a pressing challenge to scale current algorithms to work with large datasets. Our focus in this paper is matrix tri-factorization, a popular method that is not limited by the assumption of standard matrix factorization about data residing in one latent space. Matrix tri-factorization solves this by inferring a separate latent space for each dimension in a data matrix, and a latent mapping of interactions between the inferred spaces, making the approach particularly suitable for biomedical data mining.ResultsWe developed a block-wise approach for latent factor learning in matrix tri-factorization. The approach partitions a data matrix into disjoint submatrices that are treated independently and fed into a parallel factorization system. An appealing property of the proposed approach is its mathematical equivalence with serial matrix tri-factorization. In a study on large biomedical datasets we show that our approach scales well on multi-processor and multi-GPU architectures. On a four-GPU system we demonstrate that our approach can be more than 100-times faster than its single-processor counterpart.ConclusionsA general approach for scaling non-negative matrix tri-factorization is proposed. The approach is especially useful parallel matrix factorization implemented in a multi-GPU environment. We expect the new approach will be useful in emerging procedures for latent factor analysis, notably for data integration, where many large data matrices need to be collectively factorized.

Personalized clothing recommendation combining user social circle and fashion style consistency

With the rapid expansion of social networks and fashion websites, clothing recommendation has attracted more attention of researchers, since various web data bring opportunities for recommender systems to solve the problems of cold start and sparsity. For clothing recommender system, user social circle and fashion style consistency of clothing items are two important factors, which have critical impacts on user decision making. In this paper, two practical problems are considered: how to visually analyze fashion style consistency between clothing items and how to implement personalized clothing recommendation by combining user social circle and fashion style consistency. To address the first problem, a Siamese Convolutional Neural Network (SCNN) with a novel sampling strategy is employed to measure the fashion style consistency of clothing items. It can learn a feature transformation from clothing images to a latent feature space, where the representations of clothing items with similar styles locate closer than those with different styles. For the second problem, three social factors (i.e., personal interest, interpersonal interest similarity and interpersonal influence) and fashion style consistency are fused into a unified personalized recommendation model based on probabilistic matrix factorization (PMF). To comprehensively evaluate our model, extensive experiments have been conducted on two real world datasets collected from a popular social fashion website, which demonstrate the effectiveness of the proposed method for personalized clothing recommendation.

Discriminative Deep Metric Learning for Face and Kinship Verification.

This paper presents a new discriminative deep metric learning (DDML) method for face and kinship verification in wild conditions. While metric learning has achieved reasonably good performance in face and kinship verification, most existing metric learning methods aim to learn a single Mahalanobis distance metric to maximize the inter-class variations and minimize the intra-class variations, which cannot capture the nonlinear manifold where face images usually lie on. To address this, we propose a DDML method to train a deep neural network to learn a set of hierarchical nonlinear transformations to project face pairs into the same latent feature space, under which the distance of each positive pair is reduced and that of each negative pair is enlarged. To better use the commonality of multiple feature descriptors to make all the features more robust for face and kinship verification, we develop a discriminative deep multi-metric learning method to jointly learn multiple neural networks, under which the correlation of different features of each sample is maximized, and the distance of each positive pair is reduced and that of each negative pair is enlarged. Extensive experimental results show that our proposed methods achieve the acceptable results in both face and kinship verification.

Characterizing Dynamic Walking Patterns and Detecting Falls with Wearable Sensors Using Gaussian Process Methods.

By incorporating a growing number of sensors and adopting machine learning technologies, wearable devices have recently become a prominent health care application domain. Among the related research topics in this field, one of the most important issues is detecting falls while walking. Since such falls may lead to serious injuries, automatically and promptly detecting them during daily use of smartphones and/or smart watches is a particular need. In this paper, we investigate the use of Gaussian process (GP) methods for characterizing dynamic walking patterns and detecting falls while walking with built-in wearable sensors in smartphones and/or smartwatches. For the task of characterizing dynamic walking patterns in a low-dimensional latent feature space, we propose a novel approach called auto-encoded Gaussian process dynamical model, in which we combine a GP-based state space modeling method with a nonlinear dimensionality reduction method in a unique manner. The Gaussian process methods are fit for this task because one of the most import strengths of the Gaussian process methods is its capability of handling uncertainty in the model parameters. Also for detecting falls while walking, we propose to recycle the latent samples generated in training the auto-encoded Gaussian process dynamical model for GP-based novelty detection, which can lead to an efficient and seamless solution to the detection task. Experimental results show that the combined use of these GP-based methods can yield promising results for characterizing dynamic walking patterns and detecting falls while walking with the wearable sensors.

Towards Context-aware Social Recommendation via Individual Trust

Incorporating social network information and contexts to improve recommendation performance has been drawing considerable attention recently. However, a majority of existing social recommendation approaches suffer from the following problems: (1) They only employ individual trust among users to optimize prediction solutions in user latent feature space or in user-item rating space; thus, they exhibit low recommendation accuracy. (2) They use decision trees to perform context-based user-item subgrouping; thus, they can only handle categorical contexts. (3) They have difficulty coping with the data sparsity problem. To solve these problems, and accurately and realistically model recommender systems, we propose a social matrix factorization method to optimize the prediction solution in both user latent feature space and user-item rating space using the individual trust among users. To further improve the recommendation performance and alleviate the data sparsity problem, we propose a context-aware enhanced model based on Gaussian mixture model (GMM). Two real datasets (one sparse dataset and one dense dataset) based experiments show that our proposed method outperforms the state-of-the-art social matrix factorization and context-aware recommendation methods in terms of prediction accuracy.

Social Collaborative Filtering by Trust.

Recommender systems are used to accurately and actively provide users with potentially interesting information or services. Collaborative filtering is a widely adopted approach to recommendation, but sparse data and cold-start users are often barriers to providing high quality recommendations. To address such issues, we propose a novel method that works to improve the performance of collaborative filtering recommendations by integrating sparse rating data given by users and sparse social trust network among these same users. This is a model-based method that adopts matrix factorization technique that maps users into low-dimensional latent feature spaces in terms of their trust relationship, and aims to more accurately reflect the users reciprocal influence on the formation of their own opinions and to learn better preferential patterns of users for high-quality recommendations. We use four large-scale datasets to show that the proposed method performs much better, especially for cold start users, than state-of-the-art recommendation algorithms for social collaborative filtering based on trust.

Latent codebook regression for facial feature localisation

A new non-parametric inferring procedure of facial feature localisation which learns regression in latent variable space is introduced. The proposed method uses regression between latent feature and motion spaces spanned by a set of bases of nonlinear feature space and motion vector space which is trained from a large sample set in codebook. Compared with the previous method of using codebook or eigen-codebook, the proposed method achieved both a significant reduction in the memory consumption without loss of accuracy and a very low computational complexity enough to run in mobile devices.

Domain adaptation via Multi-Layer Transfer Learning

Transfer learning, which leverages labeled data in a source domain to train an accurate classifier for classification tasks in a target domain, has attracted extensive research interests recently for its effectiveness proven by many studies. Previous approaches adopt a common strategy that models the shared structure as a bridge across different domains by reducing distribution divergences. However, those approaches totally ignore specific latent spaces, which can be utilized to learn non-shared concepts. Only specific latent spaces contain specific latent factors, lacking which will lead to ineffective distinct concept learning. Additionally, only learning latent factors in one latent feature space layer may ignore those in the other layers. The missing latent factors may also help us to model the latent structure shared as the bridge. This paper proposes a novel transfer learning method Multi-Layer Transfer Learning (MLTL). MLTL first generates specific latent feature spaces. Second, it combines these specific latent feature spaces with common latent feature space into one latent feature space layer. Third, it generates multiple layers to learn the corresponding distributions on different layers with their pluralism simultaneously. Specifically, the pluralism of the distributions on different layers means that learning the distributions on one layer can help us to learn the distributions on the others. Furthermore, an iterative algorithm based on Non-Negative Matrix Tri-Factorization is proposed to solve the optimization problem. Comprehensive experiments demonstrate that MLTL can significantly outperform the state-of-the-art learning methods on topic and sentiment classification tasks.

A multi-ATL method for transfer learning across multiple domains with arbitrarily different distribution

This paper addresses a novel learning problem where data from multiple domains are generated from distributions that are allowed to be arbitrarily different. Domain adaptation and transfer learning have been addressed previously for two domains with different distributions, but based on the assumption that the distributions are sufficiently related to each other. However, in many real-world applications, multiple domains may have arbitrarily different distributions. In this paper, a general framework is proposed, called Multi-ATL, for bridging knowledge from multiple domains with arbitrarily different distributions. The proposed framework consists of three key components: latent feature space extraction, co-training with multiple views, and active sample selection. The basic concept is to explore each domain and characterize it with an extracted latent feature space, and then apply a co-training algorithm to the latent spaces to use multiple views simultaneously for training. In the co-training process, active sample selection is used to determine how much knowledge should be transferred from a source domain to the target domain. Experimental results on one synthetic and two real-world datasets show that the proposed framework significantly improves classification accuracy (13.5% on the synthetic and +17.79%, +23.87% on the real datasets) with only a few (2–5) active selections.

Transferring Compressive-Sensing-Based Device-Free Localization Across Target Diversity

Device-free localization (DFL) plays an important role in many applications, such as wildlife population and migration tracking. Most of current DFL systems leverage the distorted received signal strength (RSS) changes to localize the target(s). However, they assume a fixed distribution of the RSS change measurements, although they are distorted by different types of targets. It inevitably causes the localization to fail if the targets for modeling and testing belong to different categories. This paper presents TLCS-a transferring compressive sensing based DFL approach-which employs a rigorously designed transferring function to transfer the distorted RSS changes across different categories of targets into a latent feature space, where the distributions of the distorted RSS change measurements from different categories of targets are unified. A benefit of this approach is that the same transferred sensing matrix can be shared by different categories of targets, leading to a substantial reduction in the human efforts. The results of experiments illustrate the efficacy of the TLCS.

Model selection of extreme learning machine based on latent feature space

Recently,Extreme Learning Machine(ELM) has been a promising tool in solving a wide range of classification and regression problems.However,the generalization performance of ELM will be decreased when there exits redundant hidden neurons.To solve this problem,this paper introduced a new regularizer that was the Frobenius norm of mapping matrix from hidden space to a new latent feature space.Furthermore,an alternating optimization strategy was adopted to learn the above regularization problem and the latent feature space.The proposed algorithm was tested empirically on the classical UCI data set as well as a load identification engineering data set.The experimental results show that the proposed algorithm obviously outperforms the classical ELM in terms of predictive precision and numerical stability,and needs much less computational cost than the present ELM model selection algorithm.

A new temporal and social PMF-based method to predict users' interests in micro-blogging

Micro-blogging is becoming an increasingly popular social media platform where users can discover interesting information about the real world and especially corporations are able to understand customers' demands. The fast diffusion of information and the convenience of micro-blogging have resulted in large audiences sharing their daily activities, exchanging opinions and establishing friendships with others. By analyzing the user-generated contents, one can explore users' potential interests, which helps micro-blogging provide users with better personalized information services. Users' behaviors are affected by opinions of their friends and changes in their interests over time. Based on these intuitions, in this paper we propose a temporal and social probabilistic matrix factorization model to predict users' potential interests in micro-blogging. By exploiting the matrix factorization technique to learn latent features of users and topics, our model analyzes the impacts of time information and users' activities, including posting of tweets and establishing friendships with others, on the latent feature space of users and topics of their interests. The proposed model provides a unified way to fuse the time information and the social network structure to predict users' future interests accurately. The experimental results on Sina-weibo, one of the most popular micro-blogging sites in China, demonstrate the efficiency and effectiveness of our proposed model.

Nonlinear matrix factorization with unified embedding for social tag relevance learning

With the proliferation of social images, social image tagging is an essential issue for text-based social image retrieval. However, the original tags annotated by web users are always noisy, irrelevant and incomplete to interpret the image visual contents. In this paper, we propose a nonlinear matrix factorization method with the priors of inter- and intra-correlations among images and tags to effectively predict the tag relevance to the visual contents. In the proposed method, we attempt to discover the image latent feature space and the tag latent feature space in a unified space, that is, each image or each tag can be described as a point in the unified space. Intuitively, it is more understandable to estimate the relationships between images and tags directly based on their distances or similarities in the unified space. Thus, the task of image tagging or tag recommendation can be efficiently solved by the nearest tag-neighbors search in the unified space. Similarly, we can obtain the top relevant images corresponding to any tag so as to perform the task of image search by keywords. We investigate the performance of the proposed method on tag recommendation and image search respectively and compare to existing work on the challenging NUS-WIDE dataset. Extensive experiments demonstrate the effectiveness and potentials of the proposed method in real-world applications.

Multiview Metric Learning with Global Consistency and Local Smoothness

In many real-world applications, the same object may have different observations (or descriptions) from multiview observation spaces, which are highly related but sometimes look different from each other. Conventional metric-learning methods achieve satisfactory performance on distance metric computation of data in a single-view observation space, but fail to handle well data sampled from multiview observation spaces, especially those with highly nonlinear structure. To tackle this problem, we propose a new method called Multiview Metric Learning with Global consistency and Local smoothness (MVML-GL) under a semisupervised learning setting, which jointly considers global consistency and local smoothness. The basic idea is to reveal the shared latent feature space of the multiview observations by embodying global consistency constraints and preserving local geometric structures. Specifically, this framework is composed of two main steps. In the first step, we seek a global consistent shared latent feature space, which not only preserves the local geometric structure in each space but also makes those labeled corresponding instances as close as possible. In the second step, the explicit mapping functions between the input spaces and the shared latent space are learned via regularized locally linear regression. Furthermore, these two steps both can be solved by convex optimizations in closed form. Experimental results with application to manifold alignment on real-world datasets of pose and facial expression demonstrate the effectiveness of the proposed method.