Local Manifold Structure Of Data Research Articles

A latent representation dual manifold regularization broad learning system with incremental learning capability for fault diagnosis

Fault diagnosis models based on deep learning must spend a lot of time adjusting the model structure and parameters for retraining upon the occurrence of a new fault. To address this problem, a latent representation dual manifold regularization broad learning system (LRDMR-BLS) with incremental learning capability is proposed for fault diagnosis. The model uses the link information between data to guide feature selection via latent representation learning. Meanwhile, two manifold regularization terms are added to the objective function of latent representation learning and the objective function of BLS to maintain the local manifold structure of data and feature spaces. Finally, the incremental learning capability of the proposed model enables the proposed model to be updated quickly when a new fault occurs. The superiority of the proposed model is demonstrated by two chemical processes.

Adaptive Manifold Graph representation for Two-Dimensional Discriminant Projection

Two-dimensional (2D) local discriminant analysis is one of the popular techniques for image representation and recognition. Conventional 2D methods extract features of images relying on the affinity relationships among samples. However, affinity graph learning and dimensionality reduction are two separate processes, and the predefined affinity graph is sensitive to noisy and irrelevant features. To this end, we propose a graph embedding model, named as Two-Dimensional Discriminant Projection with Adaptive Manifold Graph (2DDP-AMG). In our model, the local manifold structure of data is characterized by the k-nearest-neighbor (kNN) graph within each class. More importantly, the above graph is updated adaptively in the process of dimensionality reduction to eliminate the interference of irrelevant features and noise. Furthermore, F-norm is introduced to measure the distance between homogeneous samples and improve the robustness of the model to outliers. To solve this model, an efficient algorithm is developed to update each variable iteratively. Extensive experiments conducted on benchmark face image data sets show the superiority and robustness of our model compared to state-of-the-art alternatives.

Robust unsupervised feature selection via sparse and minimum-redundant subspace learning with dual regularization

Unsupervised feature selection, one of the important dimensionality reduction methods for high-dimensional data, has been paid more and more attention by researchers due to its effectiveness of processing the data without labels. However, most unsupervised feature selection methods do not take into account the fact that real data tends to be mixed with noise or outliers, or only consider the sensitivity of the model to them. In addition, the local manifold structure of data is usually preserved by a simple graph based on fixed k nearest neighbors. On the one hand, the different sample distributions of different data are considered without distinction. On the other hand, simple graph can not well capture the inherent relations between samples when facing multi-modal data. In view of the above problems, a robust unsupervised feature selection algorithm, sparse and minimum-redundant subspace learning with dual regularization (SMRSDR), is proposed in this paper. Specifically, a self-paced learning regularization based on soft weight allocation is introduced into SMRSDR to strictly control the samples’ entry into model learning. Normal samples are allowed to enter model learning in order of their importance, while noise or outliers are strictly blocked out of model learning. Besides, considering the difference in sample distribution and the multimodal nature of data, the traditional simple graph Laplacian regularization is replaced by the hypergraph Laplacian regularization based on adaptive nearest neighbors selection. Furthermore, the sparsity and minimum-redundancy constraints are imposed on the subspace learning framework so that the most representative and minimum-redundant features can be selected. Finally, the objective function of SMRSDR is solved by an alternating iterative optimization algorithm. And a series of experiments are performed to comprehensively prove the effectiveness and superiority of SMRSDR.

Multiview Common Subspace Clustering via Coupled Low Rank Representation

Multi-view subspace clustering (MVSC) finds a shared structure in latent low-dimensional subspaces of multi-view data to enhance clustering performance. Nonetheless, we observe that most existing MVSC methods neglect the diversity in multi-view data by considering only the common knowledge to find a shared structure either directly or by merging different similarity matrices learned for each view. In the presence of noise, this predefined shared structure becomes a biased representation of the different views. Thus, in this article, we propose a MVSC method based on coupled low-rank representation to address the above limitation. Our method first obtains a low-rank representation for each view, constrained to be a linear combination of the view-specific representation and the shared representation by simultaneously encouraging the sparsity of view-specific one. Then, it uses the k -block diagonal regularizer to learn a manifold recovery matrix for each view through respective low-rank matrices to recover more manifold structures from them. In this way, the proposed method can find an ideal similarity matrix by approximating clustering projection matrices obtained from the recovery structures. Hence, this similarity matrix denotes our clustering structure with exactly k connected components by applying a rank constraint on the similarity matrix’s relaxed Laplacian matrix to avoid spectral post-processing of the low-dimensional embedding matrix. The core of our idea is such that we introduce dynamic approximation into the low-rank representation to allow the clustering structure and the shared representation to guide each other to learn cleaner low-rank matrices that would lead to a better clustering structure. Therefore, our approach is notably different from existing methods in which the local manifold structure of data is captured in advance. Extensive experiments on six benchmark datasets show that our method outperforms 10 similar state-of-the-art compared methods in six evaluation metrics.

Unsupervised Feature Selection With Constrained ℓ₂,₀-Norm and Optimized Graph.

In this article, we propose a novel feature selection approach, named unsupervised feature selection with constrained l2,0 -norm (row-sparsity constrained) and optimized graph (RSOGFS), which unifies feature selection and similarity matrix construction into a general framework instead of independently performing the two-stage process; thus, the similarity matrix preserving the local manifold structure of data can be determined adaptively. Unlike those sparse learning-based feature selection methods that can only solve the relaxation or approximation problems by introducing sparsity regularization term into the objective function, the proposed method directly tackles the original l2,0 -norm constrained problem to achieve group feature selection. Two optimization strategies are provided to solve the original sparse constrained problem. The convergence and approximation guarantees for the new algorithms are rigorously proved, and the computational complexity and parameter determination are theoretically analyzed. Experimental results on real-world data sets show that the proposed method for solving a nonconvex problem is superior to the state of the arts for solving the relaxed or approximate convex problems.

Graph regularized low-rank representation for submodule clustering

In this paper, a new submodule clustering method for imaging (2-D) data is proposed. Unlike most existing clustering methods that first convert such data into vectors as preprocessing, the proposed method arranges the data samples as lateral slices of a third-order tensor. Our algorithm is based on the union-of-free-submodules model and the samples are represented using t-product in the third-order tensor space. First, we impose a low-rank constraint on the representation tensor to capture the principle information of data. By incorporating manifold regularization into the tensor factorization, the proposed method explicitly exploits the local manifold structure of data. Meanwhile, a segmentation dependent term is employed to integrate the two pipeline steps of affinity learning and spectral clustering into a unified optimization framework. The proposed method can be efficiently solved based on the alternating direction method of multipliers and spectral clustering. Finally, a nonlinear extension is proposed to handle data drawn from a mixture of nonlinear manifolds. Extensive experimental results on five real-world image datasets confirm the effectiveness of the proposed methods.

Two-dimensional locality adaptive discriminant analysis

Two-dimensional Linear Discriminant Analysis (2DLDA), which is supervised and extracts the most discriminating features, has been widely used in face image representation and recognition. However, 2DLDA is inapplicable to many real-world situations because it assumes that the input data obeys the Gaussian distribution and emphasizes the global relationship of data merely. To handle this problem, we present a Two-dimensional Locality Adaptive Discriminant Analysis (2DLADA). Compared to 2DLDA, our method has two salient advantages: (1) it does not depend on any assumptions on the data distribution and is more suitable in real world applications; (2) it adaptively exploits the intrinsic local structure of data manifold. Performance on artificial dataset and real-world datasets demonstrate the superiority of our proposed method.

Discriminative Transfer Joint Matching for Domain Adaptation in Hyperspectral Image Classification

Domain adaptation, which aims at learning an accurate classifier for a new domain (target domain) using labeled information from an old domain (source domain), has shown promising value in remote sensing fields yet still been a challenging problem. In this letter, we focus on knowledge transfer between hyperspectral remotely sensed images in the context of land-cover classification under unsupervised setting where labeled samples are available only for the source image. Specifically, a discriminative transfer joint matching (DTJM) method is proposed, which matches source and target features in the kernel principal component analysis space by minimizing the empirical maximum mean discrepancy, performs instance reweighting by imposing an $\ell _{2,1}$ -norm on the embedding matrix, and preserves the local manifold structure of data from different domains and meanwhile maximizes the dependence between the embedding and labels. The proposed approach is compared with some state-of-the-art feature extraction techniques with and without using label information of source data. Experimental results on two benchmark hypersepctral data sets show the effectiveness of the proposed DTJM.

Locality Adaptive Discriminant Analysis for Spectral–Spatial Classification of Hyperspectral Images

Linear discriminant analysis (LDA) is a popular technique for supervised dimensionality reduction, but with less concern about a local data structure. This makes LDA inapplicable to many real-world situations, such as hyperspectral image (HSI) classification. In this letter, we propose a novel dimensionality reduction algorithm, locality adaptive discriminant analysis (LADA) for HSI classification. The proposed algorithm aims to learn a representative subspace of data, and focuses on the data points with close relationship in spectral and spatial domains. An intuitive motivation is that data points of the same class have similar spectral feature and the data points among spatial neighborhood are usually associated with the same class. Compared with traditional LDA and its variants, LADA is able to adaptively exploit the local manifold structure of data. Experiments carried out on several real hyperspectral data sets demonstrate the effectiveness of the proposed method.

Graph Regularized Restricted Boltzmann Machine.

The restricted Boltzmann machine (RBM) has received an increasing amount of interest in recent years. It determines good mapping weights that capture useful latent features in an unsupervised manner. The RBM and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks. However, most of the existing RBM-based models disregard the preservation of the data manifold structure. In many real applications, the data generally reside on a low-dimensional manifold embedded in high-dimensional ambient space. In this brief, we propose a novel graph regularized RBM to capture features and learning representations, explicitly considering the local manifold structure of the data. By imposing manifold-based locality that preserves constraints on the hidden layer of the RBM, the model ultimately learns sparse and discriminative representations. The representations can reflect data distributions while simultaneously preserving the local manifold structure of data. We test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem. The results demonstrate that the performance of our method exceeds the state-of-the-art alternatives.

Locality-constrained nonnegative robust shape interaction subspace clustering and its applications

In this paper, we present a locality-constrained nonnegative robust shape interaction (LNRSI) subspace clustering method. LNRSI integrates the local manifold structure of data into the robust shape interaction (RSI) in a unified formulation, which guarantees the locality and the low-rank property of the optimal affinity graph. Compared with traditional low-rank representation (LRR) learning method, LNRSI can not only pursuit the global structure of data space by low-rank regularization, but also keep the locality manifold, which leads to a sparse and low-rank affinity graph. Due to the clear block-diagonal effect of the affinity graph, LNRSI is robust to noise and occlusions, and achieves a higher rate of correct clustering. The theoretical analysis of the clustering effect is also discussed. An efficient solution based on linearized alternating direction method with adaptive penalty (LADMAP) is built for our method. Finally, we evaluate the performance of LNRSI on both synthetic data and real computer vision tasks, i.e., motion segmentation and handwritten digit clustering. The experimental results show that our LNRSI outperforms several state-of-the-art algorithms.

Rolling Elements Bearings Degradation Indicator Based on Continuous Hidden Markov Model

In order to obtain a sensitive bearing degradation indicator with significant trend, a new approach is proposed. For the capability of preserving the local structure of data manifold, locality preserving projections are used for features dimension reduction without missing the original information. Since continuous hidden Markov model (CHMM) can take multiple fault features into consideration and also capture the stochastic characteristics of fault features in the time domain, CHMM-based negative log likelihood probability is built as a new health assessment indication. The experiment results show that, compared with the general features, the proposed degradation indicator shows great trend of the bearing health and is more sensitive to incipient defects.

What regularized auto-encoders learn from the data-generating distribution

What do auto-encoders learn about the underlying data-generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. ...

Hybrid structure for robust dimensionality reduction

In recent years, dimensionality reduction has attracted a great deal of attention in the communities of machine learning and data mining. The basic goal of dimensionality reduction is to discover the low dimensional manifold embedded in a high dimensional space. Although some existing manifold learning algorithms (ISOMAP, LE, LLE, LTSA, etc.) can capture the local structure of data manifold, they have poor performance in some recognition tasks. This is mainly because that they cannot handle well with the “out of sample” problem. Moreover, these algorithms are sensitive to the choice of nearest neighbors, which is crucial in classification. To address these problems, this paper proposes a Robust Dimensionality Reduction Algorithm With Local and Global Structure (RLGS) based on a novel adaptive weighting mechanism. Hybrid structure of local and global structures is studied. By using the adaptive weight, RLGS has the capacity of adaptively exploiting non-linear structure of data manifold and is robust to parameters. Experiments demonstrate that RLGS performs better on public face databases compared with other reported algorithms.

Ordinal regularized manifold feature extraction for image ranking

This paper proposes a novel feature extraction algorithm specifically designed for learning to rank in image ranking. Different from the previous works, the proposed method not only targets at preserving the local manifold structure of data, but also keeps the ordinal information among different data blocks in the low-dimensional subspace, where a ranking model can be learned effectively and efficiently. We first define the ideal directions of preserving local manifold structure and ordinal information, respectively. Based on the two definitions, a unified model is built to leverage the two kinds of information, which is formulated as an optimization problem. The experiments are conducted on two public available data sets: the MSRA-MM image data set and the “Web Queries” image data set, and the experimental results demonstrate the power of the proposed method against the state-of-the-art methods.