Traditional Manifold Learning Methods Research Articles

Supervised Manifold Learning Based on Multi-Feature Information Discriminative Fusion within an Adaptive Nearest Neighbor Strategy Applied to Rolling Bearing Fault Diagnosis.

Rolling bearings are a key component for ensuring the safe and smooth operation of rotating machinery and are very prone to failure. Therefore, intelligent fault diagnosis research on rolling bearings has become a crucial task in the field of mechanical fault diagnosis. This paper proposes research on the fault diagnosis of rolling bearings based on an adaptive nearest neighbor strategy and the discriminative fusion of multi-feature information using supervised manifold learning (AN-MFIDFS-Isomap). Firstly, an adaptive nearest neighbor strategy is proposed using the Euclidean distance and cosine similarity to optimize the selection of neighboring points. Secondly, three feature space transformation and feature information extraction methods are proposed, among which an innovative exponential linear kernel function is introduced to provide new feature information descriptions for the data, enhancing feature sensitivity. Finally, under the adaptive nearest neighbor strategy, a novel AN-MFIDFS-Isomap algorithm is proposed for rolling bearing fault diagnosis by fusing various feature information and classifiers through discriminative fusion with label information. The proposed AN-MFIDFS-Isomap algorithm is validated on the CWRU open dataset and our experimental dataset. The experiments show that the proposed method outperforms other traditional manifold learning methods in terms of data clustering and fault diagnosis.

Multi-manifold discriminant local spline embedding

Manifold learning reveals the intrinsic low-dimensional manifold structure of high-dimensional data and has achieved great success in a wide spectrum of applications. However, traditional manifold learning methods assume that all the data lie on a common manifold, hence fail to capture the complicated geometry structure of the real-world data lying on multiple manifolds. This paper proposes a novel Multi-manifold Discriminant Local Spline Embedding (MDLSE) algorithm for high-dimensional classification, which considers a more realistic scenario where data of the same class lies on the same manifold. On the basis of this assumption, MDLSE seeks to reconstruct multiple manifolds for different classes of data in the embedding and separate them as apart as possible. In order to preserve the geometry structure of all the manifolds, MDLSE employs thin plate splines to align the local patches within each manifold compatibly in the global embedding. Meanwhile, to separate the different manifolds, MDLSE utilizes discriminative information to ensure the neighboring data from different manifolds to be mapped far from each other. Extensive experiments on both synthetic and real-world datasets demonstrate the superiority of MDLSE over the other representative manifold learning algorithms. The advantage of MDLSE is often more obvious on smaller size of training data and in lower embedding dimensions.

EEG Signal Classification Using Manifold Learning and Matrix-Variate Gaussian Model

In brain-computer interface (BCI), feature extraction is the key to the accuracy of recognition. There is important local structural information in the EEG signals, which is effective for classification; and this locality of EEG features not only exists in the spatial channel position but also exists in the frequency domain. In order to retain sufficient spatial structure and frequency information, we use one-versus-rest filter bank common spatial patterns (OVR-FBCSP) to preprocess the data and extract preliminary features. On this basis, we conduct research and discussion on feature extraction methods. One-dimensional feature extraction methods like linear discriminant analysis (LDA) may destroy this kind of structural information. Traditional manifold learning methods or two-dimensional feature extraction methods cannot extract both types of information at the same time. We introduced the bilinear structure and matrix-variate Gaussian model into two-dimensional discriminant locality preserving projection (2DDLPP) algorithm and decompose EEG signals into spatial and spectral parts. Afterwards, the most discriminative features were selected through a weight calculation method. We tested the method on BCI competition data sets 2a, data sets IIIa, and data sets collected by our laboratory, and the results were expressed in terms of recognition accuracy. The cross-validation results were 75.69%, 70.46%, and 54.49%, respectively. The average recognition accuracy of new method is improved by 7.14%, 7.38%, 4.86%, and 3.8% compared to those of LDA, two-dimensional linear discriminant analysis (2DLDA), discriminant locality property projections (DLPP), and 2DDLPP, respectively. Therefore, we consider that the proposed method is effective for EEG classification.

Hyperspectral Image Dimension Reduction and Target Detection Based on Weighted Mean Filter and Manifold Learning

Hyperspectral images have many bands, resulting in a high data volume, which complicates subsequent data processing. Using manifold learning methods to reduce the dimension of data is conducive for subsequent use. However, traditional manifold learning methods are easily disturbed by spectral uncertainty, and are primarily used for hyperspectral image classification. This paper proposes an improved manifold reconstruction preserving embedding algorithm based on weighted mean filter (WMF-IMRPE), that is not easily affected by spectral uncertainty and has excellent target detection performance. The original hyperspectral image is processed by the weighted mean filter to eliminate the influence of noise and reduce spectral differences between the homogeneous ground objects. The spectral angular distance then replaces the Euclidean distance in the original MRPE algorithm to select neighbourhood pixels, reducing spectral uncertainty interference. The experimental results show that the low dimensional features extracted by the WMF-IMRPE algorithm have better distinguishability, and the algorithm further improves the target detection accuracy. The WMF-IMRPE algorithm’s hyperspectral image target detection performance is superior to other similar algorithms.

Discriminative sparse embedding based on adaptive graph for dimension reduction

The traditional manifold learning methods usually utilize the original observed data to directly define the intrinsic structure among data. Because the original samples often contain a deal of redundant information or it is corrupted by noises, it leads to the unreliability of the obtained intrinsic structure. In addition, the intrinsic structure learning and subspace learning are completely separated. For solving above problems, this paper presents a novel dimension reduction method termed discriminative sparse embedding (DSE) based on adaptive graph. By projecting the original samples into a low-dimensional subspace, DSE learns a sparse weight matrix, which can reduce the effects of redundant information and noises of the original data, and uncover essential structural relationship among the data. In DSE, the robust subspace is learned from the original data. Meanwhile, the intrinsic local structure and the optimal subspace can be simultaneously learned, in which they are mutually improved, and the accurate structure can be captured, and the optimal subspace can be obtained. We propose an alternative and iterative method to solve the DSE model. In order to evaluate the performance of DSE, it is compared with some state-of-the-art feature extraction algorithms. Various experiments show that our DSE is effective and feasible.

Self-adaptive manifold discriminant analysis for feature extraction from hyperspectral imagery

Traditional manifold learning methods generally include a single projection stage that maps high-dimensional data into lower-dimensional space. However, these methods cannot guarantee that the projection matrix is optimal for classification, which limits their practical application. To address this issue, we propose a two-stage projection matrix optimization model termed self-adaptive manifold discriminant analysis (SAMDA). In pre-training projection stage, SAMDA obtains an initial projection matrix by constructing an interclass graph and an intraclass graph under the graph embedding (GE) framework. In weight optimization stage, a maximal manifold margin criterion is developed to further optimize the weights of projection matrix by feature similarity. A self-adaptive optimization process is introduced to increase the margins among different manifolds in low-dimensional space and extract discriminant features that are beneficial to classification. Experimental results on PaviaU, Indian Pines and Heihe data sets demonstrate that the proposed SAMDA method can achieve better classification results than some state-of-the-art methods.

Spatial-Neighborhood Manifold Learning for Nondestructive Testing of Defects in Polymer Composites

The subspace learning (dimensionality reduction) algorithms have played an important role in the analysis of thermographic data: a key step in infrared thermography-based nondestructive testing of subsurface defects in composite materials. However, one of its branches, manifold learning, with excellent ability to preserve local data structure, is rarely applied. In this article, a spatial-neighborhood manifold learning (SNML) framework is proposed for thermographic data analysis. Different from traditional manifold learning methods, SNML uses the spatial-neighborhood information instead of the traditional k-nearest neighbors, or ε-neighborhood, to construct the adjacency graph. This overcomes the difficulty of parameter selection and extracts local features in images in a more reasonable way. Additionally, the data preprocessing step and the means of thermographic data normalization in the proposed framework are discussed. For performance comparison, three traditional manifold learning methods are also implemented. The experiments on carbon fiber-reinforced polymer specimens demonstrate the validity and feasibility of SNML.

A Novel Pre-Processing Technique for Original Feature Matrix of Electronic Nose Based on Supervised Locality Preserving Projections.

An electronic nose (E-nose) consisting of 14 metal oxide gas sensors and one electronic chemical gas sensor has been constructed to identify four different classes of wound infection. However, the classification results of the E-nose are not ideal if the original feature matrix containing the maximum steady-state response value of sensors is processed by the classifier directly, so a novel pre-processing technique based on supervised locality preserving projections (SLPP) is proposed in this paper to process the original feature matrix before it is put into the classifier to improve the performance of the E-nose. SLPP is good at finding and keeping the nonlinear structure of data; furthermore, it can provide an explicit mapping expression which is unreachable by the traditional manifold learning methods. Additionally, some effective optimization methods are found by us to optimize the parameters of SLPP and the classifier. Experimental results prove that the classification accuracy of support vector machine (SVM combined with the data pre-processed by SLPP outperforms other considered methods. All results make it clear that SLPP has a better performance in processing the original feature matrix of the E-nose.

Robust dynamic process monitoring based on sparse representation preserving embedding

In this paper, a novel dimensionality reduction technique, named sparse representation preserving embedding (SRPE), is proposed by utilizing the sparse reconstruction weights and noise-removed data recovered from robust sparse representation. And a new dynamic process monitoring scheme is designed based on SRPE. Different from traditional manifold learning methods, which construct an adjacency graph from K-nearest neighbors or ɛ-ball method, the SRPE algorithm constructs the adjacency graph by solving a robust sparse representation problem through convex optimization. The delicate dynamic relationships between samples are well captured in the sparse reconstructive weights and the error-free data are recovered at the same time. By preserving the sparse weights through linear projection in the clean data space, SRPE is very efficient in detecting dynamic faults and very robust to outliers. Finally, through the case studies of a dynamic numerical example and the Tennessee Eastman (TE) benchmark problem, the superiority of SRPE is verified.

Discriminative orthogonal elastic preserving projections for classification

The traditional manifold learning methods can preserve the local sub-manifold structure or the global geometry effectively, such as elastic preserving projections (EPP). Many experimental results have been shown that EPP, a recently developed linear algorithm, is a strong analyzer for high-dimensional data. However, for classification problems, the traditional methods focused on the geometrical information and ignores discriminative information of different classes. In this paper, we propose a novel discriminative orthogonal elastic preserving projections (DOEPP) by imposing the discriminant information and the orthogonal constraint to improve its classification performance. DOEPP does not only preserve the elasticity of the training set, but also sufficiently utilizes the discriminant information by adding maximum margin criterion and the orthogonality of the projection matrix into its objective function. Extensive experiments on two well-known synthetic manifold data sets and four publicly available databases illustrate the effectiveness of our method.

Supervised locality discriminant manifold learning for head pose estimation

In this paper, we propose a novel supervised manifold learning approach, supervised locality discriminant manifold learning (SLDML), for head pose estimation. Traditional manifold learning methods focus on preserving only the intra-class geometric properties of the manifold embedded in the high-dimensional ambient space, so they cannot fully utilize the underlying discriminative knowledge of the data. The proposed SLDML aims to explore both geometric structure and discriminant information of the data, and yields a smooth and discriminative low-dimensional embedding by adding the local discriminant terms in the optimization objectives of manifold learning. Moreover, for efficiently handling out-of-sample extension and learning with the local consistency, we decompose the manifold learning as a two-step approach. We incorporate the manifold learning and the regression with a learned discriminant manifold-based projection function obtained by discriminatively Laplacian regularized least squares. The SLDML provides both the low-dimensional embedding and projection function with better intra-class compactness and inter-class separability, therefore preserves the local geometric structures more effectively. Meanwhile, the SLDML is supervised by both biased distance and continuous head pose angle information when constructing the graph, embedding the graph and learning the projection function. Our experiments demonstrate the superiority of the proposed SLDML over several current state-of-art approaches for head pose estimation on the publicly available FacePix dataset.

Hierarchical Manifold Learning for Regional Image Analysis

We present a novel method of hierarchical manifold learning which aims to automatically discover regional properties of image datasets. While traditional manifold learning methods have become widely used for dimensionality reduction in medical imaging, they suffer from only being able to consider whole images as single data points. We extend conventional techniques by additionally examining local variations, in order to produce spatially-varying manifold embeddings that characterize a given dataset. This involves constructing manifolds in a hierarchy of image patches of increasing granularity, while ensuring consistency between hierarchy levels. We demonstrate the utility of our method in two very different settings: 1) to learn the regional correlations in motion within a sequence of time-resolved MR images of the thoracic cavity; 2) to find discriminative regions of 3-D brain MR images associated with neurodegenerative disease.