Traditional Manifold Learning Algorithms Research Articles

Supervised local spline embedding for medical diagnosis

A common difficulty of intelligent medical diagnosis is the high dimensionality of medical data. Manifold learning provides an elegant way to solve this problem by mapping the high-dimensional data into the low-dimensional embedding. However, traditional manifold learning algorithms fail to fully utilize the supervised information in medical diagnosis. To overcome this problem, in this paper we propose a novel Supervised Local Spline Embedding (SLSE) algorithm, which incorporates the supervised information into the local spline manifold embedding. SLSE not only preserves the local neighborhood structure, but also utilizes the global manifold shape through spline interpolation. Moreover, SLSE leverages the supervised information by maximizing the inter-class scatterness and minimizing the intra-class scatterness in the low-dimensional embedding. The promising experimental results on real-world medical datasets illustrate the superiority of our proposed approach in comparison with the existing popular manifold learning algorithms.

Multi-manifold locality graph preserving analysis for hyperspectral image classification

Manifold learning has been successfully applied to hyperspectral image (HSI) classification by modeling different land covers as a smooth manifold embedded in a high-dimensional space. However, traditional manifold learning algorithms were proposed with the assumption of single manifold structure in HSI, while the samples in different subsets may belong to different sub-manifolds. In this paper, a novel dimensionality reduction (DR) method called multi-manifold locality graph preserving analysis (MLGPA) was proposed for feature learning of HSI data. According to the label information of HSI, MLGPA divides the samples data into different subsets, and each subset is treated as a sub-manifold. Then, it constructs a within-manifold graph and a between-manifold graph for each sub-manifold to characterize within-manifold compactness and between-manifold separability, and a discriminant projection matrix can be obtained by maximizing the between-manifold scatter and minimizing the within-manifold scatter simultaneously. Finally, low-dimensional embedding features of different sub-manifolds are fused to improve the classification performance. MLGPA can effectively reveal the multi-manifold structure and improve the classification performance of HSI. Experimental results on three real-world HSI data sets demonstrate that MLGPA is superior to some state-of-the-art methods in terms of classification accuracy.

Applying Ricci flow to high dimensional manifold learning

Traditional manifold learning algorithms often bear an assumption that the local neighborhood of any point on embedded manifold is roughly equal to the tangent space at that point without considering the curvature. The curvature indifferent way of manifold processing often makes traditional dimension reduction poorly neighborhood preserving. To overcome this drawback we propose a new algorithm called RF-ML to perform an operation on the manifold with help of Ricci flow before reducing the dimension of manifold.

Curvature-aware manifold learning

One of the fundamental assumptions of traditional manifold learning algorithms is that the embedded manifold is globally or locally isometric to Euclidean space. Under this assumption, these algorithms divided manifold into a set of overlapping local patches which are locally isometric to linear subsets of Euclidean space. Then the learnt manifold would be a flat manifold with zero Riemannian curvature. But in the general cases, manifolds may not have this property. To be more specific, the traditional manifold learning does not consider the curvature information of the embedded manifold. In order to improve the existing algorithms, we propose a curvature-aware manifold learning algorithm called CAML. Without considering the local and global assumptions, we will add the curvature information to the process of manifold learning, and try to find a way to reduce the redundant dimensions of the general manifolds which are not isometric to Euclidean space. The experiments have shown that CAML has its own advantage comparing to other traditional manifold learning algorithms in the sense of the neighborhood preserving ratios (NPR) on synthetic databases and classification accuracies on image set classification.

Rolling Element Bearing Fault Diagnosis Using Improved Manifold Learning

Fault feature can be extracted by traditional manifold learning algorithms, which construct neighborhood graphs by Euclidean distance (ED). It is difficult to get an excellent dimensionality reduction result when processed data has strong correlations. In order to improve the effect of dimensionality reduction and increase accuracy of bearing fault diagnosis in mechanical systems, an improved manifold learning method based on Mahalanobis distance (MD) is proposed. In this paper, we use time-domain analysis and frequency-domain analysis to construct high-dimensional feature vectors in the first step. Then, MD is used to replace ED in neighborhood construction of manifold learning. After using the improved manifold learning method, low-dimensional feature vectors can be extracted. Finally, fault diagnosis of rolling element bearing can be made by applying the K-nearest neighbor classifier. In part of experiment, to verify the efficiency of the improved manifold learning methods, artificial data sets and rolling element bearing fault data are adopted. The experimental comparison results of the improved manifold learning algorithm and the traditional algorithm prove that the proposed method is more effective in rolling element bearing fault diagnosis.

Extreme spectral regression for efficient regularized subspace learning

Traditional manifold learning algorithms, such as Locally Linear Embedding, Isomap and Laplacian Eigenmap, only provide the embedding results of training samples. Although many extensions of these approaches try to solve the out-of-sample extension problem, their computations cannot avoid eigen-decomposition of dense matrices which is expensive in both time and memory. To solve this problem, spectral regression (SR) casts the problem of learning an embedding function into a regression framework. Motivated by the effectiveness of extreme learning machine (ELM), in this paper, we solve the out-of-sample extension problem by seeking an embedding function in ELM feature space. An extreme spectral regression (ESR) algorithm is proposed to speed up kernel-based SR (KSR) further. In addition, it is proved that ESR is an approximation of KSR. Similar to SR, the proposed ESR algorithm can be performed in supervised, unsupervised and semi-supervised situation. Experimental results on classification and semi-supervised classification demonstrate the effectiveness and efficiency of our algorithm.

Multiple Kernel Spectral Regression for Dimensionality Reduction

Traditional manifold learning algorithms, such as locally linear embedding, Isomap, and Laplacian eigenmap, only provide the embedding results of the training samples. To solve the out-of-sample extension problem, spectral regression (SR) solves the problem of learning an embedding function by establishing a regression framework, which can avoid eigen-decomposition of dense matrices. Motivated by the effectiveness of SR, we incorporate multiple kernel learning (MKL) into SR for dimensionality reduction. The proposed approach (termed MKL-SR) seeks an embedding function in the Reproducing Kernel Hilbert Space (RKHS) induced by the multiple base kernels. An MKL-SR algorithm is proposed to improve the performance of kernel-based SR (KSR) further. Furthermore, the proposed MKL-SR algorithm can be performed in the supervised, unsupervised, and semi-supervised situation. Experimental results on supervised classification and semi-supervised classification demonstrate the effectiveness and efficiency of our algorithm.

Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Derived from the traditional manifold learning algorithms, local discriminant analysis methods identify the underlying submanifold structures while employing discriminative information for dimensionality reduction. Mathematically, they can all be unified into a graph embedding framework with different construction criteria. However, such learning algorithms are limited by the curse-of-dimensionality if the original data lie on the high-dimensional manifold. Different from the existing algorithms, we consider the discriminant embedding as a kernel analysis approach in the sample space, and a kernel-view based discriminant method is proposed for the embedded feature extraction, where both PCA pre-processing and the pruning of data can be avoided. Extensive experiments on the high-dimensional data sets show the robustness and outstanding performance of our proposed method.

A COMPUTATIONAL AND THEORETICAL ANALYSIS OF LOCAL NULL SPACE DISCRIMINANT METHOD FOR PATTERN CLASSIFICATION

Many problems in pattern classification and feature extraction involve dimensionality reduction as a necessary processing. Traditional manifold learning algorithms, such as ISOMAP, LLE, and Laplacian Eigenmap, seek the low-dimensional manifold in an unsupervised way, while the local discriminant analysis methods identify the underlying supervised submanifold structures. In addition, it has been well-known that the intraclass null subspace contains the most discriminative information if the original data exist in a high-dimensional space. In this paper, we seek for the local null space in accordance with the null space LDA (NLDA) approach and reveal that its computational expense mainly depends on the quantity of connected edges in graphs, which may be still unacceptable if a great deal of samples are involved. To address this limitation, an improved local null space algorithm is proposed to employ the penalty subspace to approximate the local discriminant subspace. Compared with the traditional approach, the proposed method can achieve more efficiency so that the overload problem is avoided, while slight discriminant power is lost theoretically. A comparative study on classification shows that the performance of the approximative algorithm is quite close to the genuine one.

LLE-based classification algorithm for MMW radar target recognition

In this paper, a new multiclass classification algorithm is proposed based on the idea of Locally Linear Embedding (LLE), to avoid the defect of traditional manifold learning algorithms, which can not deal with new sample points. The algorithm defines an error as a criterion by computing a sample’s reconstruction weight using LLE. Furthermore, the existence and characteristics of low dimensional manifold in range-profile time-frequency information are explored using manifold learning algorithm, aiming at the problem of target recognition about high range resolution MilliMeter-Wave (MMW) radar. The new algorithm is applied to radar target recognition. The experiment results show the algorithm is efficient. Compared with other classification algorithms, our method improves the recognition precision and the result is not sensitive to input parameters.

Nonlinear Discriminant Analysis on Embedded Manifold

Traditional manifold learning algorithms, such as ISOMAP, LLE, and Laplacian Eigenmap, mainly focus on uncovering the latent low-dimensional geometry structure of the training samples in an unsupervised manner where useful class information is ignored. Therefore, the derived low-dimensional representations are not necessarily optimal in discriminative capability. In this paper, we study the discriminant analysis problem by considering the nonlinear manifold structure of data space. To this end, firstly, a new clustering algorithm, called Intra-Cluster Balanced K-Means (ICBKM), is proposed to partition the samples into multiple clusters while ensure that there are balanced samples for the classes within each cluster; approximately, each cluster can be considered as a local patch on the embedded manifold. Then, the local discriminative projections for different clusters are simultaneously calculated by optimizing the global Fisher Criterion based on the cluster weighted data representation. Compared with traditional linear/kernel discriminant analysis (KDA) algorithms, our proposed algorithm has the following characteristics: 1) it essentially is a KDA algorithm with specific geometry-adaptive-kernel tailored to the specific data structure, in contrast to traditional KDA in which the kernel is fixed and independent to the data set; 2) it is approximately a locally linear while globally nonlinear discriminant analyzer; 3) it does not need to store the original samples for computing the low-dimensional representation of a new data; and 4) it is computationally efficient compared with traditional KDA when the sample number is large. The toy problem on artificial data demonstrates the effectiveness of our proposed algorithm in deriving discriminative representations for problems with nonlinear classification hyperplane. The face recognition experiments on YALE and CMU PIE databases show that our proposed algorithm significantly outperforms linear discriminant analysis (LDA) as well as Mixture LDA, and has higher accuracy than KDA with traditional kernels