Unsupervised Manifold Learning Algorithm Research Articles

Unsupervised manifold learning with polynomial mapping on symmetric positive definite matrices

In this paper, an unsupervised manifold learning algorithm with polynomial mapping on the symmetric positive-definite (SPD) matrix manifold is introduced by matrix information geometry method for data dimensional reduction. Firstly, the mathematical knowledge about the SPD matrix manifold is presented including the metric, geodesic and submanifold. And then, the high dimensional information coordinates are given by different SPD matrix data for constructing the polynomial kernel matrix, weight matrix and sparsity preserving matrix. Next, the manifold learning algorithm on the SPD matrix manifold is proposed by polynomial mapping with geodesic distance. Finally, comparing with some conventional methods in terms of accuracy rate and time cost, the preliminary analysis results indicate that the proposed approach is able to offer a consistent and comprehensive method to realize the SPD matrix data dimensional reduction.

Dissimilarity space reinforced with manifold learning and latent space modeling for improved pattern classification

Dissimilarity representation plays a very important role in pattern recognition due to its ability to capture structural and relational information between samples. Dissimilarity space embedding is an approach in which each sample is represented as a vector based on its dissimilarity to some other samples called prototypes. However, lack of neighborhood-preserving, fixed and usually considerable prototype set for all training samples cause low classification accuracy and high computational complexity. To address these challenges, our proposed method creates dissimilarity space considering the neighbors of each data point on the manifold. For this purpose, Locally Linear Embedding (LLE) is used as an unsupervised manifold learning algorithm. The only goal of this step is to learn the global structure and the neighborhood of data on the manifold and mapping or dimension reduction is not performed. In order to create the dissimilarity space, each sample is compared only with its prototype set including its k-nearest neighbors on the manifold using the geodesic distance metric. Geodesic distance metric is used for the structure preserving and is computed using the weighted LLE neighborhood graph. Finally, Latent Space Model (LSM), is applied to reduce the dimensions of the Euclidean latent space so that the second challenge is resolved. To evaluate the resulted representation ad so called dissimilarity space, two common classifiers namely K Nearest Neighbor (KNN) and Support Vector Machine (SVM) are applied. Experiments on different datasets which included both Euclidean and non-Euclidean spaces, demonstrate that using the proposed approach, classifiers outperform the other basic dissimilarity spaces in both accuracy and runtime.

An Out-of-Sample Extension to Manifold Learning via Meta-Modelling.

Unsupervised manifold learning has become accepted as an important tool for reducing dimensionality of a dataset by finding its meaningful low-dimensional representation lying on an unknown nonlinear subspace. Most manifold learning methods only embed an existing dataset, but do not provide an explicit mapping function for novel out-of-sample data, thereby potentially resulting in an ineffective tool for classification purposes, particularly for iterative methods such as active learning. To address this issue, out-of-sample extension methods have been introduced to generalize an existing embedding of new samples. In this work, a novel out-of-sample method is introduced by utilizing High Dimensional Model Representation (HDMR) as a nonlinear multivariate regression with the Tikhonov regularizer for unsupervised manifold learning algorithms. The proposed method was extensively analyzed using illustrative datasets sampled from known manifolds. Several experiments with 3D synthetic datasets and face recognition datasets were also conducted, and the performance of the proposed method was compared to several well-known out-of-sample methods. The results obtained with Locally Linear Embedding (LLE), Laplacian Eigenmaps (LE), and t-Distributed Stochastic Neighbor Embedding (t-SNE) showed that the proposed method achieves competitive even better performance than the other out-of-sample methods.

An optimized unsupervised manifold learning algorithm for manycore architectures

Multimedia data, such as images and videos, has become very popular in people’s daily life as a result of the widespread use of mobile devices. The ever-increasing amount of such data, along with the necessity for real-time retrieval, has lead to the development of new methods that can process them in a timely fashion with acceptable accuracy. In this paper, we study the performance of ReckNN, an unsupervised manifold learning algorithm based on the reciprocal neighbourhood and the authority of ranked lists. Most of the related work in this field do not fully investigate optimization strategies, an aspect that is becoming more important with the high availability of manycore machines. In order to address that issue, we fully investigate optimization opportunities in this article and make the following three main contributions. Firstly, we develop an efficient and scalable method for storing and accessing the distances between objects (e.g., video or image) based on dictionaries. Secondly, we employ memoization to speed up the computation of authority scores, leading to a significant performance gain even on single-core architectures. Lastly, we devise and implement several parallelization strategies and show that they are scalable on a 72-core Intel machine. The experimental results with MPEG-7, Corel5k and MediaEval benchmarks show that the optimized ReckNN delivers both efficiency and scalability, highlighting the importance of the proposed optimizations for manycore machines.

Remote Sensing Data Classification Using Combined Spectral and Spatial Local Linear Embedding (CSSLE)

Feature learning and extraction have been used to reduce the complexity of the representation of remote sensing data. In this paper, a novel remote sensing data feature analysis method is proposed based on the locally linear embedding (LLE) techniques, which is an unsupervised manifold learning algorithm. If the remote sensing data variability is described by a small number of spatial features, we can treat the data as lying on a low dimensional manifold in the high dimensional space of remote sensing data. The proposed method, combined spectral and spatial local linear embedding (CSSLE) makes use the neighbors in both domains. Compared with LLE, CSSLE makes up the shortage that LLE ignores the relationship among the spatial neighboring pixels which is extremely important for remote sensing images. In this paper we provide experiment results from the analysis of remote sensing data using PCA, LLE and CSSLE. Classification results show that proposed method can give higher accuracies than the linear method of PCA and the nonlinear method of LLE.

A correlation graph approach for unsupervised manifold learning in image retrieval tasks

Effectively measuring the similarity among images is a challenging problem in image retrieval tasks due to the difficulty of considering the dataset manifold. This paper presents an unsupervised manifold learning algorithm that takes into account the intrinsic dataset geometry for defining a more effective distance among images. The dataset structure is modeled in terms of a Correlation Graph (CG) and analyzed using Strongly Connected Components (SCCs). While the Correlation Graph adjacency provides a precise but strict similarity relationship, the Strongly Connected Components analysis expands these relationships considering the dataset geometry. A large and rigorous experimental evaluation protocol was conducted for different image retrieval tasks. The experiments were conducted in different datasets involving various image descriptors. Results demonstrate that the manifold learning algorithm can significantly improve the effectiveness of image retrieval systems. The presented approach yields better results in terms of effectiveness than various methods recently proposed in the literature.

Neighbourhood sensitive preserving embedding for pattern classification

Recently, a large family of supervised or unsupervised manifold learning algorithms that stem from statistical or geometrical theory has been designed to solve the problem of pattern classification. In this study, consider the fact that the data are usually sampled from a low-dimensional manifold space which resides in a high-dimensional Euclidean space, the authors propose a novel two-graph-based supervised linear classification algorithm called neighbourhood sensitive preserving embedding (NSPE). Different from local linear embedding (LLE) (or neighbourhood preserving embedding (NPE)) which preserves the local neighbourhood structure with one graph, NSPE can discover both the intrinsic and discriminant structure of the data manifold by constructing two graphs, that is, the within-class graph and the between-class graph. Thus, the data are mapped into a subspace where the nearby points with the same label are close to each other, whereas the nearby points with different labels are far apart. As a classification method, besides being defined on training samples, NSPE is also defined on testing samples. Experiments carried on the real-world face databases demonstrate that the results of all two-graph-based spectral methods are comparable and better than that of one-graph-based methods.

Supervised locally tangent space alignment for machine fault diagnosis

How to deal with the high-dimensional and nonlinear data is a challenging problem for fault diagnosis. An unsupervised locally tangent space alignment (LTSA) has recently proven to be an effective unsupervised manifold learning algorithm for high-dimensional data analysis. In this paper, a supervised expansion of LTSA (named S-LTSA) is proposed, which takes full advantage of class label information to improve classification performance. Based on S-LTSA, a novel machine fault diagnosis approach is proposed to deal with the high-dimensional fault data that contain multiple manifolds corresponding to fault classes. The experiment results with bearing fault data show that the proposed approach outperforms the other fault pattern recognition approaches such PCA, ICA, LDA and LTSA.

Facial Expression Recognition Based on Incremental Isomap with Expression Weighted Distance

The Isometric mapping algorithm is an unsupervised manifold learning algorithm, with no consideration of the class of training samples, while supervised isometric mapping treats the difference among classes equally. Considering the inner relationship between different expressions, we have proposed isometric mapping algorithm based on expression weighted distance, which assigns weighted values according to different sample distance in order to make full use of knowledge of expression classes when calculating the geodesic distance between training samples. We use incremental isometric mapping algorithm on new samples so as to simplify computation significantly when dealing with new samples. Then k-NN classifier is applied to classify different expression features. The facial expression recognition experiments are performed on the JAFFE database and the results show that this proposed algorithm performs better than ISOMAP algorithm and supervised ISOMAP algorithm, and it is more feasible and effective. /mtp_HPc (Envelope Mean Variance method), M2M4 (Second order and Fourth order Moment method)). Furthermore, according to the characteristics of FM modulation system, we put forward the improvement of M2M4 and also discuss the optimal algorithm for underwater acoustic Rayleigh channel, Rician channel and Bellhop channel by comparing these methods.

Synchronized Submanifold Embedding for Person-Independent Pose Estimation and Beyond

Precise 3-D head pose estimation plays a significant role in developing human-computer interfaces and practical face recognition systems. This task is challenging due to the particular appearance variations caused by pose changes for a certain subject. In this paper, the pose data space is considered as a union of submanifolds which characterize different subjects, instead of a single continuous manifold as conventionally regarded. A novel manifold embedding algorithm dually supervised by both identity and pose information, called synchronized submanifold embedding (SSE), is proposed for person-independent precise 3-D pose estimation, which means that the testing subject may not appear in the model training stage. First, the submanifold of a certain subject is approximated as a set of simplexes constructed using neighboring samples. Then, these simplexized submanifolds from different subjects are embedded by synchronizing the locally propagated poses within the simplexes and at the same time maximizing the intrasubmanifold variances. Finally, the pose of a new datum is estimated as the propagated pose of the nearest point within the simplex constructed by its nearest neighbors in the dimensionality reduced feature space. The experiments on the 3-D pose estimation database, CHIL data for CLEAR07 evaluation, and the extended application for age estimation on FG-NET aging database, demonstrate the superiority of SSE over conventional regression algorithms as well as unsupervised manifold learning algorithms.