Nonlinear Manifold Learning Methods Research Articles

Linear versus nonlinear dimensionality reduction for banks’ credit rating prediction

Dimensionality reduction methods have shown their usefulness for both supervised and unsupervised tasks in a wide range of application domains. Several linear and nonlinear approaches have been proposed in order to derive meaningful low-dimensional representations of high-dimensional data. Among nonlinear algorithms manifold learning methods, such as isometric feature mapping (Isomap), have recently attracted great attention by providing noteworthy results on artificial and real world data sets.The paper presents an empirical evaluation of two linear and nonlinear techniques, namely principal component analysis (PCA) and double-bounded tree-connected Isomap (dbt-Isomap), in order to assess their effectiveness for dimensionality reduction in banks’ credit rating prediction, and to determine the key financial variables endowed with the most explanatory power. Extensive computational tests concerning the classification of six banks’ rating data sets showed that the use of dimensionality reduction accomplished by nonlinear projections often induced an improvement in the classification accuracy, and that dbt-Isomap outperformed PCA by consistently providing more accurate predictions.

Theoretical–experimental study on a rotor with a residual shaft bow

This paper presents the influence of the residual shaft bow on the rotor longitudinal responses through simulation and experiment. Dynamics model of a warped rotor was established, and the influences of residual shaft bow on rotor vibration in different cases were simulated. The simulation results show that the residual shaft bow can seriously affect the rotor vibration as the eccentricity and the damping ratio are relatively small. Conversely, the influence will be extremely small. Simultaneously, a three-span double-disk rotor test rig was developed, and the experimental studies were carried out. The experimental signals of the rotor with multi-fault were collected, and the wavelet and nonlinear manifold learning methods were employed to extract the fault features. Then, the faults were identified by utilizing the principal component analysis method, and the fault signal of the residual shaft bow was obtained. The test modified equation of the rotor with residual shaft bow was proposed. The experimental and simulation results are consistent, and also verify the validity of the equation of motion.

Study of Fault Diagnosis Based on Manifold Learning

Characteristic signals in rotating machinery fault diagnosis with the issues of complex and difficult to deal with, while the use of non-linear manifold learning method can effectively extract low-dimensional manifold characteristics embedded in the high-dimensional non-linear data. It greatly maintains the overall geometric structure of the signals and improves the efficiency and reliability of the rotating machinery fault diagnosis. According to the development prospects of manifold learning, this paper describes four classical manifold learning methods and each advantages and disadvantages. It reviews the research status and application of fault diagnosis based on manifold learning, as well as future direction of researches in the field of manifold learning fault diagnosis.

A comparative study of nonlinear manifold learning methods for cancer microarray data classification

The paper presents an empirical comparison of the most prominent nonlinear manifold learning techniques for dimensionality reduction in the context of high-dimensional microarray data classification. In particular, we assessed the performance of six methods: isometric feature mapping, locally linear embedding, Laplacian eigenmaps, Hessian eigenmaps, local tangent space alignment and maximum variance unfolding. Unlike previous studies on the subject, the experimental framework adopted in this work properly extends to dimensionality reduction the supervised learning paradigm, by regarding the test set as an out-of-sample set of new points which are excluded from the manifold learning process. This in order to avoid a possible overestimate of the classification accuracy which may yield misleading comparative results. The different empirical approach requires the use of a fast and effective out-of-sample embedding method for mapping new high-dimensional data points into an existing reduced space. To this aim we propose to apply multi-output kernel ridge regression, an extension of linear ridge regression based on kernel functions which has been recently presented as a powerful method for out-of-sample projection when combined with a variant of isometric feature mapping. Computational experiments on a wide collection of cancer microarray data sets show that classifiers based on Isomap, LLE and LE were consistently more accurate than those relying on HE, LTSA and MVU. In particular, under different experimental conditions LLE-based classifier emerged as the most effective method whereas Isomap algorithm turned out to be the second best alternative for dimensionality reduction.

Process monitoring based on generalized orthogonal neighborhood preserving embedding

A new orthogonal neighborhood preserving embedding (ONPE) and its kernel generalization based process monitoring approaches are presented in this paper. ONPE aims at preserving local neighborhood structure of process data while reducing data dimensionality. As an approximation of the nonlinear manifold learning method, ONPE is capable to handle process nonlinearity. Moreover, to enhance the nonlinear modeling performance, the nonlinear extension of ONPE is also developed, with the introduction of kernel-tricks. By constructing monitoring statistics, both ONPE and its generalization are applied for fault detection in nonlinear processes. Two case studies show the superiority of the proposed methods in process monitoring.

Dependence of locally linear embedding on the regularization parameter

This paper deals with a method, called locally linear embedding (LLE). It is a nonlinear dimensionality reduction technique that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional data and attempts to discover a nonlinear structure (including manifolds) in high-dimensional data. In practice, the nonlinear manifold learning methods are applied in image processing, text mining, etc. The implementation of the LLE algorithm is fairly straightforward, because the algorithm has only two control parameters: the number of neighbors of each data point and the regularization parameter. The mapping quality is quite sensitive to these parameters. In this paper, we propose a new way of selecting a regularization parameter of a local Gram matrix.

Incremental Laplacian eigenmaps by preserving adjacent information between data points

Traditional nonlinear manifold learning methods have achieved great success in dimensionality reduction and feature extraction, most of which are batch modes. However, if new samples are observed, the batch methods need to be calculated repeatedly, which is computationally intensive, especially when the number or dimension of the input samples are large. This paper presents incremental learning algorithms for Laplacian eigenmaps, which computes the low-dimensional representation of data set by optimally preserving local neighborhood information in a certain sense. Sub-manifold analysis algorithm together with an alternative formulation of linear incremental method is proposed to learn the new samples incrementally. The locally linear reconstruction mechanism is introduced to update the existing samples’ embedding results. The algorithms are easy to be implemented and the computation procedure is simple. Simulation results testify the efficiency and accuracy of the proposed algorithms.

Markerless gating for lung cancer radiotherapy based on machine learning techniques

In lung cancer radiotherapy, radiation to a mobile target can be delivered by respiratory gating, for which we need to know whether the target is inside or outside a predefined gating window at any time point during the treatment. This can be achieved by tracking one or more fiducial markers implanted inside or near the target, either fluoroscopically or electromagnetically. However, the clinical implementation of marker tracking is limited for lung cancer radiotherapy mainly due to the risk of pneumothorax. Therefore, gating without implanted fiducial markers is a promising clinical direction. We have developed several template-matching methods for fluoroscopic marker-less gating. Recently, we have modeled the gating problem as a binary pattern classification problem, in which principal component analysis (PCA) and support vector machine (SVM) are combined to perform the classification task. Following the same framework, we investigated different combinations of dimensionality reduction techniques (PCA and four nonlinear manifold learning methods) and two machine learning classification methods (artificial neural networks—ANN and SVM). Performance was evaluated on ten fluoroscopic image sequences of nine lung cancer patients. We found that among all combinations of dimensionality reduction techniques and classification methods, PCA combined with either ANN or SVM achieved a better performance than the other nonlinear manifold learning methods. ANN when combined with PCA achieves a better performance than SVM in terms of classification accuracy and recall rate, although the target coverage is similar for the two classification methods. Furthermore, the running time for both ANN and SVM with PCA is within tolerance for real-time applications. Overall, ANN combined with PCA is a better candidate than other combinations we investigated in this work for real-time gated radiotherapy.

Topology Preservation Measures in the Visualization of Manifold-Type Multidimensional Data

Most of real-life data are not often truly high-dimensional. The data points just lie on a low-dimensional manifold embedded in a high-dimensional space. Nonlinear manifold learning methods automatically discover the low-dimensional nonlinear manifold in a high-dimensional data space and then embed the data points into a low-dimensional embedding space, preserving the underlying structure in the data. In this paper, we have used the locally linear embedding method on purpose to unravel a manifold. In order to quantitatively estimate the topology preservation of a manifold after unfolding it in a low-dimensional space, some quantitative numerical measure must be used. There are lots of different measures of topology preservation. We have investigated three measures: Spearman's rho, Konig's measure (KM), and mean relative rank errors (MRRE). After investigating different manifolds, it turned out that only KM and MRRE gave proper results of manifold topology preservation in all the cases. The main reason is that Spearman's rho considers distances between all the pairs of points from the analysed data set, while KM and MRRE evaluate a limited number of neighbours of each point from the analysed data set.

Robust locally linear embedding

In the past few years, some nonlinear dimensionality reduction (NLDR) or nonlinear manifold learning methods have aroused a great deal of interest in the machine learning community. These methods are promising in that they can automatically discover the low-dimensional nonlinear manifold in a high-dimensional data space and then embed the data points into a low-dimensional embedding space, using tractable linear algebraic techniques that are easy to implement and are not prone to local minima. Despite their appealing properties, these NLDR methods are not robust against outliers in the data, yet so far very little has been done to address the robustness problem. In this paper, we address this problem in the context of an NLDR method called locally linear embedding (LLE). Based on robust estimation techniques, we propose an approach to make LLE more robust. We refer to this approach as robust locally linear embedding (RLLE). We also present several specific methods for realizing this general RLLE approach. Experimental results on both synthetic and real-world data show that RLLE is very robust against outliers.