Nonlinear Dimensionality Reduction Algorithm Research Articles

LEICA: Laplacian eigenmaps for group ICA decomposition of fMRI data

Independent component analysis (ICA) is a data-driven method that has been increasingly used for analyzing functional Magnetic Resonance Imaging (fMRI) data. However, generalizing ICA to multi-subject studies is non-trivial due to the high-dimensionality of the data, the complexity of the underlying neuronal processes, the presence of various noise sources, and inter-subject variability. Current group ICA based approaches typically use several forms of the Principal Component Analysis (PCA) method to extend ICA for generating group inferences. However, linear dimensionality reduction techniques have serious limitations including the fact that the underlying BOLD signal is a complex function of several nonlinear processes. In this paper, we propose an effective non-linear ICA-based model for extracting group-level spatial maps from multi-subject fMRI datasets. We use a non-linear dimensionality reduction algorithm based on Laplacian eigenmaps to identify a manifold subspace common to the group, such that this mapping preserves the correlation among voxels’ time series as much as possible. These eigenmaps are modeled as linear mixtures of a set of group-level spatial features, which are then extracted using ICA. The resulting algorithm is called LEICA (Laplacian Eigenmaps for group ICA decomposition). We introduce a number of methods to evaluate LEICA using 100-subject resting state and 100-subject working memory task fMRI datasets from the Human Connectome Project (HCP). The test results show that the extracted spatial maps from LEICA are meaningful functional networks similar to those produced by some of the best known methods. Importantly, relative to state-of-the-art methods, our algorithm compares favorably in terms of the functional cohesiveness of the spatial maps generated, as well as in terms of the reproducibility of the results.

Structure-Aware Data Consolidation.

We present a structure-aware technique to consolidate noisy data, which we use as a pre-process for standard clustering and dimensionality reduction. Our technique is related to mean shift, but instead of seeking density modes, it reveals and consolidates continuous high density structures such as curves and surface sheets in the underlying data while ignoring noise and outliers. We provide a theoretical analysis under a Gaussian noise model, and show that our approach significantly improves the performance of many non-linear dimensionality reduction and clustering algorithms in challenging scenarios.

Quasi-curvature Local Linear Projection and Extreme Learning Machine for nonlinear dimensionality reduction

Abstract As one of the classical nonlinear dimensionality reduction algorithms, Locally Linear Embedding (LLE) has shown powerful performance in many research fields. However, there are still two limitations in LLE: (1) traditional LLE is sensitive to high-curvature noise; (2) the computation is too expensive. To solve these problems, we present Quasi-curvature LLE (QLLE) through taking the curvature of local neighborhoods into consideration when mapping local configuration into low-dimensional coordinates. And then a novel learning framework called Quasi-curvature Local Linear Projection (QLLP) is proposed for efficient dimensionality reduction. This framework first selects small landmarks from original data to obtain the low-dimensional coordinates in QLLE, and then adopts Extreme Learning Machine (ELM) to learn the explicit mapping function from original data to low-dimensional coordinates for nonlinear dimensionality reduction. The extensive experiments in synthetic and Frey facial expression datasets demonstrate that this framework can greatly improve the efficiency in nonlinear dimensionality reduction.

Robust locally linear embedding algorithm for machinery fault diagnosis

Locally linear embedding (LLE) is a classical nonlinear dimensionality reduction algorithm, and it has been widely used in machinery fault diagnosis. LLE reduces the dimensions of a data set only by exploring the geometry structure, that is, the geometry structure is one of the key factors for the embedding result. In conventional LLE algorithm, the geometry structure is calculated by ordinary least square (OLS) algorithm, which makes the embedding result be sensitive to noise. In order to resolve the problem, a robust LLE (RLLE) is investigated. In RLLE algorithm, the Least Angle Regression and the Elastic Net (LARS-EN) technologies are employed to compute the local structure. Besides, a novel fault diagnosis method based on RLLE and support vector machine (SVM) are proposed for machinery fault diagnosis. Experiments performed on both synthetic and real data sets demonstrate the advantages of the proposed method in the term of fault diagnosis.

Nonlinear Dimensionality Reduction for Data with Disconnected Neighborhood Graph

Neighborhood graph based nonlinear dimensionality reduction algorithms, such as Isomap and LLE, perform well under an assumption that the neighborhood graph is connected. However, for datasets consisting of multiple clusters or lying on multiple manifolds, the neighborhood graphs are often disconnected, or in other words, have multiple connected components. Neighborhood graph based dimensionality reduction techniques cannot recognize both the local and global properties of such datasets. In this paper, a new method, called enhanced neighborhood graph, is proposed to solve the problem. The concept is to add edges to the neighborhood graph adaptively and iteratively until it becomes connected. Nonlinear dimensionality reduction can then be performed based on the enhanced neighborhood graph. As a result, both local and global properties of the data can be exactly recognized. In this study, thorough simulations on synthetic datasets and natural datasets are conducted. The experimental results corroborate that the proposed method provides significant improvements on dimensionality reduction for data with disconnected neighborhood graph.

Mapping the conformational free energy of aspartic acid in the gas phase and in aqueous solution

The conformational free energy landscape of aspartic acid, a proteogenic amino acid involved in a wide variety of biological functions, was investigated as an example of the complexity that multiple rotatable bonds produce even in relatively simple molecules. To efficiently explore such a landscape, this molecule was studied in the neutral and zwitterionic forms, in the gas phase and in water solution, by means of molecular dynamics and the enhanced sampling method metadynamics with classical force-fields. Multi-dimensional free energy landscapes were reduced to bi-dimensional maps through the non-linear dimensionality reduction algorithm sketch-map to identify the energetically stable conformers and their interconnection paths. Quantum chemical calculations were then performed on the minimum free energy structures. Our procedure returned the low energy conformations observed experimentally in the gas phase with rotational spectroscopy [M. E. Sanz et al., Phys. Chem. Chem. Phys. 12, 3573 (2010)]. Moreover, it provided information on higher energy conformers not accessible to experiments and on the conformers in water. The comparison between different force-fields and quantum chemical data highlighted the importance of the underlying potential energy surface to accurately capture energy rankings. The combination of force-field based metadynamics, sketch-map analysis, and quantum chemical calculations was able to produce an exhaustive conformational exploration in a range of significant free energies that complements the experimental data. Similar protocols can be applied to larger peptides with complex conformational landscapes and would greatly benefit from the next generation of accurate force-fields.

Dimensionality reduction based on distance preservation to local mean for symmetric positive definite matrices and its application in brain–computer interfaces

Objective. In this paper, we propose a nonlinear dimensionality reduction algorithm for the manifold of symmetric positive definite (SPD) matrices that considers the geometry of SPD matrices and provides a low-dimensional representation of the manifold with high class discrimination in a supervised or unsupervised manner. Approach. The proposed algorithm tries to preserve the local structure of the data by preserving distances to local means (DPLM) and also provides an implicit projection matrix. DPLM is linear in terms of the number of training samples. Main results. We performed several experiments on the multi-class dataset IIa from BCI competition IV and two other datasets from BCI competition III including datasets IIIa and IVa. The results show that our approach as dimensionality reduction technique—leads to superior results in comparison with other competitors in the related literature because of its robustness against outliers and the way it preserves the local geometry of the data. Significance. The experiments confirm that the combination of DPLM with filter geodesic minimum distance to mean as the classifier leads to superior performance compared with the state of the art on brain–computer interface competition IV dataset IIa. Also the statistical analysis shows that our dimensionality reduction method performs significantly better than its competitors.

Manifold learning: Dimensionality reduction and high dimensional data reconstruction via dictionary learning

Nonlinear dimensionality reduction (DR) algorithms can reveal the intrinsic characteristic of the high dimensional data in a succinct way. However, most of these methods suffer from two problems. First, the incremental dimensionality reduction problem, which means the algorithms cannot compute the embedding of new added data incrementally. Second, the high dimensional data reconstruction problem, which means the algorithms cannot recover the original high dimensional data from the embeddings. Both problems limit the application of the existing DR algorithms. In this paper, a dictionary-based algorithm for manifold learning is proposed to address the problems of incremental dimensionality reduction and high dimensional data reconstruction. In this algorithm, two dictionaries are trained. One is for the manifold in the high dimensional space and the other one is for the embeddings which can be computed by any existing DR method in the low dimensional space. When new data is added, dimensionality reduction and data reconstruction can just be conducted by coding this input data over one dictionary, and then use this code to recover the output data via the other dictionary. The proposed algorithm provides a general framework for manifold learning. It can be integrated into many existing DR algorithms to make them feasible to both incremental dimensionality reduction and high dimensional data reconstruction. The algorithm is efficient due to the closed-form solution for sparse coding and dictionary updating. Furthermore, the proposed algorithm is space-saving because it only needs to store two dictionaries instead of the whole training samples. Experiments conducted on synthetic datasets and real world datasets show that, no matter for incremental dimensionality reduction or high dimensional data reconstruction, the proposed algorithm is accurate and efficient.

Water resources climate change projections using supervised nonlinear and multivariate soft computing techniques

Summary Accurate projection of global warming on the probabilistic behavior of hydro-climate variables is one of the main challenges in climate change impact assessment studies. Due to the complexity of climate-associated processes, different sources of uncertainty influence the projected behavior of hydro-climate variables in regression-based statistical downscaling procedures. The current study presents a comprehensive methodology to improve the predictive power of the procedure to provide improved projections. It does this by minimizing the uncertainty sources arising from the high-dimensionality of atmospheric predictors, the complex and nonlinear relationships between hydro-climate predictands and atmospheric predictors, as well as the biases that exist in climate model simulations. To address the impact of the high dimensional feature spaces, a supervised nonlinear dimensionality reduction algorithm is presented that is able to capture the nonlinear variability among projectors through extracting a sequence of principal components that have maximal dependency with the target hydro-climate variables. Two soft-computing nonlinear machine-learning methods, Support Vector Regression (SVR) and Relevance Vector Machine (RVM), are engaged to capture the nonlinear relationships between predictand and atmospheric predictors. To correct the spatial and temporal biases over multiple time scales in the GCM predictands, the Multivariate Recursive Nesting Bias Correction (MRNBC) approach is used. The results demonstrate that this combined approach significantly improves the downscaling procedure in terms of precipitation projection.

LLE for submersible plunger pump fault diagnosis via joint wavelet and SVD approach

Locally linear embedding (LLE) is a classical non-linear dimension reduction algorithm. LLE can deal with a nonlinear dataset and extract the significant feature from the dataset. However, LLE may fail, when signal is non-stationary or contains strong noise. In this paper, wavelet transformation, singular value decomposition (SVD), and LLE are assembled to extract the feature of a dataset based on which a new fault diagnosis method is developed for submersible plunger pump fault recognition. First, each sample is preprocessed by wavelet transformation and SVD, upon which a new feature space is constructed; then, LLE is utilized to reduce the dimensions of the feature space; finally, support vector machines (SVM) is invoked to recognize the device status. The experiments performed on submersible plunger pump dataset and bearing dataset demonstrate that the proposed method is effective and can achieve a high diagnosis accuracy.

Multifeatures Fusion and Nonlinear Dimension Reduction for Intelligent Bearing Condition Monitoring

Condition-based maintenance is critical to reduce the costs of maintenance and improve the production efficiency. Data-driven method based on neural network (NN) is one of the most used models for mechanical components condition recognition. In this paper, we introduce a new bearing condition recognition method based on multifeatures extraction and deep neural network (DNN). First, the method calculates time domain, frequency domain, and time-frequency domain features to represent characteristic of vibration signals. Then the nonlinear dimension reduction algorithm based on deep learning is proposed to reduce the redundancy information. Finally, the top-layer classifier of deep neural network outputs the bearing condition. The proposed method is validated using experiment test-bed bearing vibration data. Meanwhile some comparative studies are performed; the results show the advantage of the proposed method in adaptive features selection and superior accuracy in bearing condition recognition.

Maturation and Phenotypic Diversity of Human CD4+ Regulatory T Cells in Umbilical Cord Blood and Peripheral Blood from Healthy Donors

Introduction: CD4+ FoxP3+ CD25+ regulatory T cells (Treg) are required for maintenance of immune tolerance and immune homeostasis. Quantitative or functional Treg deficiency has been correlated with autoimmune disease, allergy, allograft rejection and graft versus host disease. Conversely, increased Treg can suppress tumor immunity resulting in tumor progression. Treg express a large number of cellular markers that reflect their level of maturation, functionality, activation and migratory capacity. Nevertheless, it has not previously been possible to integrate the expression of these various markers and correlate their expression with human Treg differentiation in vivo.Methods: We used single cell mass cytometry (CyTOF) to simultaneously study 36 phenotypic and functional markers of human Treg in samples obtained from umbilical cord blood (CB) (n=4) and healthy adult donors (n=10). Wanderlust trajectory detection algorithm was used to analyze temporal positioning of Treg across development. To quantify Treg heterogeneity we used ACCENSE; an analysis tool that combines a nonlinear dimensionality reduction algorithm (t-SNE) with spectral clustering algorithm and automated cell classification into subpopulations.Results: Using Wanderlust to characterize Treg maturation, the majority of CB Treg were naive (CD45RA+) and CB memory Treg (CD45RA-) were poorly differentiated with minimal expression of activation (HLA-DR) and pro-apoptotic (CD95) markers (Figure 1A). Adult Treg contained few naive cells and mature Treg effector cells expressed high levels of activation and pro-apoptotic markers. (Figure 1B). Using Wanderlust together with spearman correlation, 5 discrete stages of Treg maturation were identified; 1) recent thymic emigrants (RTE), 2) naive, 3) effector, 4) activated and 5) terminal effector. RTE Treg defined by expression of CD45RA, CD31 and CCR7, also expressed markers of proliferation (KI67) and functionality (Tbet, PDL-1, Helios) at low levels but lacked functional CTLA4 and TIM-3. Naive Treg lacked expression of CD31 but expressed other markers characteristic of RTE. Effector Treg expressed increased levels of CD95, CTLA-4, CCR7, GITR, Helios and FoxP3 but lacked HLA-DR. Activated effector Treg expressed the highest levels of FoxP3, Helios and Ki67, along with functional markers (CD28, CXCR3, ICOS, GITR, CD39, CTLA-4, TIM-3) and homing molecules (vascular endothelial CCR5, gut addressin ACT-1, skin addressins CCR4, CLA). Activated Treg express the highest levels and diversity of functional markers along with the ability to migrate to different tissues. Lastly, terminal effector Treg express a more restricted set of functional and homing markers (CD28, CTLA-4, ICOS and CCR5) with less diversity. Markers of exhaustion (PD-1 and TIM-3) are also expressed by effector, activated and terminal effector Treg. Pro-apoptotic markers (CD95high BCL2low) are primarily expressed by activated and terminal effector Treg.Using ACCENSE to evaluate Treg diversity allowed further identification of discrete Treg sub-populations within each maturation state. RTE and naive Treg appear very homogeneous and appear as a single cluster in both CB and adults. In contrast, effector, activated and terminal effector Treg are more heterogeneous and are visualized as 9 distinct clusters (Figure 1C, D). This clustering reflects distinct subsets of memory Treg that co-express various combinations of functional markers in our panel. All 9 Treg effector populations are present in CB, but at much lower levels. Treg effector cell diversity is maintained but does not increase as Treg mature and expand in adults and RTE/naive Treg become less prevalent.Conclusion: Our study is the first to quantify human Treg heterogeneity based on expression of a large set of activation, proliferation, tissue homing and functional markers in conjunction with stages of Treg maturation and differentiation. These studies define 5 stages of Treg maturation and 10 clusters of Treg diversity based on differential expression of phenotypic and functional markers. Similar approaches can be applied to describe maturation and diversity of other cell populations. Further application of this CyTOF panel can be used to study Treg maturation and diversity after hematopoietic stem cell transplantation and in immune and inflammatory diseases, to identify specific defects that may contribute to immune dysfunction. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

Longjing tea quality classification by fusion of features collected from E-nose

Electronic nose (E-nose) was employed for quality classification of Xihu-Longjing tea in this paper. Four grades of Xihu-Longjing tea, which were classified by tea experts, were attempted in the experiment. The aim of this study is to conduct feature fusion method with two kinds of features — time domain features and frequency domain features. Accounting for the redundant information provided by some sensors, Fisher criterion was conducted for sensor selection, and dimensionality reduction algorithms were applied for further feature selection. Experimental results showed that the fused features could better represent signal characteristics compared with the single features. Based on the fused features, the performances of linear and nonlinear dimensionality reduction algorithms were compared. Experimental results indicated that nonlinear algorithms were more effective in feature selection than linear algorithms, and the highest recognition rate could reach to 100% by KLDA. The results achieved in this paper showed the superiority of the fused features in representing signal characteristics, and indicated that E-nose can be successfully used in quality classification of Xihu-Longjing tea.

Orthogonal Margin Maximization Projection for Gait Recognition

An efficient supervised orthogonal nonlinear dimensionality reduction algorithm, namely orthogonal margin maximization projection (OMMP), is presented for gait recognition in this paper. Taking the local neighborhood geometry structure and class information into account, the proposed algorithm aims to find a projecting matrix by maximizing the local neighborhood margin between the different classes and preserving the local geometry structure of the data. After projecting, the data points in the same class are pulled as close as possible, while the data points in different classes are pushed as far as possible. The highlights of OMMP include (1) takes both of the local information and class information of the data into account; (2) considers the effect of the noisy points and outliers; (3) it is supervised and orthogonal; and (4) its physical meaning is very clear. The experimental results on a public gait database show the effectiveness of the proposed method.

Improved time series land cover classification by missing-observation-adaptive nonlinear dimensionality reduction

Dimensionality reduction (DR) is a widely used technique to address the curse of dimensionality when high-dimensional remotely sensed data, such as multi-temporal or hyperspectral imagery, are analyzed. Nonlinear DR algorithms, also referred to as manifold learning algorithms, have been successfully applied to hyperspectral data and provide improved performance compared with linear DR algorithms. However, DR algorithms cannot handle missing data that are common in multi-temporal imagery. In this paper, the Laplacian Eigenmaps (LE) nonlinear DR algorithm was refined for application to multi-temporal satellite data with large proportions of missing data. Refined LE algorithms were applied to 52-week Landsat time series for three study areas in Texas, Kansas and South Dakota that have different amounts of missing data and land cover complexity. A series of random forest classifications were conducted on the refined LE DR bands using varying proportions of training data provided by the United States Department of Agriculture (USDA) National Agricultural Statistics Service (NASS) Cropland Data Layer (CDL); these classification results were compared with conventional metrics-based random forest classifications. Experimental results show that compared with the metrics approach, higher per-class and overall classification accuracies were obtained using the refined LE DR bands of multispectral reflectance time series, and the number of training samples required to achieve a given degree of classification accuracy was also reduced. The approach of applying the refined LE to multispectral reflectance time series is promising in that it is automated and provides dimensionality-reduced data with desirable classification properties. The implications of this research and possibilities for future algorithm development and application are discussed.

Linear and nonlinear subspace analysis of hand movements during grasping.

This study investigated nonlinear patterns of coordination, or synergies, underlying whole-hand grasping kinematics. Prior research has shed considerable light on roles played by such coordinated degrees-of-freedom (DOF), illuminating how motor control is facilitated by structural and functional specializations in the brain, peripheral nervous system, and musculoskeletal system. However, existing analyses suppose that the patterns of coordination can be captured by means of linear analyses, as linear combinations of nominally independent DOF. In contrast, hand kinematics is itself highly nonlinear in nature. To address this discrepancy, we sought to to determine whether nonlinear synergies might serve to more accurately and efficiently explain human grasping kinematics than is possible with linear analyses. We analyzed motion capture data acquired from the hands of individuals as they grasped an array of common objects, using four of the most widely used linear and nonlinear dimensionality reduction algorithms. We compared the results using a recently developed algorithm-agnostic quality measure, which enabled us to assess the quality of the dimensional reductions that resulted by assessing the extent to which local neighborhood information in the data was preserved. Although qualitative inspection of this data suggested that nonlinear correlations between kinematic variables were present, we found that linear modeling, in the form of Principle Components Analysis, could perform better than any of the nonlinear techniques we applied.

Automatic computation of left ventricular volume changes over a cardiac cycle from echocardiography images by nonlinear dimensionality reduction.

Curve of left ventricular (LV) volume changes throughout the cardiac cycle is a fundamental parameter for clinical evaluation of various cardiovascular diseases. Currently, this evaluation is often performed manually which is tedious and time consuming and suffers from significant interobserver and intraobserver variability. This paper introduces a new automatic method, based on nonlinear dimensionality reduction (NLDR) for extracting the curve of the LV volume changes over a cardiac cycle from two-dimensional (2-D) echocardiography images. Isometric feature mapping (Isomap) is one of the most popular NLDR algorithms. In this study, a modified version of Isomap algorithm, where image to image distance metric is computed using nonrigid registration, is applied on 2-D echocardiography images of one cycle of heart. Using this approach, the nonlinear information of these images is embedded in a 2-D manifold and each image is characterized by a symbol on the constructed manifold. This new representation visualizes the relationship between these images based on LV volume changes and allows extracting the curve of the LV volume changes automatically. Our method in comparison to the traditional segmentation algorithms does not need any LV myocardial segmentation and tracking, particularly difficult in the echocardiography images. Moreover, a large data set under various diseases for training is not required. The results obtained by our method are quantitatively evaluated to those obtained manually by the highly experienced echocardiographer on ten healthy volunteers and six patients which depict the usefulness of the presented method.

The Globally Linear Embedding Algorithm

LLE is a very effective non-linear dimension reduction algorithm and widely explored in machine learning, pattern recognition, data mining and etc. Locally linear, Globally non-linear has always been regarded as the features and advantages of LLE. However, the theoretical derivation presented in this paper shows that when the size of neighborhood is larger than the dimension of the space in which the data are presented, LLE is no longer global nonlinear and almost has the same effect as PCA in dimensionality reduction. At present, a lot of literatures on LLE verify their results on Swiss Roll, Punctured Sphere, Twin Peaks, etc. These manifolds are presented in the three-dimensional Euclidean space and the size of neighborhood is always larger than three to prevent too small to be effective. But in these cases, LLE cannot play its advantage of nonlinearity.

Global Non-linear Dimensionality Reduction Algorithm in Image Sets

Most of the real world image is in the area of all possible pictures level instance of low dimensional characteristics. Know this low dimension to understand many image set is a key first step, has been applied to the reduced-order measures in different jurisdictions, including facial recognition, machine learning, image retrieval and handwritten character recognition. Traditional evaluation of statistical relationship distance global dimension reduction arithmetic. This assignment optimization assessment measures the relationship between the proposed a global data--GODR. The relationship between the measurement in the implementation of the data set and they are the intersection point. As the point of view, global dimension reduces the built-in implementations. Compared to worldwide arithmetic, should the divisor, and have more useful result. Facial expression implemented the change the efficiency of the arithmetic is verified by the experiments.

An Improved Nonlinear Dimensionality Reduction Algorithm and Its Application

An Improved Nonlinear Dimensionality Reduction Algorithm and Its Application