Nonlinear Dimensionality Reduction Method Research Articles

Consensus embedding: theory, algorithms and application to segmentation and classification of biomedical data

BackgroundDimensionality reduction (DR) enables the construction of a lower dimensional space (embedding) from a higher dimensional feature space while preserving object-class discriminability. However several popular DR approaches suffer from sensitivity to choice of parameters and/or presence of noise in the data. In this paper, we present a novel DR technique known as consensus embedding that aims to overcome these problems by generating and combining multiple low-dimensional embeddings, hence exploiting the variance among them in a manner similar to ensemble classifier schemes such as Bagging. We demonstrate theoretical properties of consensus embedding which show that it will result in a single stable embedding solution that preserves information more accurately as compared to any individual embedding (generated via DR schemes such as Principal Component Analysis, Graph Embedding, or Locally Linear Embedding). Intelligent sub-sampling (via mean-shift) and code parallelization are utilized to provide for an efficient implementation of the scheme.ResultsApplications of consensus embedding are shown in the context of classification and clustering as applied to: (1) image partitioning of white matter and gray matter on 10 different synthetic brain MRI images corrupted with 18 different combinations of noise and bias field inhomogeneity, (2) classification of 4 high-dimensional gene-expression datasets, (3) cancer detection (at a pixel-level) on 16 image slices obtained from 2 different high-resolution prostate MRI datasets. In over 200 different experiments concerning classification and segmentation of biomedical data, consensus embedding was found to consistently outperform both linear and non-linear DR methods within all applications considered.ConclusionsWe have presented a novel framework termed consensus embedding which leverages ensemble classification theory within dimensionality reduction, allowing for application to a wide range of high-dimensional biomedical data classification and segmentation problems. Our generalizable framework allows for improved representation and classification in the context of both imaging and non-imaging data. The algorithm offers a promising solution to problems that currently plague DR methods, and may allow for extension to other areas of biomedical data analysis.

Nonlinear manifold representations for functional data

For functional data lying on an unknown nonlinear low-dimensional space, we study manifold learning and introduce the notions of manifold mean, manifold modes of functional variation and of functional manifold components. These constitute nonlinear representations of functional data that complement classical linear representations such as eigenfunctions and functional principal components. Our manifold learning procedures borrow ideas from existing nonlinear dimension reduction methods, which we modify to address functional data settings. In simulations and applications, we study examples of functional data which lie on a manifold and validate the superior behavior of manifold mean and functional manifold components over traditional cross-sectional mean and functional principal components. We also include consistency proofs for our estimators under certain assumptions.

Kernel ridge regression for out-of-sample mapping in supervised manifold learning

Manifold learning methods for unsupervised nonlinear dimensionality reduction have proven effective in the visualization of high dimensional data sets. When dealing with classification tasks, supervised extensions of manifold learning techniques, in which class labels are used to improve the embedding of the training points, require an appropriate method for out-of-sample mapping.In this paper we propose multi-output kernel ridge regression (KRR) for out-of-sample mapping in supervised manifold learning, in place of general regression neural networks (GRNN) that have been adopted by previous studies on the subject. Specifically, we consider a supervised agglomerative variant of Isomap and compare the performance of classification methods when the out-of-sample embedding is based on KRR and GRNN, respectively. Extensive computational experiments, using support vector machines and k-nearest neighbors as base classifiers, provide statistical evidence that out-of-sample mapping based on KRR consistently dominates its GRNN counterpart, and that supervised agglomerative Isomap with KRR achieves a higher accuracy than direct classification methods on most data sets.

Comparison of graph-based methods for non-linear dimensionality reduction

In this paper, four broadly representative graph-based techniques for manifold learning namely Isomap, Maximum Variance Unfolding (MVU), locally linear embedding and Laplacian eigenmaps have been reviewed and compared for non-linear dimensionality reduction. These methods begin by constructing a sparse graph in which the nodes represent input patterns and the edges represent neighbourhood relations. From these graphs, matrices can be constructed whose spectral decompositions reveal the low dimensional structure of the submanifold. All the four techniques are implemented on Swiss roll, helix, twin peak and broken Swiss roll dataset.

Metadynamics in the conformational space nonlinearly dimensionally reduced by Isomap

Atomic motions in molecules are not linear. This infers that nonlinear dimensionality reduction methods can outperform linear ones in analysis of collective atomic motions. In addition, nonlinear collective motions can be used as potentially efficient guides for biased simulation techniques. Here we present a simulation with a bias potential acting in the directions of collective motions determined by a nonlinear dimensionality reduction method. Ad hoc generated conformations of trans,trans-1,2,4-trifluorocyclooctane were analyzed by Isomap method to map these 72-dimensional coordinates to three dimensions, as described by Brown and co-workers [J. Chem. Phys. 129, 064118 (2008)]. Metadynamics employing the three-dimensional embeddings as collective variables was applied to explore all relevant conformations of the studied system and to calculate its conformational free energy surface. The method sampled all relevant conformations (boat, boat-chair, and crown) and corresponding transition structures inaccessible by an unbiased simulation. This scheme allows to use essentially any parameter of the system as a collective variable in biased simulations. Moreover, the scheme we used for mapping out-of-sample conformations from the 72D to 3D space can be used as a general purpose mapping for dimensionality reduction, beyond the context of molecular modeling.

An intuitive graphical visualization technique for the interrogation of transcriptome data.

The complexity of gene expression data generated from microarrays and high-throughput sequencing make their analysis challenging. One goal of these analyses is to define sets of co-regulated genes and identify patterns of gene expression. To date, however, there is a lack of easily implemented methods that allow an investigator to visualize and interact with the data in an intuitive and flexible manner. Here, we show that combining a nonlinear dimensionality reduction method, t-statistic Stochastic Neighbor Embedding (t-SNE), with a novel visualization technique provides a graphical mapping that allows the intuitive investigation of transcriptome data. This approach performs better than commonly used methods, offering insight into underlying patterns of gene expression at both global and local scales and identifying clusters of similarly expressed genes. A freely available MATLAB-implemented graphical user interface to perform t-SNE and nearest neighbour plots on genomic data sets is available at www.nimr.mrc.ac.uk/research/james-briscoe/visgenex.

SU‐E‐I‐134: Integration of Multiparametric and Multimodality Whole Body Radiological Imaging (MRI/PET/CT)

Purpose: To investigate the use of non linear dimensionality reduction (NLDR) techniques for integration, segmentation and quantification of multiparametric whole body (WB) radiological imaging methods. Methods: Twenty subjects were prospectively scanned with WB‐MRI and PET/CT, ten patients(age:61±12) and ten normal subjects(age:48± 11). WB‐MRI of T2WI, T1‐STIR and DWI were acquired at 3T. PET/CT using 18FDG or other advanced radiotracers, 11C‐acetate, 11C‐choline and Na18F was obtained on patients. If present, metastatic disease was confirmed by clinical standards. Whole body trace ADC maps, T1 and T2 were constructed for quantitative analysis and ROIs were drawn in regions of normal and abnormal appearing signal intensity. WB‐MRI and PET/CT data were co‐registered before application of the NDLR methods using a non‐rigid model. NLDR methods used were Isomap, locally linear embedding(LLE) and Diffusion‐Maps. The embedded image was constructed by projecting the feature spaces (image intensities) associated with each of the radiological modalities into a one dimensional embedding space (manifold). Dice similarity(DS) indices were calculated and statistical analysis was performed on them. Results: Integration of multiparametric WB‐MRI and PET/CT is feasible. WB‐DWI was able to distinguish potential metastatic regions in 7 patients, none were found in the other three with colocalization on WB‐MRI and 11C‐choline PET, but not in the NAF. These regions had decreased ADC values compared to normal tissue. The ADC values were significantly different(p=0.01) between patients and normal subjects. The segmented regions were congruent with a mean DS of (90.4+/−2.6)%. The computational load varied with the LLE as the having lowest compared to Isomap and Diffusion‐Maps. Conclusions: The integration of different WB modalities will provide a method to identify “global” tumor burdens and monitor treatment. There were significant differences in the ADC values in areas of metastatic disease compared to normal subjects. Our data suggest the potential of multimodality WB methods for characterization of metastatic disease.

Nonlinear dimensionality reduction method with the use of reference nodes for analysis of digital image collections

This method is based on hierarchical clusterization of data in a multidimensional space and the Sammon mapping. The key feature of the dimensionality reduction stage is the use of reference node lists created by the results of the hierarchical clusterization stage. The experimental results given in this work include the method performance quality estimates for some systems of the features extracted from digital image collections of different sizes.

Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing and applying computerized image analysis and multi-modal data fusion algorithms to digitized patient data (e.g. imaging, tissue, genomic) for helping physicians predict disease outcome and patient survival. While a number of data channels, ranging from the macro (e.g. MRI) to the nano-scales (proteins, genes) are now being routinely acquired for disease characterization, one of the challenges in predicting patient outcome and treatment response has been in our inability to quantitatively fuse these disparate, heterogeneous data sources. At the Laboratory for Computational Imaging and Bioinformatics (LCIB) 1 1 http://lcib.rutgers.edu. at Rutgers University, our team has been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities including MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on non-linear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate information from multiple data sources and modalities with the overarching goal of optimizing meta-classifiers for making prognostic predictions. In this paper, we briefly describe 4 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of Estrogen receptor positive breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in human epidermal growth factor amplified breast cancers) from digitized histopathology, (3) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitized needle biopsy specimens, and (4) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence following radical prostatectomy.

Biological pathway selection through nonlinear dimension reduction

In the analysis of high-throughput biological data, it is often believed that the biological units such as genes behave interactively by groups, that is, pathways in our context. It is conceivable that utilization of priorly available pathway knowledge would greatly facilitate both interpretation and estimation in statistical analysis of such high-dimensional biological data. In this article, we propose a 2-step procedure for the purpose of identifying pathways that are related to and influence the clinical phenotype. In the first step, a nonlinear dimension reduction method is proposed, which permits flexible within-pathway gene interactions as well as nonlinear pathway effects on the response. In the second step, a regularized model-based pathway ranking and selection procedure is developed that is built upon the summary features extracted from the first step. Simulations suggest that the new method performs favorably compared to the existing solutions. An analysis of a glioblastoma microarray data finds 4 pathways that have evidence of support from the biological literature.

Constraints on the Neighborhood Size in LLE

Locally linear embedding (LLE) is a well-known method for nonlinear dimensionality reduction. The mathematical proof and experimental results presented in this paper show that the neighborhood sizes in LLE must be smaller than the dimensions of input data spaces, otherwise LLE would degenerate from a nonlinear method for dimensionality reduction into a linear method for dimensionality reduction. Furthermore, when the neighborhood sizes are larger than the dimensions of input data spaces, the solutions to LLE are not unique. In these cases, the addition of some regularization method is often proposed. The experimental results presented in this paper show that the regularization method is not robust. Too large or too small regularization parameters cannot unwrap S-curve. Although a moderate regularization parameters can unwrap S-curve, the relative distance in the input data will be distorted in unwrapping. Therefore, in order to make LLE play fully its advantage in nonlinear dimensionality reduction and avoid multiple solutions happening, the best way is to make sure that the neighborhood sizes are smaller than the dimensions of input data spaces.

Locally Linear Embedding by Linear Programming

Dimensionality reduction has always been one of the most challenging tasks in the field of data mining. As a nonlinear dimensionality reduction method, locally linear embedding (LLE) has drawn more and more attention and applied widely in face image processing and text data processing. But this method is usually sensitive to noise, which limits its application in many fields. In this paper, we propose a locally linear embedding algorithm by linear programming (LLE by LP), and the experiments demonstrate the effectiveness of the approach in reducing the sensitivity to noise.

A new quality assessment criterion for nonlinear dimensionality reduction

A new quality assessment criterion for evaluating the performance of the nonlinear dimensionality reduction (NLDR) methods is proposed in this paper. Differing from the current quality assessment criteria focusing on the local-neighborhood-preserving performance of the NLDR methods, the proposed criterion capitalizes on a new aspect, the global-structure-holding performance, of the NLDR methods. By taking both properties into consideration, the intrinsic capability of the NLDR methods can be more faithfully reflected, and hence more rational measurement for the proper selection of NLDR methods in real-life applications can be offered. The theoretical argument is supported by experiment results implemented on a series of benchmark data sets.

Comparación de métodos de reducción de dimensión basados en análisis por localidades

En este trabajo se realiza una comparación de las principales técnicas de reducción de dimensión no lineal basadas en análisis por localidades, tales como: Locally linear embedding, Isometric feature mapping y Maximum variance unfolding. El estudio pretende determinar, bajo criterios objetivos, cuál de las técnicas consideradas conserva de mejor manera las propiedades locales de la variedad, y la estructura global de los datos de entrada al realizar un mapeo a un espacio de menor dimensión. Los métodos son especialmente analizados en aplicaciones de visualización. Las inmersiones obtenidas son evaluadas por medio de dos criterios: Error de Conservación de Vecindarios y Promedio de Vecinos Conservados. Para la validación experimental se utilizan bases de datos artificiales y reales que permiten confirmar visualmente la calidad de las inmersiones obtenidas. Con base en los resultados se observa que la técnica Maximum variance unfolding presenta inmersiones de mejor calidad, debido a que la técnica de optimización de este algoritmo preserva exactamente las distancias entre puntos cercanos en el espacio de baja dimensión, conservando la estructura global de la variedad analizada.

An automatic patient-specific seizure onset detection method in intracranial EEG based on incremental nonlinear dimensionalityreduction

Epileptic seizure features always include the morphology and spatial distribution of nonlinear waveforms in the electroencephalographic (EEG) signals. In this study, we propose a novel incremental learning scheme based on nonlinear dimensionality reduction for automatic patient-specific seizure onset detection. The method allows for identification of seizure onset times in long-term EEG signals acquired from epileptic patients. Firstly, a nonlinear dimensionality reduction (NDR) method called local tangent space alignment (LTSA) is used to reduce the dimensionality of available initial feature sets extracted with continuous wavelet transform (CWT). One-dimensional manifold which reflects the intrinsic dynamics of seizure onset is obtained. For each patient, IEEG recordings containing one seizure onset is sufficient to train the initial one-dimensional manifold. Secondly, an unsupervised incremental learning scheme is proposed to update the initial manifold when the unlabelled EEG segments flow in sequentially. The incremental learning scheme can cluster the new coming samples into the trained patterns (containing or not containing seizure onsets). Intracranial EEG recordings from 21 patients with duration of 193.8 h and 82 seizures are used for the evaluation of the method. Average sensitivity of 98.8%, average uninteresting false positive rate of 0.24/h, average interesting false positives rate of 0.25/h, and average detection delay of 10.8 s are obtained. Our method offers simple, accurate training with less human intervening and can be well used in off-line seizure detection. The unsupervised incremental learning scheme has the potential in identifying novel IEEG classes (different onset patterns) within thedata.

Curvature analysis of frequency modulated manifolds in dimensionality reduction

Recent advances in the analysis of high-dimensional signal data have triggered an increasing interest in geometry-based methods for nonlinear dimensionality reduction (NDR). In many applications, high-dimensional datasets typically contain redundant information, and NDR methods are important for an efficient analysis of their properties. During the last few years, concepts from differential geometry were used to create a new range of NDR methods. In the construction of such geometry-based strategies, a natural question is to understand their interaction with classical and modern signal processing tools (convolution transforms, Fourier analysis, wavelet functions). In particular, an important task is the analysis of the incurred geometrical deformation when applying signal transforms to the elements of a dataset. In this paper, we propose the concepts of modulation maps and modulation manifolds for the construction of particular datasets relevant in signal processing and NDR. We consider numerical methods for analyzing geometrical properties of the modulation manifolds, with a particular focus on their scalar and mean curvature. In our numerical examples, we apply the resulting geometry-based analysis to simple test cases, where we present geometrical and topological effects of relevance in manifold learning.

The Shape and Dimensionality of Phylogenetic Tree-Space Based on Mitochondrial Genomes

AbstractPhylogenetic analyses of large and diverse data sets generally result in large sets of competing phylogenetic trees. Consensus tree methods used to summarize sets of competing trees discard important information regarding the similarity and distribution of competing trees. A more fine grain approach is to use a dimensionality reduction method to project tree-to-tree distances in 2D or 3D space. In this study, we systematically evaluate the performance of several nonlinear dimensionality reduction (NLDR) methods on tree-to-tree distances obtained from independent nonparametric bootstrap analyses of genes from three mid- to large-sized mitochondrial genome alignments.

SU-GG-I-107: Unsupervised Nonlinear Dimensionality Reduction for Multiparametric Oncological Image Segmentation

Purpose: To investigate the use of unsupervised non‐linear dimensionality reduction (NLDR) techniques for segmentation and quantification of multi‐parametric breast MRI.Methods and Materials: Five patients underwent breast MRI after suspicious findings on mammograms or clinical breast exams. All patients underwent either biopsy or mastectomy and histopathological analysis was performed. MR sequences were T1‐weighted (T1WI:TR/TE=13/4.6ms), T2‐weighted (T2WI:TR/TE=5700/102, FOV=18×18cm,256×192,ST=4mm), Dynamic Contrast‐Enhanced (DCE:TR/TE=20/40, FOV=18×18cm,) after injection of GdDTPA contrast agent and Diffusion Weighted Imaging (DWI:TR/TE=5000/90ms,128×128,b=0,500, 750,1000s/mm2). MR data was co‐registered before application of the NDLR methods using a non‐deformable model. NLDR methods used were Isomap, Fuzzy Isomap, Locally Linear Embedding (LLE) and Diffusion‐Maps. The embedded image was constructed by projecting the feature spaces (image intensities) associated with each of the MR sequences into a one dimensional embedding space (manifold). The segmented regions were defined and lesion contours (LCs) were then detected by both embedded and DCE‐MR. DCE‐MR was selected for the ground truth for malignant breast lesions. Dice similarity (DS) indices were calculated and statistical analysis was performed on them. Results: All patients had invasive ductal carcinoma. The segmented regions were congruent with a median DS of (Isomap:86.47%±17.6%), (Fuzzy Isomap:86.48%±15.3%), (Diffusion‐Maps: 86.44%±15.9%) and (LEE:91.51%±4.3%). The computational load varied with the LLE as the having lowest compared to Isomap and Diffusion‐Maps. Interesting, the LEE was able to efficiently segment breast normal and abnormal tissue using only T1/T2 and DCE. However by including DWI with multiple b‐values, isomap showed better performance but with increased computationally costs. Conclusion: There was high similarity between multi‐parametric embedded and DCE‐MR images using NLDR methods. Using unsupervised NDLR, we demonstrated an improvement in the accuracy of the lesion segmentation and performed better than a single parameter.

Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization

Nonlinear dimensionality reduction methods are often used to visualize high-dimensional data, although the existing methods have been designed for other related tasks such as manifold learning. It ...

Efficient spatiotemporal analysis of the flagellar waveform of Chlamydomonas reinhardtii.

The 9 + 2 axoneme is a microtubule-based machine that powers the oscillatory beating of cilia and flagella. Its highly regulated movement is essential for the normal function of many organs; ciliopathies cause congenital defects, chronic respiratory tract infections and infertility. We present an efficient method to obtain a quantitative description of flagellar motion, with high spatial and temporal resolution, from high speed video recording of bright field images. This highly automated technique provides the shape, shear angle, curvature, and bend propagation speeds along the length of the flagellum, with approximately 200 temporal samples per beat. We compared the waveforms of uniflagellated wild-type and ida3 mutant cells, which lack the I1 inner dynein complex. Video images were captured at 350 fps. Rigid-body motion was eliminated by fast Fourier transform (FFT)-based registration, and the Cartesian (x-y) coordinates of points on the flagellum were identified. These x-y "point clouds" were embedded in two data dimensions using Isomap, a nonlinear dimension reduction method, and sorted by phase in the flagellar cycle. A smooth surface was fitted to the sorted point clouds, which provides high-resolution estimates of shear angle and curvature. Wild-type and ida3 cells exhibit large differences in shear amplitude, but similar maximum and minimum curvature values. In ida3 cells, the reverse bend begins earlier and travels more slowly relative to the principal bend, than in wild-type cells. The regulation of flagellar movement must involve I1 dynein in a manner consistent with these results.