Nonlinear Dimensionality Reduction Approach Research Articles

Exploring local interpretability in dimensionality reduction: Analysis and use cases

Dimensionality reduction is a crucial area in artificial intelligence that enables the visualization and analysis of high-dimensional data. The main use of dimensionality reduction is to lower the dimensional complexity of data, improving the performance of machine learning models. Non-linear dimensionality reduction approaches, which provide higher quality representations than linear ones, lack interpretability, prohibiting their application in tasks requiring interpretability. This paper presents LXDR (Local eXplanation of Dimensionality Reduction), a local, model-agnostic technique that can be applied to any DR technique. LXDR trains linear models around a neighborhood of a specific instance and provides local interpretations using a variety of neighborhood generation techniques. Variations of the proposed technique are also introduced. The effectiveness of LXDR’s interpretations is evaluated by quantitative and qualitative experiments, as well as demonstrations of its practical implementation in diverse use cases. The experiments emphasize the importance of interpretability in dimensionality reduction and how LXDR reinforces it.

MANAGEMENT OF FLOOD AND SEPARATING DAMAGE FROM ENVIRONMENTAL EFFECTS USING BAYESIAN CONVOLUTIONAL NEURAL NETWORK

<p>Flood management is the act of determining the frequency, magnitude, and duration of flood episodes based on the elements of a river basin that may be monitored. Floods endanger human lives and inflict significant property damage. It is essential for creating suitable flood risk management plans, lowering flood dangers, and evacuating people from flood-prone locations. Hydrodynamic methods for managing floods may be replaced with deep learning. Existing methods, however, concentrate on employing CNNs or RNNs to capture either the spatial or temporal flood patterns. Despite several advancements in flood control technology, less focus has been placed on minimising the damage that these systems do to the environment in order to boost their dependability and effectiveness. When the data is skewed, CNNs might overfit. We demonstrate how automated regularisation and uncertainty quantification allows Bayesian-CNN to get beyond these drawbacks. We have created a unique method to make use of the uncertainties that the Bayesian-CNN provides, which greatly improves performance on a big portion of the test data (around a 6% increase in accuracy on 77% of test data). By projecting the data into a low-dimensional space using a nonlinear dimensionality reduction approach, we also provide an entirely novel rationale for the uncertainty. This dimensionality reduction makes it possible to visualize the test data for interpretation and displays the data's structure in a low-dimensional feature space. This paper discusses and makes use of uncertainties for flood control while demonstrating the benefits of Bayesian-CNN over state-of-the-art technology. As a consequence, the Bayesian-CNN obtains 95.7% F1-score, 99.3% F1-score, 98.5% precision, and 98.3% recall.</p>

Evaluating Linear and Non-linear Dimensionality Reduction Approaches for Deep Learning-based Network Intrusion Detection Systems

Evaluating Linear and Non-linear Dimensionality Reduction Approaches for Deep Learning-based Network Intrusion Detection Systems

English

Breast cancer is among the top causes of fatalities related to cancer in females. Radiologists commonly use mammogram images to detect breast tumors in their early stages. However, mammography can produce low-contrast images, making it difficult and time-consuming to segment abnormal regions. Deep convolutional neural networks (CNNs) are commonly used for image evaluations. This study used deep CNN models to develop a computer-aided diagnostic (CAD) system for feature extraction and classification. The proposed approach consists of three phases. In the first phase, a shallow, deep CNN model comprising five convolutional layers, five max-pooling layers, one batch normalization layer, and one dropout layer was developed and used to extract recombined images and novel features. In the second phase, the Inception-v3 model was used for label smoothing and classification due to its multiple filters with different sizes. In the third phase, features were extracted using shallow, deep CNN and Inception-v3 models. The Infallible Euclidean distance-based nonlinear dimensionality reduction approach was used to minimize dimensionality. Finally, the Gini-index-based C4.5 decision tree was used for the binary classification of mammogram images from the Digital Database for Screening Mammography (DDSM) + Curated Breast Imaging Subset of DDSM (CBIS-DDSM) and Mammographic Image Analysis Society (MIAS) datasets. The proposed hybrid shallow, deep CNN and Inception-v3 model achieved 99.52% accuracy, a 96% AUC on the DDSM + CBISDDSM dataset, and an accuracy of 97.53% and an AUC value of 97% on the MIAS dataset. Compared with other cutting-edge CAD systems, the proposed hybrid approach achieved higher accuracy by combining in-depth features across both datasets.

Surface Approximation by Means of Gaussian Process Latent Variable Models and Line Element Geometry

The close relation between spatial kinematics and line geometry has been proven to be fruitful in surface detection and reconstruction. However, methods based on this approach are limited to simple geometric shapes that can be formulated as a linear subspace of line or line element space. The core of this approach is a principal component formulation to find a best-fit approximant to a possibly noisy or impartial surface given as an unordered set of points or point cloud. We expand on this by introducing the Gaussian process latent variable model, a probabilistic non-linear non-parametric dimensionality reduction approach following the Bayesian paradigm. This allows us to find structure in a lower dimensional latent space for the surfaces of interest. We show how this can be applied in surface approximation and unsupervised segmentation to the surfaces mentioned above and demonstrate its benefits on surfaces that deviate from these. Experiments are conducted on synthetic and real-world objects.

Verbal and Non-Verbal Color-Emotional Subjective Spaces Are Four-Dimensional, Spherical and Isomorphic

The brain at wakefulness is active even in the absence of goal-directed behavior or salient stimuli. However, patterns of this resting-state (RS) activity can undergo long-term alterations following exposure to preceding meaningful stimuli. This study was aimed to develop an unbiased method to detect such changes in the RS activity after exposure to emotionally meaningful stimuli. For this purpose, we used functional magnetic resonance imaging (fMRI) of RS brain activity before and after acquisition and extinction of experimental conditioned fear. A group of healthy volunteers participated in three fMRI sessions: a RS before fear conditioning, a fear extinction session, and a RS immediately after fear extinction. The fear-conditioning paradigm consisted of three neutral visual stimuli paired with a partial reinforcement by a mild electric current. We used both linear and non-linear dimensionality reduction approaches to distinguish between the initial RS and the RS after stimuli exposure. The principal component analysis (PCA) as a linear dimensionality reduction method showed significantly worse results than non-linear methods (Isomap, LLE, Laplacian eigenmaps). Using the Laplacian eigenmaps manifold learning method, we were able to show significant differences between the two RSs at the level of individual participants. This detection was further improved by smoothing the BOLD signal with the wavelet multiresolution analysis. The developed method can improve the discrimination of functional states collected in longitudinal fMRI studies.

The Laplacian eigenmaps dimensionality reduction of fMRI data for discovering stimulus-induced changes in the resting-state brain activity

The brain at wakefulness is active even in the absence of goal-directed behavior or salient stimuli. However, patterns of this resting-state (RS) activity can undergo long-term alterations following exposure to preceding meaningful stimuli. This study was aimed to develop an unbiased method to detect such changes in the RS activity after exposure to emotionally meaningful stimuli. For this purpose, we used functional magnetic resonance imaging (fMRI) of RS brain activity before and after acquisition and extinction of experimental conditioned fear. A group of healthy volunteers participated in three fMRI sessions: a RS before fear conditioning, a fear extinction session, and a RS immediately after fear extinction. The fear-conditioning paradigm consisted of three neutral visual stimuli paired with a partial reinforcement by a mild electric current. We used both linear and non-linear dimensionality reduction approaches to distinguish between the initial RS and the RS after stimuli exposure. The principal component analysis (PCA) as a linear dimensionality reduction method showed significantly worse results than non-linear methods (Isomap, LLE, Laplacian eigenmaps). Using the Laplacian eigenmaps manifold learning method, we were able to show significant differences between the two RSs at the level of individual participants. This detection was further improved by smoothing the BOLD signal with the wavelet multiresolution analysis. The developed method can improve the discrimination of functional states collected in longitudinal fMRI studies.

Prediction model based on the Laplacian eigenmap method combined with a random forest algorithm for rainstorm satellite images during the first annual rainy season in South China

The recent emergence of satellite detection and imaging technologies has increased the demand for the application of satellite cloud images to current weather forecasting. However, approaches based on nonlinear prediction technology to forecast satellite images are lacking, and forecasting timelines are relatively short, e.g., 1–3 h in advance. In the present study, a nonlinear dimensionality reduction approach based on Laplacian eigenmaps (LEs) was combined with a random forest (RF) algorithm to construct an intelligent computing prediction model for rainstorm satellite images obtained from the first annual rainy season (April–June) in South China from 2010 to 2018. Results showed that the proposed forecasting model based on nonlinear intelligent calculation can accurately predict the key features and trends of the development and movement of strong precipitation clouds. The predicted satellite images described by the model were also consistent with the major features of the observed satellite images. This study then used a multiple linear regression (MLR) method based on the same prediction factors to establish a model for predicting satellite images for the same modeling and forecasting samples. Comparative results of the two prediction schemes showed that the LE + RF algorithm satellite image prediction scheme yields more samples exhibiting a high correlation with observed satellite images than the MLR method. Compared with that of the proposed scheme, the amount of samples of the MLR scheme in the low-correlation area was significantly larger. In general, the nonlinear intelligent computing scheme developed in this study is superior to the MLR method for predicting satellite cloud images. Thus, the LE + RF algorithm satellite image prediction scheme provides an objective and practical method for observed satellite cloud image predictions.

The Low Dimensionality of Development

The World Bank routinely publishes over 1500 “World Development Indicators” to track the socioeconomic development at the country level. A range of indices has been proposed to interpret this information. For instance, the “Human Development Index” was designed to specifically capture development in terms of life expectancy, education, and standard of living. However, the general question which independent dimensions are essential to capture all aspects of development still remains open. Using a nonlinear dimensionality reduction approach we aim to extract the core dimensions of development in a highly efficient way. We find that more than 90% of variance in the WDIs can be represented by solely five uncorrelated dimensions. The first dimension, explaining 74% of variance, represents the state of education, health, income, infrastructure, trade, population, and pollution. Although this dimension resembles the HDI, it explains much more variance. The second dimension (explaining 10% of variance) differentiates countries by gender ratios, labor market, and energy production patterns. Here, we differentiate societal structures when comparing e.g. countries from the Middle-East to the Post-Soviet area. Our analysis confirms that most countries show rather consistent temporal trends towards wealthier and aging societies. We can also find deviations from the long-term trajectories during warfare, environmental disasters, or fundamental political changes. The data-driven nature of the extracted dimensions complements classical indicator approaches, allowing a broader exploration of global development space. The extracted independent dimensions represent different aspects of development that need to be considered when proposing new metric indices.

Principal Tensor Embedding for Unsupervised Tensor Learning

Tensors and multiway analysis aim to explore the relationships between the variables used to represent the data and find a summarization of the data with models of reduced dimensionality. However, although in this context a great attention was devoted to this problem, dimension reduction of high-order tensors remains a challenge. The aim of this article is to provide a nonlinear dimensionality reduction approach, named principal tensor embedding (PTE), for unsupervised tensor learning, that is able to derive an explicit nonlinear model of data. As in the standard manifold learning (ML) technique, it assumes multidimensional data lie close to a low-dimensional manifold embedded in a high-dimensional space. On the basis of this assumption a local parametrization of data that accurately captures its local geometry is derived. From this mathematical framework a nonlinear stochastic model of data that depends on a reduced set of latent variables is obtained. In this way the initial problem of unsupervised learning is reduced to the regression of a nonlinear input-output function, i.e. a supervised learning problem. Extensive experiments on several tensor datasets demonstrate that the proposed ML approach gives competitive performance when compared with other techniques used for data reconstruction and classification.

Artifacts in Simultaneous hdEEG/fMRI Imaging: A Nonlinear Dimensionality Reduction Approach.

Simultaneous recordings of electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI) are at the forefront of technologies of interest to physicians and scientists because they combine the benefits of both modalities—better time resolution (hdEEG) and space resolution (fMRI). However, EEG measurements in the scanner contain an electromagnetic field that is induced in leads as a result of gradient switching slight head movements and vibrations, and it is corrupted by changes in the measured potential because of the Hall phenomenon. The aim of this study is to design and test a methodology for inspecting hidden EEG structures with respect to artifacts. We propose a top-down strategy to obtain additional information that is not visible in a single recording. The time-domain independent component analysis algorithm was employed to obtain independent components and spatial weights. A nonlinear dimension reduction technique t-distributed stochastic neighbor embedding was used to create low-dimensional space, which was then partitioned using the density-based spatial clustering of applications with noise (DBSCAN). The relationships between the found data structure and the used criteria were investigated. As a result, we were able to extract information from the data structure regarding electrooculographic, electrocardiographic, electromyographic and gradient artifacts. This new methodology could facilitate the identification of artifacts and their residues from simultaneous EEG in fMRI.

Metabolic regulation of pluripotency and germ cell fate through α-ketoglutarate.

An intricate link is becoming apparent between metabolism and cellular identities. Here, we explore the basis for such a link in an in vitro model for early mouse embryonic development: from naïve pluripotency to the specification of primordial germ cells (PGCs). Using single‐cell RNA‐seq with statistical modelling and modulation of energy metabolism, we demonstrate a functional role for oxidative mitochondrial metabolism in naïve pluripotency. We link mitochondrial tricarboxylic acid cycle activity to IDH2‐mediated production of alpha‐ketoglutarate and through it, the activity of key epigenetic regulators. Accordingly, this metabolite has a role in the maintenance of naïve pluripotency as well as in PGC differentiation, likely through preserving a particular histone methylation status underlying the transient state of developmental competence for the PGC fate. We reveal a link between energy metabolism and epigenetic control of cell state transitions during a developmental trajectory towards germ cell specification, and establish a paradigm for stabilizing fleeting cellular states through metabolic modulation.

Linear dynamical modes as new variables for data-driven ENSO forecast

A new data-driven model for analysis and prediction of spatially distributed time series is proposed. The model is based on a linear dynamical mode (LDM) decomposition of the observed data which is derived from a recently developed nonlinear dimensionality reduction approach. The key point of this approach is its ability to take into account simple dynamical properties of the observed system by means of revealing the system’s dominant time scales. The LDMs are used as new variables for empirical construction of a nonlinear stochastic evolution operator. The method is applied to the sea surface temperature anomaly field in the tropical belt where the El Nino Southern Oscillation (ENSO) is the main mode of variability. The advantage of LDMs versus traditionally used empirical orthogonal function decomposition is demonstrated for this data. Specifically, it is shown that the new model has a competitive ENSO forecast skill in comparison with the other existing ENSO models.

Abstract A78: Serum metabolomics, cytokine measurements, and tumor RNA-seq identified phospholipids correlated with a molecular subclass as strong predictor for outcome in high-grade serous ovarian cancer.

Abstract Background: Currently, the field of ‘omics’ is still growing, giving step by step deeper insight into the biological system of cancer. Increasing evidence suggests that alterations in cancer metabolism might have a tremendous impact on survival and could guide new treatment strategies. Therefore, including metabolic phenotyping in a study design would allow better understanding of the biological function and status of tumor cells and their hosts. Epithelial ovarian cancer (EOC) is very heterogeneous in both, local tumor spread and survival. Molecular subtypes presenting with distinct whole-genome expression profiles and significant survival differences have been described and awareness of the molecular alterations is required to identify new targets for therapy. In this study, serum metabolite changes were identified and evaluated as potential prognostic markers in a test set of 65 high grade serous EOC (HGSOC) patients. The prognostic impact was validated in an independent cohort of 165 EOC patients. Methods: Targeted metabolomics was performed on serum using the AbsoluteIDQ p180 kit (Biocrates Life Sciences), measuring 188 metabolites. In addition to a test and validation set of 65 HGSOC and 165 serous advanced stage EOC patients, the metabolomics profile of 62 healthy women was determined. In a subset of patients 16 cancer biomarkers and 40 chemokines were measured using Luminex bead-based assays and whole genome transcriptomes were assessed from different tumor tissues (primary, metastatic, and ascitic) by RNA sequencing. Results: 53 metabolites were significantly negatively correlated to overall survival (OS). Thereof, the majority (43) constituted glycerophospholipids (GPhL), followed by five acylcarnitines, four sphingolipids and one biogenic amine. A non-linear dimensionality reduction approach was used to reduce the information of the 43 GPhLs to three dimensions. The first showed the strongest impact on OS and revealed, as GPhL predictor, a significant impact on OS in uni- and multivariate (HR 0.38; p<0.001) Cox regression analyses, i.e. low GPhL concentrations in blood lead to worse OS. To determine the biologic background of these changes in the serum lipidome, correlations to cytokines and tumor markers were evaluated. IL6, CXCL2, osteopontin, and CCL23 were significantly negatively and leptin and sEGFR positively correlated to the GPhL-predictor. In a next step, RNA sequencing of different tumor tissues identified 393 genes negatively correlated (i.e. the carbohydrate-responsive element-binding protein, ChREPB) and 429 genes positively correlated (i.e. complete respiratory chain, many histones) with the GPhL-predictor. To provide more profound survival analysis, a validation was performed using expression data initially used for validation of a molecular subclassification with a tremendous impact on OS in 165 serous advanced stage EOC patients (Pils et al. Cancer Sci. 103:1334). Using the 50 most significantly up- and the 50 down-regulated genes a robust predictor was built. This gene predictor represents the GPhL-predictor and had a significant independent impact on OS (HR 0.66; p=0.004). In addition to this validation of the survival impact, a very strong correlation between the observed GPhL changes (via the gene predictor) and the molecular subclass was found, indicating a functional connection between these molecular subclasses and the serum lipidome. Conclusion: Targetd metabolomics revealed a strong independent impact of the serum concentrations of glycerophospholipids on HGSOC outcome in the test and a validation cohort. Transcriptomics data indicate changes in the energy metabolism of the tumor. Interestingly, the lipidome-changes were strongly correlated to a previously described molecular subclass in serous EOC. Citation Format: Stefanie Aust, Anna Bachmayr-Heyda, Katharina Auer, Nyamdelger Sukhbaatar, Samuel M. Meier, Christopher Gerner, Dietmar Pils. Serum metabolomics, cytokine measurements, and tumor RNA-seq identified phospholipids correlated with a molecular subclass as strong predictor for outcome in high-grade serous ovarian cancer. [abstract]. In: Proceedings of the AACR Special Conference on Advances in Ovarian Cancer Research: Exploiting Vulnerabilities; Oct 17-20, 2015; Orlando, FL. Philadelphia (PA): AACR; Clin Cancer Res 2016;22(2 Suppl):Abstract nr A78.

Learning Flexible Graph-Based Semi-Supervised Embedding.

This paper introduces a graph-based semi-supervised embedding method as well as its kernelized version for generic classification and recognition tasks. The aim is to combine the merits of flexible manifold embedding and nonlinear graph-based embedding for semi-supervised learning. The proposed linear method will be flexible since it estimates a nonlinear manifold that is the closest one to a linear embedding. The proposed kernelized method will also be flexible since it estimates a kernel-based embedding that is the closest to a nonlinear manifold. In both proposed methods, the nonlinear manifold and the mapping (linear transform for the linear method and the kernel multipliers for the kernelized method) are simultaneously estimated, which overcomes the shortcomings of a cascaded estimation. The dimension of the final embedding obtained by the two proposed methods is not limited to the number of classes. They can be used by any kind of classifiers once the data are embedded into the new subspaces. Unlike nonlinear dimensionality reduction approaches, which suffer from out-of-sample problem, our proposed methods have an obvious advantage that the learnt subspace has a direct out-of-sample extension to novel samples, and are thus easily generalized to the entire high-dimensional input space. We provide extensive experiments on seven public databases in order to study the performance of the proposed methods. These experiments demonstrate much improvement over the state-of-the-art algorithms that are based on label propagation or graph-based semi-supervised embedding.

A supervised non-linear dimensionality reduction approach for manifold learning

In this paper we introduce a novel supervised manifold learning technique called Supervised Laplacian Eigenmaps (S-LE), which makes use of class label information to guide the procedure of non-linear dimensionality reduction by adopting the large margin concept. The graph Laplacian is split into two components: within-class graph and between-class graph to better characterize the discriminant property of the data. Our approach has two important characteristics: (i) it adaptively estimates the local neighborhood surrounding each sample based on data density and similarity and (ii) the objective function simultaneously maximizes the local margin between heterogeneous samples and pushes the homogeneous samples closer to each other. Our approach has been tested on several challenging face databases and it has been conveniently compared with other linear and non-linear techniques, demonstrating its superiority. Although we have concentrated in this paper on the face recognition problem, the proposed approach could also be applied to other category of objects characterized by large variations in their appearance (such as hand or body pose, for instance).

Non-linear dimensionality reduction of signaling networks

BackgroundSystems wide modeling and analysis of signaling networks is essential for understanding complex cellular behaviors, such as the biphasic responses to different combinations of cytokines and growth factors. For example, tumor necrosis factor (TNF) can act as a proapoptotic or prosurvival factor depending on its concentration, the current state of signaling network and the presence of other cytokines. To understand combinatorial regulation in such systems, new computational approaches are required that can take into account non-linear interactions in signaling networks and provide tools for clustering, visualization and predictive modeling.ResultsHere we extended and applied an unsupervised non-linear dimensionality reduction approach, Isomap, to find clusters of similar treatment conditions in two cell signaling networks: (I) apoptosis signaling network in human epithelial cancer cells treated with different combinations of TNF, epidermal growth factor (EGF) and insulin and (II) combination of signal transduction pathways stimulated by 21 different ligands based on AfCS double ligand screen data. For the analysis of the apoptosis signaling network we used the Cytokine compendium dataset where activity and concentration of 19 intracellular signaling molecules were measured to characterise apoptotic response to TNF, EGF and insulin. By projecting the original 19-dimensional space of intracellular signals into a low-dimensional space, Isomap was able to reconstruct clusters corresponding to different cytokine treatments that were identified with graph-based clustering. In comparison, Principal Component Analysis (PCA) and Partial Least Squares – Discriminant analysis (PLS-DA) were unable to find biologically meaningful clusters. We also showed that by using Isomap components for supervised classification with k-nearest neighbor (k-NN) and quadratic discriminant analysis (QDA), apoptosis intensity can be predicted for different combinations of TNF, EGF and insulin. Prediction accuracy was highest when early activation time points in the apoptosis signaling network were used to predict apoptosis rates at later time points. Extended Isomap also outperformed PCA on the AfCS double ligand screen data. Isomap identified more functionally coherent clusters than PCA and captured more information in the first two-components. The Isomap projection performs slightly worse when more signaling networks are analyzed; suggesting that the mapping function between cues and responses becomes increasingly non-linear when large signaling pathways are considered.ConclusionWe developed and applied extended Isomap approach for the analysis of cell signaling networks. Potential biological applications of this method include characterization, visualization and clustering of different treatment conditions (i.e. low and high doses of TNF) in terms of changes in intracellular signaling they induce.