Locally Linear Embedding Research Articles

Non-linear dimension reduction in factor-augmented vector autoregressions

This paper introduces non-linear dimension reduction in factor-augmented vector autoregressions to analyze the effects of different economic shocks. I argue that controlling for non-linearities between a large-dimensional dataset and the latent factors is particularly useful during turbulent times of the business cycle. In simulations, I show that non-linear dimension reduction techniques yield good forecasting performance, especially when data is highly volatile. In an empirical application, I identify a monetary policy as well as an uncertainty shock excluding and including observations of the COVID-19 pandemic. Those two applications suggest that the non-linear FAVAR approaches are capable of dealing with the large outliers caused by the COVID-19 pandemic and yield reliable results in both scenarios.

Variable selection for nonlinear dimensionality reduction of biological datasets through bootstrapping of correlation networks

Identifying the most relevant variables or features in massive datasets for dimensionality reduction can lead to improved and more informative display, faster computation times, and more explainable models of complex systems. Despite significant advances and available algorithms, this task generally remains challenging, especially in unsupervised settings. In this work, we propose a method that constructs correlation networks using all intervening variables and then selects the most informative ones based on network bootstrapping. The method can be applied in both supervised and unsupervised scenarios. We demonstrate its functionality by applying Uniform Manifold Approximation and Projection for dimensionality reduction to several high-dimensional biological datasets, derived from 4D live imaging recordings of hundreds of morpho-kinetic variables, describing the dynamics of thousands of individual leukocytes at sites of prominent inflammation. We compare our method with other standard ones in the field, such as Principal Component Analysis and Elastic Net, showing that it outperforms them. The proposed method can be employed in a wide range of applications, encompassing data analysis and machine learning.

Information maximization-based clustering of histopathology images using deep learning

Pancreatic cancer is one of the most adverse diseases and it is very difficult to treat because the cancer cells formed in the pancreas intertwine themselves with nearby blood vessels and connective tissue. Hence, the surgical procedure of treatment becomes complicated and it does not always lead to a cure. Histopathological diagnosis is the usual approach for cancer diagnosis. However, the pancreas remains so deep inside the body that experts sometimes struggle to detect cancer in it. Computer-aided diagnosis can come to the aid of pathologists in this scenario. It assists experts by supporting their diagnostic decisions. In this research, we carried out a deep learning-based approach to analyze histopathology images. We collected whole-slide images of KPC mice to implement this work. The pancreatic abnormalities observed in KPC mice develop similar histological features to human beings. We created random patches from whole-slide images. Then, a convolutional autoencoder framework was used to embed these patches into an integrated latent space. We applied ‘information maximization’, a deep learning clustering technique to cluster the identical patches in an unsupervised manner since our dataset does not have annotation. Moreover, Uniform manifold approximation and projection, a nonlinear dimension reduction technique was utilized to visualize the embedded patches in a 2-dimensional space. Finally, we calculated a few internal cluster validation metrics to determine the optimal cluster set. Our work concentrated on patch-based anomaly detection in the whole slide histopathology images of KPC mice.

Fault detection using generalized autoencoder with neighborhood restriction for electrical drive systems of high-speed trains

Over the past two decades, fault detection of high-speed trains has become an active issue in the transportation area. Recent work has demonstrated the benefits of autoencoder for fault detection in high-speed trains. The latent variable of autoencoder is generally viewed as features generated by the learning process, without explainable interpretation and assessment of the latent variable with respect to its information quality. In the context of representation learning, the latent variable of autoencoder plays an important role towards an fault detection. There is a lack of a methodical framework for explainable applications of AE-based technique to approach fault detection issues. This paper proposes a generalized autoencoder based on neighborhood restriction (NRGAE) to minimize the information loss caused by data compression and reconstruction. The main objective of this paper is to establish a framework for learning autoencoder-based fault detection in the electrical drive systems of high-speed trains. Meanwhile, this work introduces the concepts of mutual information and local linear embedding for electrical drive systems and fault detection specifications. A case study on an electrical drive system is given to illustrate the capability of proposed fault detection.

Multi-target HSIC-Lasso via exploiting target correlations

Multi-target regression (MTR) has been widely applied due to its compact structure and better computational efficiency in learning multiple related tasks, and has received extensive attention in machine learning, while great challenge arises from handling complex nonlinear relationships in high-dimensional data. In this study, we propose a novel approach to nonlinear dimensionality reduction in MTR. Specifically, the Hilbert–Schmidt Independence Criterion Lasso (HSIC-Lasso) is introduced to captures nonlinear dependencies between input–output and inter-target relationships in a unified framework. Additionally, sparse learning is incorporated to effectively eliminate redundant and irrelevant features. The proposed method is guaranteed to find an optimal subset of maximally predictive features with minimal redundancy, and explores target correlations by treating the targets as input features, which achieves a better learning of the target dependencies and thus improves the effectiveness of the feature selection. Extensive experiments on twelve public data sets have shown that the proposed methods outperform the compared methods in feature selection for the MTR problem.

Predicting Bordeaux red wine origins and vintages from raw gas chromatograms

Connecting chemical properties to various wine characteristics is of great interest to the science of olfaction as well as the wine industry. We explored whether Bordeaux wine chemical identities and vintages (harvest year) can be inferred from a common and affordable chemical analysis, namely, a combination of gas chromatography (GC) and electron ionization mass spectrometry. Using 12 vintages (within the 1990–2007 range) from 7 estates of the Bordeaux region, we report that, remarkably, nonlinear dimensionality reduction techniques applied to raw gas chromatograms recover the geography of the Bordeaux region. Using machine learning, we found that we can not only recover the estate perfectly from gas chromatograms, but also the vintage with up to 50% accuracy. Interestingly, we observed that the entire chromatogram is informative with respect to geographic location and age, thus suggesting that the chemical identity of a wine is not defined by just a few molecules but is distributed over a large chemical spectrum. This study demonstrates the remarkable potential of GC analysis to explore fundamental questions about the origin and age of wine.

Multiple-frequency attribute blending via adaptive uniform manifold approximation and projection and its application on hydrocarbon reservoir delineation

Multifrequency attribute blending is a highly effective tool for characterizing hydrocarbon reservoirs. It begins by extracting multifrequency attributes of seismic data based on time-frequency transformation. Subsequently, a blending algorithm is used to fuse the extracted multifrequency components, thereby obtaining the interpretation results of the interested reservoirs. The red-green-blue (RGB) algorithm is commonly used to fuse the multifrequency components. However, it should be noted that the RGB blending algorithm can only fuse three frequency components, i.e., the low-, middle-, and high-frequency components. Moreover, it can occasionally introduce ambiguities, making it difficult to interpret areas that appear white or yellow. To address these issues, we develop a workflow for multiple-frequency component analysis to delineate hydrocarbon reservoirs. First, we apply the generalized S-transform to obtain the multiple-frequency components of seismic data. Then, the correlation analysis is developed and implemented to select the sensitive frequency components. Finally, we use the uniform manifold approximation and projection, a nonlinear dimension reduction algorithm, to blend the extracted multiple-frequency components and obtain reservoir interpretation results. We apply the suggested workflow to synthetic data and a 3D field data volume to evaluate its effectiveness. Our mathematical analysis demonstrates that the suggested workflow can effectively fuse multiple-frequency components to accurately characterize hydrocarbon reservoirs.

Comparison of dimensionality reduction techniques for multi-variable spatiotemporal flow fields

In the field of fluid mechanics, it is a potential consensus that nonlinear dimensionality reduction (DR) techniques outperform linear methods. However, this conclusion has been obtained based on simple fluid phenomena and an incomplete evaluation system of dimensionality reduction algorithms. In this study, we use an improved evaluation system of DR methods to compare and evaluate the performance of four DR methods, including two linear techniques: Principal Component Analysis (PCA) and Independent Component Analysis (ICA), and two non-linear techniques: Isometric Mapping (ISOMAP) and Locally Linear Embedding (LLE). The four methods are applied to analyze a complex hydrodynamic flow field with cavitation by considering the joint features of multiple variables. Results show that LLE can capture redundant features that do not contribute to understanding the characteristics of flow fields, while ISOMAP is more suitable for handling datasets with multiple scales than LLE. PCA and ISOMAP can successfully capture the characteristics and evolution of flow fields. In addition, 3D supplementary information can assist ICA in improving the problem of identifying unstable flow field states.

Application of kernel principal component analysis for optical vector atomic magnetometry

Vector atomic magnetometers that incorporate electromagnetically induced transparency (EIT) allow for precision measurements of magnetic fields that are sensitive to the directionality of the observed field by virtue of fundamental physics. However, a practical methodology of accurately recovering the longitudinal angle of the local field through observations of EIT spectra has not been established. In this work, we address this problem of angle determination with an unsupervised machine learning algorithm utilizing nonlinear dimensionality reduction. The proposed algorithm was developed to interface with spectroscopic measurements from an EIT-based atomic rubidium magnetometer and uses kernel principal component analysis (KPCA) as an unsupervised feature extraction tool. The resulting KPCA features allow each EIT spectrum measurement to be represented by a single coordinate in a new reduced dimensional feature space, thereby streamlining the process of angle determination. A supervised support vector regression (SVR) machine was implemented to model the resulting relationship between the KPCA projections and field direction. If the magnetometer is configured so that the azimuthal angle of the field is defined with a polarization lock, the KPCA-SVR algorithm is capable of predicting the longitudinal angle of the local magnetic field within 1 degree of accuracy and the magnitude of the absolute field with a resolution of 70 nT. The combined scalar and angular sensitivity of this method make the KPCA-enabled EIT magnetometer competitive with conventional vector magnetometry methods. © 2023. All rights reserved.

Predicting cognitive decline in a low‐dimensional representation of brain morphology

AbstractBackgroundIdentifying early signs of neurodegeneration due to Alzheimer’s disease (AD) is a necessary first step towards preventing cognitive decline. Individual cortical thickness measures, available after processing anatomical magnetic resonance imaging (MRI), are sensitive markers of neurodegeneration. However, cortical decline in aging and high inter‐individual variability complicates the determination of AD‐related neurodegeneration on trajectories. Further, the high‐dimensional nature of these trajectories necessitate the use of a transformation into lower‐dimensional spaces in order to perform comparative statistical analyses, model decline, and identify criteria differentiating pathological from normal trajectories.MethodIn this project, we computed trajectories in a 2D representation of a 62‐dimensional manifold of individual cortical thickness measures. To compute this representation, we used a nonlinear dimension reduction algorithm called Uniform Manifold Approximation and Projection (UMAP). We first created a UMAP embedding on measurements from 6,237 cognitively healthy participants (3,556 women) aged 18 to 100 years old from the NOMIS database. We then projected longitudinal data from 537 mild cognitively impaired (MCI) subjects and 340 AD subjects from ADNI into this embedding. Each participant had multiple visits (n = 2), one year apart. Finally, differences in trajectories were analysed by clustering the reduced data and comparing the positional variations through time between MCI and AD subjects. The validity of the approach was verified through cross‐validation.ResultFirst, the embedding was shown to be positively associated (r = 0.65) with participants' age (see figure 1) in the NOMIS data. When projected in this space, differences between ADNI’s MCI and AD distributions were found (see figure 2). Average trajectories between clusters were shown to be significantly different between MCI and AD subjects (see figure 3). Moreover, we showed that some clusters and trajectories between clusters were more prone to host AD subjects. Finally, cross‐validation showed an accuracy of up to 72% at predicting cognitive decline in MCI participants over 2,000 bootstrap iterations.ConclusionA 2D low‐dimensional cortical thickness representation embeds sufficient discriminatory information as to predict decline to AD in MCI participants with strong accuracy.

Predicting cognitive decline in a low‐dimensional representation of brain morphology

AbstractBackgroundIdentifying early signs of neurodegeneration due to Alzheimer’s disease (AD) is a necessary first step towards preventing cognitive decline. Individual cortical thickness measures, available after processing anatomical magnetic resonance imaging (MRI), are sensitive markers of neurodegeneration. However, cortical decline in aging and high inter‐individual variability complicates the determination of AD‐related neurodegeneration on trajectories. Further, the high‐dimensional nature of these trajectories necessitate the use of a transformation into lower‐dimensional spaces in order to perform comparative statistical analyses, model decline, and identify criteria differentiating pathological from normal trajectories.MethodIn this project, we computed trajectories in a 2D representation of a 62‐dimensional manifold of individual cortical thickness measures. To compute this representation, we used a nonlinear dimension reduction algorithm called Uniform Manifold Approximation and Projection (UMAP). We first created a UMAP embedding on measurements from 6,237 cognitively healthy participants (3,556 women) aged 18 to 100 years old from the NOMIS database. We then projected longitudinal data from 537 mild cognitively impaired (MCI) subjects and 340 AD subjects from ADNI into this embedding. Each participant had multiple visits (n ≥ 2), one year apart. Finally, differences in trajectories were analysed by clustering the reduced data and comparing the positional variations through time between MCI and AD subjects. The validity of the approach was verified through cross‐validation.ResultFirst, the embedding was shown to be positively associated (r = 0.65) with participants' age (see figure 1) in the NOMIS data. When projected in this space, differences between ADNI’s MCI and AD distributions were found (see figure 2). Average trajectories between clusters were shown to be significantly different between MCI and AD subjects (see figure 3). Moreover, we showed that some clusters and trajectories between clusters were more prone to host AD subjects. Finally, cross‐validation showed an accuracy of up to 72% at predicting cognitive decline in MCI participants over 2,000 bootstrap iterations.ConclusionA 2D low‐dimensional cortical thickness representation embeds sufficient discriminatory information as to predict decline to AD in MCI participants with strong accuracy.

Manifold Learning: What, How, and Why

Manifold learning (ML), also known as nonlinear dimension reduction, is a set of methods to find the low-dimensional structure of data. Dimension reduction for large, high-dimensional data is not merely a way to reduce the data; the new representations and descriptors obtained by ML reveal the geometric shape of high-dimensional point clouds and allow one to visualize, denoise, and interpret them. This review presents the underlying principles of ML, its representative methods, and their statistical foundations, all from a practicing statistician's perspective. It describes the trade-offs and what theory tells us about the parameter and algorithmic choices we make in order to obtain reliable conclusions.

Manifold Structure Analysis of Tactical Network Traffic Matrix Based on Maximum Variance Unfolding Algorithm

As modern weapons and equipment undergo increasing levels of informatization, intelligence, and networking, the topology and traffic characteristics of battlefield data networks built with tactical data links are becoming progressively complex. In this paper, we employ a traffic matrix to model the tactical data link network. We propose a method that utilizes the Maximum Variance Unfolding (MVU) algorithm to conduct nonlinear dimensionality reduction analysis on high-dimensional open network traffic matrix datasets. This approach introduces novel ideas and methods for future applications, including traffic prediction and anomaly analysis in real battlefield network environments.

Defining the Drivers of Idelalisib-Related Early Autoimmune Toxicity in Chronic Lymphocytic Leukemia

Chronic lymphocytic leukemia (CLL) patients with disease resistant to BTK and BCL-2 inhibitors can potentially be treated with PI3K inhibitors to overcome resistance and improve outcomes. PI3K inhibitors such as idelalisib and duvelisib have shown promising efficacy in CLL patients, but severe toxicities remain a significant hurdle to their implementation in the clinic (PMID: 24615777). By investigating the molecular mechanisms that lead to severe early autoimmune toxicity with idelalisib, we hope to predict or prevent toxicity, thus expanding the use of this efficacious PI3K inhibitor class of drugs. To this end, the Brown lab showed that idelalisib-induced hepatitis is autoimmune. CyTOF analysis in CLL patients with early severe toxicity in a front-line idelalisib clinical trial cohort showed a significant decrease in CD4+ T regulatory cells (Treg) with a relative shift in favor of T effectors in peripheral blood. In addition, the Brown lab showed that the Th17 pathway is more active at baseline, with a further increase in patients with toxicity (PMID: 27247136, 36732326). Since the CYTOF data is restricted to fewer mostly T-cell markers in our studies, an unbiased single cell RNA sequencing (scRNA-seq) approach is warranted to explore the full effects of idelalisib on immune cells and CLL cells in patients with and without autoimmune toxicity. The question remains what molecular mechanisms drive the idelalisib-induced decrease in the Treg population, its effect on natural killer (NK) and CD8 T-cell function, and the activation of the Th17 pathway in CLL patients with early toxicity. We therefore carried out the first systematic effort to understand the mechanisms of idelalisib toxicity in molecular detail using combined scRNA and scTCR-seq. In this study, we profile T-cell enriched PBMCs of matched pre-treatment and on-idelalisib treatment samples from CLL patients with early severe toxicity and without any toxicity (n=4 CLL patients from each group, a total of 16 samples) using 10X single cell transcriptomic and TCR profiling. Raw sequencing data files were processed using the Cell Ranger suite (version 7.1.0) from 10x Genomics. Sample identification based on SNP information was done using the Popscle suite. Seurat package (version 4.9.9.9) was used for the single cell analysis. After filtering cells with fewer than 600 features, less than 1000 UMIs, mitochondrial content higher than 15%, 51,487 cells were left for the downstream analysis. To explore transcriptional heterogeneity and perform initial cell clustering, principal component analysis and nonlinear dimensional reduction using UMAP were applied using 20 dimensions. Leiden algorithm was used for cluster identification. Cell types were identified using Azimuth with the PBMC reference dataset and confirmed with expression plots. The initial observations of the cell proportions show that baseline samples from patients with early idelalisib toxicity have fewer Tregs, NK, and CD4+ cytotoxic T (CTL) cells compared to patients without toxicity. In addition, we observed a decline in Tregs, gamma-delta (γδ) T, and CD4+ CTL cells and a concurrent increase in proliferative CD4+ and CD8+ T-cells and proliferative NK on idelalisib treatment in the early toxicity group. On the contrary, CD4+ CTL and CD4+ T effector memory (TEM) cells are enriched with idelalisib therapy in patients without toxicities. At the same time, more CD4+ T-Cell Memory (TCM), γδ T, CD8+ naive T, CD8+ TCM, and Mucosal-associated invariant T (MAIT) cells are present in pretreatment samples from patients with toxicity compared to patients without toxicity. Furthermore, proliferative T-cells are depleted with idelalisib therapy in patients without toxicity whereas they are enriched in patients with early toxicity. To further characterize idelalisib-induced depletion of Tregs, we reclustered them to identify distinct subgroups and found particular clusters of Treg were depleted in patients with early toxicity. In the non-toxicity group, more diversity of Tregs was identified with depletion of some clusters but also an increase in a cluster. At present, we are integrating the scRNA & scTCR-seq data to characterize the cellular diversity and heterogeneous phenotypic states of the T-cell populations and updated results unraveling molecular mechanisms that drive early severe toxicity in CLL patients with idelalisib treatment will be presented at the meeting.

Soft sensor for predicting indoor PM2.5 concentration in subway with adaptive boosting deep learning model

Public health depends on indoor air quality (IAQ), hence soft measurement techniques must be implemented in the subway environment for more precise and reliable monitoring of indoor particulate matter concentration levels. Adaptive boosting (AdaBoost), an ensemble learning technique, is simple to code and less prone to overfitting. Compared to a single model, it is better able to take into consideration the intricate elements included in air quality data. It is suggested to use an adaptive boosting of long short-term memory (AdaBoost-LSTM) model and kernel principal component analysis (KPCA) for ensemble learning. The kernel function and PCA are first coupled to create KPCA, which is a nonlinear dimensionality reduction method for IAQ. This removes the negative impacts of noise interference. The learning performance of LSTM is then enhanced using AdaBoost as an ensemble learning technique. The KPCA-AdaBoost-LSTM model can deliver higher modeling performance, according to the results. The R2 reached 0.9007 and 0.8995 when predicting PM2.5 in the hall and platform. SHapley Additive exPlanations (SHAP) analysis was used to interpret the input contributions of the model, enhancing the interpretability and transparency of the proposed soft sensor.

Human Activity Recognition based on Local Linear Embedding and Geodesic Flow Kernel on Grassmann manifolds

Human Activity Recognition (HAR) plays a crucial role in various applications(e.g., medical treatment, video surveillance and sports monitoring). Transfer learning is a promising solution to cross-domain identification problems in HAR. However, existing methods usually ignore the negative transfer caused by using the features of each source domain in equal proportions, as well as the distribution difference between the source and target domains. In this paper, an HAR method based on manifold learning is proposed. Firstly, the similarity between the domain and multiple source domains is calculated using the Multi-Kernel-Maximum Mean Difference (MK-MMD), and the source domain most similar to the target domain is selected as the optimal source domain in the transfer task. Secondly, Locally Linear Embedding (LLE) is leveraged to reduce the dimensionality of both optimal source domain and target domain data to remove redundant information, and the Geodesic Flow Kernel (GFK) is utilized to project low-dimensional data into the Grassmann manifold space and reduce the distribution difference between the two domains. Finally, the source domain action training model is applied to the target domain. Three public datasets (i.e., PAMAP2, OPPORTUNITY and UCI DSADS) are utilized to validate the effectiveness of the proposed approach. Experimental results are presented to demonstrate that the proposed HAR method can predict a large number of unlabeled samples in the target domain while preserving the original data structure.

Performance evaluation of KPCA pre-imaging methods for speech signal denoising

<p class="Abstract">Kernel principal component analysis (KPCA) has gained wider interest amongst the researchers in nonlinear dimensionality reduction, data compression, feature extraction, and denoising applications. In KPCA, data from low dimensional input space implicitly mapped to higher dimensional feature space where linear PCA is enforced. However, for denoising application data need to invert back to input space, which is impossible and known as pre-imaging problems. Over recent years, several pre-imaging methods have proposed each with its benefits and disadvantages. In this paper, we evaluated the performance for selected pre-imaging methods for denoising speech signal, whose intelligibility and quality degraded by background noises. Further, we extend our work by comparing the performance of these pre-imaging methods by objective evaluation of denoised speech signal on generated toy examples and NOIZUES database.</p>

Model-based evaluation of spatiotemporal data reduction methods with unknown ground truth through optimal visualization and interpretability metrics.

Optimizing and benchmarking data reduction methods for dynamic or spatial visualization and interpretation (DSVI) face challenges due to many factors, including data complexity, lack of ground truth, time-dependent metrics, dimensionality bias and different visual mappings of the same data. Current studies often focus on independent static visualization or interpretability metrics that require ground truth. To overcome this limitation, we propose the MIBCOVIS framework, a comprehensive and interpretable benchmarking and computational approach. MIBCOVIS enhances the visualization and interpretability of high-dimensional data without relying on ground truth by integrating five robust metrics, including a novel time-ordered Markov-based structural metric, into a semi-supervised hierarchical Bayesian model. The framework assesses method accuracy and considers interaction effects among metric features. We apply MIBCOVIS using linear and nonlinear dimensionality reduction methods to evaluate optimal DSVI for four distinct dynamic and spatial biological processes captured by three single-cell data modalities: CyTOF, scRNA-seq and CODEX. These data vary in complexity based on feature dimensionality, unknown cell types and dynamic or spatial differences. Unlike traditional single-summary score approaches, MIBCOVIS compares accuracy distributions across methods. Our findings underscore the joint evaluation of visualization and interpretability, rather than relying on separate metrics. We reveal that prioritizing average performance can obscure method feature performance. Additionally, we explore the impact of data complexity on visualization and interpretability. Specifically, we provide optimal parameters and features and recommend methods, like the optimized variational contractive autoencoder, for targeted DSVI for various data complexities. MIBCOVIS shows promise for evaluating dynamic single-cell atlases and spatiotemporal data reduction models.

Similarity-assisted variational autoencoder for nonlinear dimension reduction with application to single-cell RNA sequencing data

BackgroundDeep generative models naturally become nonlinear dimension reduction tools to visualize large-scale datasets such as single-cell RNA sequencing datasets for revealing latent grouping patterns or identifying outliers. The variational autoencoder (VAE) is a popular deep generative method equipped with encoder/decoder structures. The encoder and decoder are useful when a new sample is mapped to the latent space and a data point is generated from a point in a latent space. However, the VAE tends not to show grouping pattern clearly without additional annotation information. On the other hand, similarity-based dimension reduction methods such as t-SNE or UMAP present clear grouping patterns even though these methods do not have encoder/decoder structures.ResultsTo bridge this gap, we propose a new approach that adopts similarity information in the VAE framework. In addition, for biological applications, we extend our approach to a conditional VAE to account for covariate effects in the dimension reduction step. In the simulation study and real single-cell RNA sequencing data analyses, our method shows great performance compared to existing state-of-the-art methods by producing clear grouping structures using an inferred encoder and decoder. Our method also successfully adjusts for covariate effects, resulting in more useful dimension reduction.ConclusionsOur method is able to produce clearer grouping patterns than those of other regularized VAE methods by utilizing similarity information encoded in the data via the highly celebrated UMAP loss function.

Feature Extraction from Satellite-Derived Hydroclimate Data: Assessing Impacts on Various Neural Networks for Multi-Step Ahead Streamflow Prediction

Enhancing the generalization capability of time-series models for streamflow prediction using dimensionality reduction (DR) techniques remains a major challenge in water resources management (WRM). In this study, we investigated eight DR techniques and their effectiveness in mitigating the curse of dimensionality, which hinders the performance of machine learning (ML) algorithms in the field of WRM. Our study delves into the most non-linear unsupervised representative DR techniques, including principal component analysis (PCA), kernel PCA (KPCA), multi-dimensional scaling (MDS), isometric mapping (ISOMAP), locally linear embedding (LLE), t-distributed stochastic neighbor embedding (t-SNE), Laplacian eigenmaps (LE), and autoencoder (AE), examining their effectiveness in multi-step ahead (MSA) streamflow prediction. In this study, we conducted a conceptual comparison of these techniques. Subsequently, we focused on their performance in four different case studies in the USA. Moreover, we assessed the quality of the transformed feature spaces in terms of the MSA streamflow prediction improvement. Through our investigation, we gained valuable insights into the performance of different DR techniques within linear/dense/convolutional neural network (CNN)/long short-term memory neural network (LSTM) and autoregressive LSTM (AR-LSTM) architectures. This study contributes to a deeper understanding of suitable feature extraction techniques for enhancing the capabilities of the LSTM model in tackling high-dimensional datasets in the realm of WRM.