Nonlinear Dimensionality Reduction Method Research Articles

Learnable faster kernel-PCA for nonlinear fault detection: Deep autoencoder-based realization

Kernel principal component analysis (KPCA) is a well-recognized nonlinear dimensionality reduction method that has been widely used in nonlinear fault detection tasks. As a kernel trick-based method, KPCA inherits two major problems. First, the form and the parameters of the kernel function are usually selected blindly, depending seriously on trial-and-error. As a result, there may be serious performance degradation in case of inappropriate selections. Second, at the online monitoring stage, KPCA has much computational burden and poor real-time performance, because the kernel method requires to leverage all the offline training data. In this work, to deal with the two drawbacks, a learnable faster realization of the conventional KPCA is proposed. The core idea is to parameterize all feasible kernel functions using the novel nonlinear DAE-FE (deep autoencoder based feature extraction) framework and propose DAE-PCA (deep autoencoder based principal component analysis) approach in detail. The proposed DAE-PCA method is proved to be equivalent to KPCA but has more advantage in terms of automatic searching of the most suitable nonlinear high-dimensional space according to the inputs, which helps to improve the accuracy of fault detection. Furthermore, the online computational efficiency improves by many times compared with the conventional KPCA. Finally, the Tennessee Eastman (TE) process benchmark and wastewater treatment plant (WWTP) benchmark are employed to illustrate the effectiveness of the proposed method, where the average fault detection rates of DAE-PCA are at least 0.27% and 4.69% higher than those of other methods, and its online computational efficiency is faster 90.48% and 24.57% times than that of KPCA respectively.

Interpretation of autoencoder-learned collective variables using Morse-Smale complex and sublevelset persistent homology: An application on molecular trajectories.

Dimensionality reduction often serves as the first step toward a minimalist understanding of physical systems as well as the accelerated simulations of them. In particular, neural network-based nonlinear dimensionality reduction methods, such as autoencoders, have shown promising outcomes in uncovering collective variables (CVs). However, the physical meaning of these CVs remains largely elusive. In this work, we constructed a framework that (1) determines the optimal number of CVs needed to capture the essential molecular motions using an ensemble of hierarchical autoencoders and (2) provides topology-based interpretations to the autoencoder-learned CVs with Morse-Smale complex and sublevelset persistent homology. This approach was exemplified using a series of n-alkanes and can be regarded as a general, explainable nonlinear dimensionality reduction method.

Identification of control equations using low-dimensional flow representations of pitching airfoil

This study investigates the application of data-driven modeling techniques for understanding the complex dynamics of pitching airfoils at low Reynolds numbers and high angles of attack. Linear and nonlinear dimensionality reduction methods, namely principal component analysis (PCA) and isometric mapping (ISOMAP), are employed to obtain low-dimensional representations of the flow field. Subsequently, sparse identification of nonlinear dynamics (SINDy) is utilized to model the governing equations. The key findings are as follows: PCA primarily captures linear information, with the first two to three dimensions maintaining relatively low reconstruction errors. In contrast, ISOMAP excels in capturing nonlinear features, exhibiting noticeably smaller reconstruction errors. The main information is concentrated in the two-dimensional plane constructed by PCA1 and PCA2 (or ISOMAP1 and ISOMAP2). Differences in trajectory planes formed by combinations of other axes reflect flow field disparities. ISOMAP provides a nonlinear low-dimensional representation, advantageous for capturing nonlinear relationships between flow field characteristics and governing equations. The combination of ISOMAP and SINDy yields virtually no errors in identifying governing equations. Conversely, PCA and SINDy result in significantly different linear trajectories, leading to higher reconstruction errors. The identified governing equations using ISOMAP and SINDy remain consistent across different datasets, demonstrating the method's stability and robustness in accurately characterizing flow field properties under similar conditions.

Automatic Active Lesion Tracking in Multiple Sclerosis Using Unsupervised Machine Learning.

Identifying active lesions in magnetic resonance imaging (MRI) is crucial for the diagnosis and treatment planning of multiple sclerosis (MS). Active lesions on MRI are identified following the administration of Gadolinium-based contrast agents (GBCAs). However, recent studies have reported that repeated administration of GBCA results in the accumulation of Gd in tissues. In addition, GBCA administration increases health care costs. Thus, reducing or eliminating GBCA administration for active lesion detection is important for improved patient safety and reduced healthcare costs. Current state-of-the-art methods for identifying active lesions in brain MRI without GBCA administration utilize data-intensive deep learning methods. To implement nonlinear dimensionality reduction (NLDR) methods, locally linear embedding (LLE) and isometric feature mapping (Isomap), which are less data-intensive, for automatically identifying active lesions on brain MRI in MS patients, without the administration of contrast agents. Fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images were included in the multiparametric MRI dataset used in this study. Subtracted pre- and post-contrast T1-weighted images were labeled by experts as active lesions (ground truth). Unsupervised methods, LLE and Isomap, were used to reconstruct multiparametric brain MR images into a single embedded image. Active lesions were identified on the embedded images and compared with ground truth lesions. The performance of NLDR methods was evaluated by calculating the Dice similarity (DS) index between the observed and identified active lesions in embedded images. LLE and Isomap, were applied to 40 MS patients, achieving median DS scores of 0.74 ± 0.1 and 0.78 ± 0.09, respectively, outperforming current state-of-the-art methods. NLDR methods, Isomap and LLE, are viable options for the identification of active MS lesions on non-contrast images, and potentially could be used as a clinical decision tool.

Nonlinear dimensionality reduction with q-Gaussian distribution

In recent years, the dimensionality reduction has become more important as the number of dimensions of data used in various tasks such as regression and classification has increased. As popular nonlinear dimensionality reduction methods, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) have been proposed. However, the former outputs only one low-dimensional space determined by the t-distribution and the latter is difficult to control the distribution of distance between each pair of samples in low-dimensional space. To tackle these issues, we propose novel t-SNE and UMAP extended by q-Gaussian distribution, called q-Gaussian-distributed stochastic neighbor embedding (q-SNE) and q-Gaussian-distributed uniform manifold approximation and projection (q-UMAP). The q-Gaussian distribution is a probability distribution derived by maximizing the tsallis entropy by escort distribution with mean and variance, and a generalized version of Gaussian distribution with a hyperparameter q. Since the shape of the q-Gaussian distribution can be tuned smoothly by the hyperparameter q, q-SNE and q-UMAP can in- tuitively derive different embedding spaces. To show the quality of the proposed method, we compared the visualization of the low-dimensional embedding space and the classification accuracy by k-NN in the low-dimensional space. Empirical results on MNIST, COIL-20, OliverttiFaces and FashionMNIST demonstrate that the q-SNE and q-UMAP can derive better embedding spaces than t-SNE and UMAP.

Predicting S. aureus antimicrobial resistance with interpretable genomic space maps.

Increasing antimicrobial resistance (AMR) represents a global healthcare threat. To decrease the spread of AMR and associated mortality, methods for rapid selection of optimal antibiotic treatment are urgently needed. Machine learning (ML) models based on genomic data to predict resistant phenotypes can serve as a fast screening tool prior to phenotypic testing. Nonetheless, many existing ML methods lack interpretability. Therefore, we present a methodology for visualization of sequence space and AMR prediction based on the non-linear dimensionality reduction method - generative topographic mapping (GTM). This approach, applied to AMR data of >5000 S. aureus isolates retrieved from the PATRIC database, yielded GTM models with reasonable accuracy for all drugs (balanced accuracy values ≥0.75). The Generative Topographic Maps (GTMs) represent data in the form of illustrative maps of the genomic space and allow for antibiotic-wise comparison of resistant phenotypes. The maps were also found to be useful for the analysis of genetic determinants responsible for drug resistance. Overall, the GTM-based methodology is a useful tool for both the illustrative exploration of the genomic sequence space and AMR prediction.

Abstract B063: Investigation of tumor heterogeneity using integrated single-cell RNA sequence data based on HER2 status in patients with breast cancer

Abstract [Introduction] Human epidermal growth factor receptor 2 (HER2), which is characterized by ERBB2 amplification, is one of the important markers for treatment decision related to breast cancer. Many targeted therapies for HER2-positive (immunohistochemistry [IHC] scores of 3+ or 2+ with an in situ hybridization [ISH] gene amplification) or HER2-low (IHC score 1+ or 2+ with no ISH gene amplification) breast cancers have been extensively developed thus far. Meanwhile, tumoral heterogeneity is considered one of the mechanisms for drug resistance; however, it has not been fully elucidated on each HER2 status at single-cell level. Therefore, an integrated analysis of the breast cancer single-cell gene expression data of both public datasets and our cohort was used to investigate tumor heterogeneity based on HER2 status. [Methods] We collected 21 and 6 scRNA-seq samples of primary breast cancer from the public Gene Expression Omnibus (GEO) and our institution, respectively. A total of 27 samples included HER2-positive cases (pure HER2 cases: estrogen receptor [ER]-negative/HER2-positive and Luminal-HER2 cases: ER-positive/HER2-positive) and Luminal cases (ER-positive/HER2-negative). These datasets were imported into R software version 4.2.0. and transformed into Seurat objects with the package Seurat version 4.3.0. UMAP plots, which is a non-linear dimension reduction method, were used for clustering analyses. Seurat in R was used to generate UMAP, feature, and violin plots. Additionally, we performed pathway enrichment analysis with the significant gene list from each cluster between the ERBB2-high and ERBB2-low groups. [Results] Clustering analysis revealed heterogeneous distribution in each gene related to breast cancer (ESR1, PGR, ERBB2, and MKI67). One of the clusters revealed high MKI67 expression. ERBB2 expressions were diffusely distributed on each cluster of pure HER2 cases; however, the expressions were considerably higher in one of the clusters in cases of Luminal-HER2. The ERBB2 expression in Luminal cases, which included HER2-low status, was lower than that in HER2-positive cases, albeit slight expression was observed. Cell proliferation factors, including IGF, IGF1R, and EGF, were included in the ERBB2-high expression group compared with the ERBB2-low expression group for pathway analysis. Further, we examined typical gene expressions which are associated with markers related to breast cancer, cancer stem cell, and epithelial-to-mesenchymal transition in each case and revealed heterogeneous patterns across patients. [Conclusion] Heterogeneous ERBB2 expression distribution was observed in HER2-positive cases, and slight ERBB2 expression was identified in the Luminal cases. Moreover, Luminal-HER2 cases could be considered a more heterogeneous subtype compared with pure HER2 cases. Additionally, gene expressions of typical gene markers varied across patients. These results indicated that breast cancer displays heterogeneous patterns on each HER2 status not only intra-tumoral heterogeneity but also inter-patient heterogeneity. Citation Format: Sho Shiino, Momoko Tokura, Jun Nakayama, Masayuki Yoshida, Akihiko Suto, Yusuke Yamamoto. Investigation of tumor heterogeneity using integrated single-cell RNA sequence data based on HER2 status in patients with breast cancer [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Advances in Breast Cancer Research; 2023 Oct 19-22; San Diego, California. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_1):Abstract nr B063.

Soft sensor for predicting indoor PM2.5 concentration in subway with adaptive boosting deep learning model

Public health depends on indoor air quality (IAQ), hence soft measurement techniques must be implemented in the subway environment for more precise and reliable monitoring of indoor particulate matter concentration levels. Adaptive boosting (AdaBoost), an ensemble learning technique, is simple to code and less prone to overfitting. Compared to a single model, it is better able to take into consideration the intricate elements included in air quality data. It is suggested to use an adaptive boosting of long short-term memory (AdaBoost-LSTM) model and kernel principal component analysis (KPCA) for ensemble learning. The kernel function and PCA are first coupled to create KPCA, which is a nonlinear dimensionality reduction method for IAQ. This removes the negative impacts of noise interference. The learning performance of LSTM is then enhanced using AdaBoost as an ensemble learning technique. The KPCA-AdaBoost-LSTM model can deliver higher modeling performance, according to the results. The R2 reached 0.9007 and 0.8995 when predicting PM2.5 in the hall and platform. SHapley Additive exPlanations (SHAP) analysis was used to interpret the input contributions of the model, enhancing the interpretability and transparency of the proposed soft sensor.

Model-based evaluation of spatiotemporal data reduction methods with unknown ground truth through optimal visualization and interpretability metrics.

Optimizing and benchmarking data reduction methods for dynamic or spatial visualization and interpretation (DSVI) face challenges due to many factors, including data complexity, lack of ground truth, time-dependent metrics, dimensionality bias and different visual mappings of the same data. Current studies often focus on independent static visualization or interpretability metrics that require ground truth. To overcome this limitation, we propose the MIBCOVIS framework, a comprehensive and interpretable benchmarking and computational approach. MIBCOVIS enhances the visualization and interpretability of high-dimensional data without relying on ground truth by integrating five robust metrics, including a novel time-ordered Markov-based structural metric, into a semi-supervised hierarchical Bayesian model. The framework assesses method accuracy and considers interaction effects among metric features. We apply MIBCOVIS using linear and nonlinear dimensionality reduction methods to evaluate optimal DSVI for four distinct dynamic and spatial biological processes captured by three single-cell data modalities: CyTOF, scRNA-seq and CODEX. These data vary in complexity based on feature dimensionality, unknown cell types and dynamic or spatial differences. Unlike traditional single-summary score approaches, MIBCOVIS compares accuracy distributions across methods. Our findings underscore the joint evaluation of visualization and interpretability, rather than relying on separate metrics. We reveal that prioritizing average performance can obscure method feature performance. Additionally, we explore the impact of data complexity on visualization and interpretability. Specifically, we provide optimal parameters and features and recommend methods, like the optimized variational contractive autoencoder, for targeted DSVI for various data complexities. MIBCOVIS shows promise for evaluating dynamic single-cell atlases and spatiotemporal data reduction models.

Trade-offs in the latent representation of microstructure evolution

Characterizing and quantifying microstructure evolution is critical to forming quantitative relationships between material processing conditions, resulting microstructure, and observed properties. Machine-learning methods are increasingly accelerating the development of these relationships by treating microstructure evolution as a pattern recognition problem, discovering relationships explicitly or implicitly. These methods often rely on identifying low-dimensional microstructural fingerprints as latent variables. However, using inappropriate latent variables can lead to challenges in learning meaningful relationships. In this work, we survey and discuss the ability of various linear and nonlinear dimensionality reduction methods including principal component analysis, autoencoders, and diffusion maps to quantify and characterize the learned latent space microstructural representations and their time evolution. We characterize latent spaces by their ability to represent high-dimensional microstructural data in terms of compression achieved as a function of the number of latent dimensions required to represent the data accurately, their accuracy based on their reconstruction performance, and the smoothness of the microstructural trajectories in latent dimension. We quantify these metrics for common microstructure evolution problems in material science including spinodal decomposition of a binary metallic alloy, thin film deposition of a binary metallic alloy, dendritic growth, and grain growth in a polycrystal. This study provides considerations and guidelines for choosing dimensionality reduction methods when considering materials problems that involve high dimensional data and a variety of features over a range of lengths and time scales.

A software defect prediction method based on learnable three-line hybrid feature fusion

Software defect prediction (SDP) plays a crucial role in ensuring the security and quality of software systems. However, it faces challenges posed by high-dimensional features present in software defect datasets and the limited effectiveness of traditional nonlinear dimensionality reduction methods in extracting essential feature information. To address these issues, we propose a novel approach called learnable three-line hybrid feature fusion (LTHFFA), which incorporates the principle of three-line hybrid breeding into feature fusion for the first time. In this method, three distinct dimensionality reduction techniques are initially employed to obtain three separate sets of features. Subsequently, a learnable weight factor feature fusion method is proposed to facilitate automatically learn and dynamically update of feature weights. By integrating the three feature sets based on the principle of three-line hybrid breeding, we derive learnable three-line hybrid fusion features. These features are then utilized in the context of software defect prediction. Experimental results demonstrate the superior performance of LTHFFA compared to nine other dimensionality reduction methods across seventeen publicly available software defect datasets. LTHFFA exhibits the ability to effectively integrate multiple feature sets, reduce feature redundancy, and enhance predictive accuracy. Moreover, statistical analysis using Friedman ranking and Holm's post-hoc test confirms the significant advantage of LTHFFA over alternative dimensionality reduction methods.

Nonlinear dimensionality reduction methods for potentiometric multisensor systems data analysis

AbstractElectrochemical multisensor systems were proven to be a very perspective research direction in modern analytical chemistry. The multisensor approach assumes an employment of cross‐sensitive chemical sensors in combination with multivariate data processing methods. Dimensionality reduction of the data obtained from multisensor systems is a very important step and it is mostly based on the traditional tools of chemometrics, such as Principal Component Analysis (PCA). In case of chemically complex samples, the response of multisensor systems may have a complex nonlinear nature and the use of linear modelling methods does not seem optimal. However, the potential of nonlinear dimensionality reduction methods in the processing of multisensor data has not yet been systematically studied. In this report we aim to fill this gap and assess the performance of various nonlinear dimensionality reduction tools: Isomap, Self‐Organizing Kohonen Maps, and Autoencoder. These methods were explored using three datasets from potentiometric multisensor systems obtained in various real applications. It was shown that nonlinear dimensionality reduction methods give the possibility to obtain additional and more detailed information about the analyzed objects/processes compared to PCA. However, calculation time for nonlinear dimensionality reduction methods essentially exceeds that for PCA, and it can be a limiting factor for application of such algorithms.

Absence of enterotypes in the human gut microbiomes reanalyzed with non-linear dimensionality reduction methods.

Enterotypes of the human gut microbiome have been proposed to be a powerful prognostic tool to evaluate the correlation between lifestyle, nutrition, and disease. However, the number of enterotypes suggested in the literature ranged from two to four. The growth of available metagenome data and the use of exact, non-linear methods of data analysis challenges the very concept of clusters in the multidimensional space of bacterial microbiomes. Using several published human gut microbiome datasets of variable 16S rRNA regions, we demonstrate the presence of a lower-dimensional structure in the microbiome space, with high-dimensional data concentrated near a low-dimensional non-linear submanifold, but the absence of distinct and stable clusters that could represent enterotypes. This observation is robust with regard to diverse combinations of dimensionality reduction techniques and clustering algorithms.

Laplacian-based Cluster-Contractive t-SNE for High-Dimensional Data Visualization

Dimensionality reduction techniques aim at representing high-dimensional data in low-dimensional spaces to extract hidden and useful information or facilitate visual understanding and interpretation of the data. However, few of them take into consideration the potential cluster information contained implicitly in the high-dimensional data. In this article, we propose Lap tSNE, a new graph-layout nonlinear dimensionality reduction method based on t-SNE, one of the best techniques for visualizing high-dimensional data as 2D scatter plots. Specifically, Lap tSNE leverages the eigenvalue information of the graph Laplacian to shrink the potential clusters in the low-dimensional embedding when learning to preserve the local and global structure from high-dimensional space to low-dimensional space. It is nontrivial to solve the proposed model because the eigenvalues of normalized symmetric Laplacian are functions of the decision variable. We provide a majorization-minimization algorithm with convergence guarantee to solve the optimization problem of Lap tSNE and show how to calculate the gradient analytically, which may be of broad interest when considering optimization with Laplacian-composited objective. We evaluate our method by a formal comparison with state-of-the-art methods on seven benchmark datasets, both visually and via established quantitative measurements. The results demonstrate the superiority of our method over baselines such as t-SNE and UMAP. We also provide out-of-sample extension, large-scale extension, and mini-batch extension for our Lap tSNE to facilitate dimensionality reduction in various scenarios.

Machine learning methods for nonlinear dimensionality reduction of the thermospheric density field

Accurate prediction of the thermospheric density field has recently been gaining a lot of attention, due to an outstanding increase in space operations in Low-Earth Orbit, in the context of the NewSpace. In order to model such high-dimensional, non-Gaussian systems, Reduced-Order Models (ROMs) have been developed against existing physics-based density models. In general, the data-driven reconstruction of dynamical systems usually consists of two steps of compression and prediction. In this paper, we focus on the compression step and assess state-of-the-art order reduction methodologies such as autoencoders, linear, and nonlinear Principal Component Analysis (PCA). We show that Kernel-based PCA, a nonlinear generalization of PCA, can perform better than Neural Networks in representing the model in low-dimensional space in terms of accuracy, computational time, and energy consumption for both the Lorenz system, a chaotic dynamical system developed in the context of atmospheric modeling, and the thermospheric density.

A normalized differential sequence feature encoding method based on amino acid sequences.

Protein interactions are the foundation of all metabolic activities of cells, such as apoptosis, the immune response, and metabolic pathways. In order to optimize the performance of protein interaction prediction, a coding method based on normalized difference sequence characteristics (NDSF) of amino acid sequences is proposed. By using the positional relationships between amino acids in the sequences and the correlation characteristics between sequence pairs, NDSF is jointly encoded. Using principal component analysis (PCA) and local linear embedding (LLE) dimensionality reduction methods, the coded 174-dimensional human protein sequence vector is extracted using sequence features. This study compares the classification performance of four ensemble learning methods (AdaBoost, Extra trees, LightGBM, XGBoost) applied to PCA and LLE features. Cross-validation and grid search methods are used to find the best combination of parameters. The results show that the accuracy of NDSF is generally higher than that of the sequence matrix-based coding method (MOS) coding method, and the loss and coding time can be greatly reduced. The bar chart of feature extraction shows that the classification accuracy is significantly higher when using the linear dimensionality reduction method, PCA, compared to the nonlinear dimensionality reduction method, LLE. After classification with XGBoost, the model accuracy reaches 99.2%, which provides the best performance among all models. This study suggests that NDSF combined with PCA and XGBoost may be an effective strategy for classifying different human protein interactions.

Feature importance-based interpretation of UMAP-visualized polymer space.

Dimensionality reduction (DR) techniques are used for various purposes such as exploratory data analysis. A commonly employed linear DR technique is principal component analysis (PCA), which is one of the most popular methods for DR. Owing to its linear nature, PCA enables the determination of axes in a low-dimensional space and the calculation of corresponding loading vectors. However, PCA cannot necessarily extract important features of non-linearly distributed data. This study presents a technique aimed at aiding the interpretation of data reduced through non-linear DR methods. In the proposed method, non-linear dimensionally reduced data was clustered via a density-based clustering method. Thereafter, the obtained cluster labels were classified by random forest (RF) classifiers. Further, feature importance (FI) of RF classifiers and Spearman's rank correlation coefficients between predictive probabilities to obtained clusters and original feature values were utilized for characterizing the visualized dimensionally reduced data. The results revealed that the proposed method can provide the interpretable FI-based images of the handwritten digits dataset. Moreover, the proposed method was also applied to the polymer dataset. The study found that incorporating signed FI was advantageous in achieving a meaningful interpretation. Furthermore, Gaussian process regression was utilized to produce intuitive FI-based heatmaps on a 2-dimensional space for greater ease of understanding. Additionally, to enhance the interpretability of the obtained clusters, a feature selection technique called Boruta was applied. The Boruta feature selection method worked effectively to interpret the obtained clusters with limited and commonly important features. Additionally, the study suggested that computing FI solely from substructure-based descriptors could further enhance the interpretability of the results. Finally, the automation of the proposed method was investigated, and through maximizing the target score based on the quality of both the DR and clustering, indicative results were automatically obtained for both the handwritten digits and polymer datasets.

Rapid detection of lignin content in corn straw based on Laplacian Eigenmaps

Lignin is an essential components of corn stalk and has a wide range of application. To realize the rapid detection of lignin content in corn straw and increase the detection accuracy, we initially preprocess the gathered corn straw near-infrared spectral data with Standard Normal Variable transformation (SNV). After that, the nonlinear dimensionality reduction method Laplacian Eigenmaps (LE) and Local Tangent Space Alignment (LTSA) are applied independently to reduce the dimension of spectral data. principal component analysis (PCA), a linear dimensionality reduction method, is also utilized for spectral data dimensionality reduction. Finally, models for Partial Least Squares Regression (PLSR) and Support Vector Regression (SVR) are constructed. According to the model findings, LE-SVR model offers the best prediction accuracy and stability. The determination coefficient and root mean square error of the training and tests sets are 97.17%, 0.1875 and 96.25%, 0.2718 respectively. Furthermore, the relative analytical error is 5.0776. In addition, the study findings show that the number of neighbor points k has no discernible effect on the model performance. According to the findings, nonlinear modeling using LE-SVR for NIR spectral data of corn stover can lower model complexity, while improving model prediction accuracy and stability. NIR spectroscopy may be used to determine lignin content in corn straw. At the same time, the technique in this work offers a novel approach for the rapidly detecting lignin content in other crops straw.

Abnormal Status Detection of Catenary Based on TSNE Dimensionality Reduction Method and IGWO-LSSVM Model

Background: Catenary is a crucial component of an electrified railroad's traction power supply system. There is a considerable incidence of abnormal status and failures due to prolonged outside exposure. Driving safety will be directly impacted if an abnormal status or failure occurs. Currently, catenary detection vehicles are the most often utilized technique for gathering data and identifying faults based on manual experience. However, this technology cannot meet the demands of prompt detection and correction of faults in railways engineering due to its extremely low work efficiency. Objective: Based on the above, an abnormal status detection method of catenary based on the improved gray wolf (IGWO) algorithm optimized the least squares support vector machine (LSSVM) with the t-distributed stochastic neighbor embedding (TSNE) is proposed in this paper. In order to improve the accuracy of catenary abnormal status detection and shorten the detection time. Methods: Firstly, the TSNE dimensionality reduction technology is used to reduce the original catenary data to three-dimensional space. Then, in order to address the issue that the parameters of the LSSVM detection model are hard to determine, the improved GWO algorithm is used to optimize the penalty factor and kernel parameter in the LSSVM and establish the TSNE-IGWO-LSSVM catenary abnormal status detection model. Finally, contrasting experimental results of different detection models. The T-distributed Stochastic Domain Embedding (TSNE) is an improved nonlinear dimensionality reduction method based on the Stochastic Neighbor Embedding (SNE). TSNE no longer adopts the distance invariance in linear dimensionality reduction methods such as ISOMAP. TSNE is much better than the linear dimensionality reduction method in the reduction degree of the original dimension. The GWO algorithm, which is frequently used in engineering research, has the advantages of a simple model, great generalization capability, and good optimization performance. The premature convergence is one of the remaining flaws. By applying a good point set to initialize the gray wolf population and the nonlinear control parameters, the gray wolf algorithm is improved in this research. The IGWO algorithm effectively makes up for the problem of balancing the local exploitation and global search capabilities of GWO. Additionally, this IGWO algorithm performs the Cauchy variation operation on the current generation optimal solution to improve population diversity, enlarge the search space, and increase the likelihood of the algorithm escaping the local optimal solution in order to prevent the algorithm from failing the local optimum. The Least Squares Support Vector Machine (LSSVM) is an improved version of the Support Vector Machine (SVM), which replaces the original inequality constraint with a linear least squares criterion for the loss function. The kernel parameters of the RBF function and the penalty factor, these two parameters directly determine the detection effect of LSSVM. In this paper, the IGWO is utilized to adjust and determine the LSSVM parameters in order to enhance the detection capacity of the LSSVM model. Results:: In this paper, in order to minimize the experiment's bias, the training data and the test data are allocated in a ratio of 4:1, the training data are set to 400 groups, and the test data are set to 100 groups. After training the five models, the test data is used to validate and compare the detection capacity of the models. After each of the five detection models was tested ten times, the TSNE-IGWO-LSSVM model is compared with the IGWO-LSSVM model, the TSNE-FA-LSSVM model, the GWO-LSSVM model, and the GWO-ELM model, the results show that the TSNE-IGWO-LSSVM model has the highest average detection accuracy of 97.1% and the shortest running time of 26.9s. For the root mean squared error (RMSE) and the root mean squared error (RMSE), the TSNE-IGWO-LSSVM model is 0.17320 and 2.51% respectively, which is the best among the five models, indicating that it not only has higher detection accuracy but also better convergence of detection accuracy than the other models. Conclusions: With the thousands of miles of catenary and the complexity of the data, it is crucial to shorten the running time in order to improve the efficiency and ease the burden of the processors. The experiments demonstrate that the TSNE-IGWO-LSSVM detection model can detect the abnormal status of catenary more accurately and quickly, providing a new method for the abnormal status detection of catenary, which has certain application value and engineering significance in the era of fully electrified railways.

Local nonlinear dimensionality reduction via preserving the geometric structure of data

Dimensionality reduction has many applications in data visualization and machine learning. Existing methods can be classified into global ones and local ones. The global methods usually learn the linear relationship in data, while the local ones learn the manifold intrinsic geometry structure, which has a significant impact on pattern recognition. However, most of existing local methods obtain an embedding with eigenvalue or singular value decomposition, where the computational complexities are very high in a large amount of high-dimensional data. In this paper, we propose a local nonlinear dimensionality reduction method named Vec2vec, which employs a neural network with only one hidden layer to reduce the computational complexity. We first build a neighborhood similarity graph from the input matrix, and then define the context of data points with the random walk properties in the graph. Finally, we train the neural network with the context of data points to learn the embedding of the matrix. We conduct extensive experiments of data classification and clustering on nine image and text datasets to evaluate the performance of our method. Experimental results show that Vec2vec is better than several state-of-the-art dimensionality reduction methods, except that it is equivalent to UMAP on data clustering tasks in the statistical hypothesis tests, but Vec2vec needs less computational time than UMAP in high-dimensional data. Furthermore, we propose a more lightweight method named Approximate Vec2vec (AVec2vec) with little performance degradation, which employs an approximate method to build the neighborhood similarity graph. AVec2vec is still better than some state-of-the-art local dimensionality reduction methods and competitive with UMAP on data classification and clustering tasks in the statistical hypothesis tests.