Manifold Learning Algorithm Research Articles

Distortion corrected kernel density estimator on Riemannian manifolds

Manifold learning obtains a low-dimensional representation of an underlying Riemannian manifold supporting high-dimensional data. Kernel density estimates of the low-dimensional embedding with a fixed bandwidth fail to account for the way manifold learning algorithms distort the geometry of the Riemannian manifold. We propose a novel distortion-corrected kernel density estimator (DC-KDE) for any manifold learning embedding, with a bandwidth that depends on the estimated Riemannian metric at each data point. Exploiting the geometric information of the manifold leads to more accurate density estimation, which subsequently could be used for anomaly detection. To compare our proposed estimator with a fixed-bandwidth kernel density estimator, we run two simulations including one with data lying in a 100 dimensional ambient space. We demonstrate that the proposed DC-KDE improves the density estimates as long as the manifold learning embedding is of sufficient quality, and has higher rank correlations with the true manifold density. Further simulation results are provided via a supplementary R shiny app. The proposed method is applied to density estimation in statistical manifolds of electricity usage with the Irish smart meter data.

Linearized Wasserstein dimensionality reduction with approximation guarantees

We introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as probability measures rather than points in Rn, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available algorithms are based on computing the pairwise Wasserstein distance matrix, which can be computationally challenging for large datasets in high dimensions. Our algorithm leverages approximation schemes such as Sinkhorn distances and linearized optimal transport to speed-up computations, and in particular, avoids computing a pairwise distance matrix. We provide guarantees on the embedding quality under such approximations, including when explicit descriptions of the probability measures are not available and one must deal with finite samples instead. Experiments demonstrate that LOT Wassmap attains correct embeddings and that the quality improves with increased sample size. We also show how LOT Wassmap significantly reduces the computational cost when compared to algorithms that depend on pairwise distance computations.

Variability of morphology in photoplethysmographic waveform quantified with unsupervised wave-shape manifold learning for clinical assessment

Objective.We investigated fluctuations of the photoplethysmography (PPG) waveform in patients undergoing surgery. There is an association between the morphologic variation extracted from arterial blood pressure (ABP) signals and short-term surgical outcomes. The underlying physiology could be the numerous regulatory mechanisms on the cardiovascular system. We hypothesized that similar information might exist in PPG waveform. However, due to the principles of light absorption, the noninvasive PPG signals are more susceptible to artifacts and necessitate meticulous signal processing.Approach.Employing the unsupervised manifold learning algorithm, dynamic diffusion map, we quantified multivariate waveform morphological variations from the PPG continuous waveform signal. Additionally, we developed several data analysis techniques to mitigate PPG signal artifacts to enhance performance and subsequently validated them using real-life clinical database.Main results.Our findings show similar associations between PPG waveform during surgery and short-term surgical outcomes, consistent with the observations from ABP waveform analysis.Significance.The variation of morphology information in the PPG waveform signal in major surgery provides clinical meanings, which may offer new opportunity of PPG waveform in a wider range of biomedical applications, due to its non-invasive nature.

Manifold learning localization based on local tangent space alignment for wireless sensor networks

At present, nodes localization in wireless sensor networks are one of the focuses of many scholars' research. But now the current localization algorithms usually face the problems of low localization accuracy and high computation. Since the information captured by wireless sensor networks is essentially nonlinear, thus this paper proposes a localization algorithm based on manifold learning algorithm local tangent space alignment for wireless sensor networks. It can process nonlinear data to achieve good positioning effect, and the variables in the algorithm have little impact on the localization effect. Two types of positioning data are considered for localization: the measurement distance and the signal strength between sensor nodes. Firstly, the local k-neighborhoods of the localized data points can be found for determining the low-dimensional geometric structure of their local tangent spaces by principal component analysis. Next, the local tangent space of the localized sample points is aligned to obtain the relative locations of the sensor nodes. Then, the physical locations of the sensor nodes can be get by lining up the relative positions with anchor points. Finally, the ending of the paper by doing simulation experiments demonstrate that the algorithm has good precision and few variables in performing nodes localization.

Quantum Isomap algorithm for manifold learning

Quantum Isomap algorithm for manifold learning

Machine learning for the identification of phase transitions in interacting agent-based systems: A Desai-Zwanzig example.

Deriving closed-form analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM-the Desai-Zwanzig model-in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and we show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates that facilitates, in our example, the identification of a single parameter-dependent ordinary differential equation(ODE) in these coordinates. We identify this ODE through a residual neural network inspired by a numerical integration scheme (forward Euler). We then use the identified ODE-enabled through an odd symmetry transformation-to construct the bifurcation diagram exhibiting the phase transition.

Basic research for identification and classification of organophosphorus pesticides in water based on ultraviolet-visible spectroscopy information.

In this study, the goal was to develop a method for detecting and classifying organophosphorus pesticides (OPPs) in bodies of water. Sixty-five samples with different concentrations were prepared for each of the organophosphorus pesticides, namely chlorpyrifos, acephate, parathion-methyl, trichlorphon, dichlorvos, profenofos, malathion, dimethoate, fenthion, and phoxim, respectively. Firstly, the spectral data of all the samples was obtained using a UV-visible spectrometer. Secondly, five preprocessing methods, six manifold learning methods, and five machine learning algorithms were utilized to build detection models for identifying OPPs in water bodies. The findings indicate that the accuracy of machine learning models trained on data preprocessed using convolutional smoothing + first-order derivatives (SG + FD) outperforms that of models trained on data preprocessed using other methods. The backpropagation neural network (BPNN) model exhibited the highest accuracy rate at 99.95%, followed by the support vector machine (SVM) and convolutional neural network (CNN) models, both at 99.92%. The extreme learning machine (ELM) and K-nearest neighbors (KNN) models demonstrated accuracy rates of 99.84% and 99.81%, respectively. Following the application of a manifold learning algorithm to the full-wavelength data set for the purpose of dimensionality reduction, the data was then visualized in the first three dimensions. The results demonstrate that the t-distributed domain embedding (t-SNE) algorithm is superior, exhibiting dense clustering of similar clusters and clear classification of dissimilar ones. SG + FD-t-SNE-SVM ranks highest among the feature extraction models in terms of performance. The feature extraction dimension was set to 4, and the average classification accuracy was 99.98%, which slightly improved the prediction performance over the full-wavelength model. As shown in this study, the ultraviolet-visible (UV-visible) spectroscopy system combined with the t-SNE and SVM algorithms can effectively identify and classify OPPs in waterbodies.

A Bearing Signal Adaptive Denoising Technique Based on Manifold Learning and Genetic Algorithm

A Bearing Signal Adaptive Denoising Technique Based on Manifold Learning and Genetic Algorithm

Probabilistic Nearest Neighbors Based Locality Preserving Projections for Unsupervised Metric Learning

Dimensionality reduction based unsupervised metric learning consists in finding meaningful compact data representations previously to clustering and classification problems. One of the major aspects of these algorithms is the approximation of the underlying manifold by a weighted graph. A limitation with most manifold learning algorithms is that edge weights in the proximity graph rely heavily on the Euclidean distance, which is known to be quite sensitive to the presence of outliers. In this paper, we propose to improve the Locality Preserving Projections (LPP) algorithm by incorporating a recently proposed graph inference method called Probabilistic Nearest Neighbors (PNN), an extension of the Clustering with Adaptive Neighbors (CAN) approach, used with success in graph-based semi-supervised learning. The proposed PNN-LPP algorithm is able to achieve better classification results than regular LPP, showing competitive performance against state-of-the-art approaches for dimensionality reduction, such as the UMAP algorithm, especially in datasets with a limited number of samples.

Multi-view data visualisation via manifold learning

Non-linear dimensionality reduction can be performed by manifold learning approaches, such as stochastic neighbour embedding (SNE), locally linear embedding (LLE) and isometric feature mapping (ISOMAP). These methods aim to produce two or three latent embeddings, primarily to visualise the data in intelligible representations. This manuscript proposes extensions of Student’s t-distributed SNE (t-SNE), LLE and ISOMAP, for dimensionality reduction and visualisation of multi-view data. Multi-view data refers to multiple types of data generated from the same samples. The proposed multi-view approaches provide more comprehensible projections of the samples compared to the ones obtained by visualising each data-view separately. Commonly, visualisation is used for identifying underlying patterns within the samples. By incorporating the obtained low-dimensional embeddings from the multi-view manifold approaches into the K-means clustering algorithm, it is shown that clusters of the samples are accurately identified. Through extensive comparisons of novel and existing multi-view manifold learning algorithms on real and synthetic data, the proposed multi-view extension of t-SNE, named multi-SNE, is found to have the best performance, quantified both qualitatively and quantitatively by assessing the clusterings obtained. The applicability of multi-SNE is illustrated by its implementation in the newly developed and challenging multi-omics single-cell data. The aim is to visualise and identify cell heterogeneity and cell types in biological tissues relevant to health and disease. In this application, multi-SNE provides an improved performance over single-view manifold learning approaches and a promising solution for unified clustering of multi-omics single-cell data.

Motor fault diagnosis based on composite multi-scale weighted reverse slope entropy and neighborhood preserving embedding

The motors are critical components of the electromechanical transmission in vehicles, and its operating status directly affects the maneuverability of vehicles. To quickly and accurately identify the operating status of motors, this paper proposes a new entropy - Composite Multi-scale Weighted Reverse Slope Entropy (CMWRSlE) for motor fault diagnosis, which is a more interpretable entropy due to its deep exploration of signals. Firstly, the composite multi-scale weighted reverse slope entropy values are extracted from the vibration signals of the motor in different states; Secondly, the extracted features are dimensionally reduced by the manifold learning algorithm – Neighbourhood Preserving Embedding (NPE) and classified by the hierarchical prototype-based approach (HPA) to achieve the fault diagnosis of the motor. Finally, the method proposed in this paper is validated through two sets of experimental data: motor rotor faults and motor bearing faults. The results show that the accuracy of the proposed method in motor fault diagnosis reaches 100 %, which indicates the effectiveness of the proposed method.

A Web Data Mining Algorithm Based on Manifold Distance for Mixed Data in Cloud Service Architecture

Due to the complex distribution of web data and frequent updates under the cloud service architecture, the existing methods for global consistency of data ignore the global consistency of distance measurement and the inability to obtain neighborhood information of data. To overcome these problems, we transform the multi-information goal and multi-user demand (constraint conditions) in web data mining into a constrained multi-objective optimization model and solve it by a constrained particle swarm multi-objective optimization algorithm. While we measure the distance between data by manifold distance. In order to make it easier for the constrained multi-objective particle swarm algorithm to solve different types of problems to find an effective solution set closer to the real Pareto front, a new manifold learning algorithm based on the constrained multi-objective particle swarm algorithm is built and used to solve this problem. Experiments results demonstrate that this can improve the service efficiency of cloud computing.

Weakly supervised classification through manifold learning and rank-based contextual measures

Over the last decade, significant advances have been achieved by machine learning approaches, notably in supervised learning scenarios. Supported by the advent of deep learning and comprehensive training sets, the accuracy achieved on classification tasks has improved significantly. Simultaneously, we have experienced massive growth in multimedia data and applications, which have become ubiquitous in several domains. However, with the increase in multimedia data collections, significant bottlenecks associated with the lack of labeled data emerged. To surpass this critical issue, developing methods capable of exploiting the unlabeled data and operating under weak supervision has become imperative. This work proposes a rank-based model capable of using contextual information encoded in the unlabeled data to perform weakly supervised classification. We evaluated the proposed weakly supervised approach on multimedia classification tasks with and without manifold learning algorithms, considering several combinations of rank correlation measures and classifiers. An experimental evaluation was conducted on 6 public image datasets considering different features, including convolutional neural networks and visual transformers. Positive gains were achieved compared to supervised and semi-supervised baselines for the same amount of labeled data. For instance, the proposed approach with manifold learning enhanced the accuracy of the Optimum-Path Forest (OPF) classifier from 71.77% to 83.24% when applied to the Flowers dataset and ResNet features. Among the conclusions, this work reveals that rank-based correlation measures and manifold learning can be used for a more effective labeled set expansion.

Datascape: exploring heterogeneous dataspace

Data science is a powerful field for gaining insights, comparing, and predicting behaviors from datasets. However, the diversity of methods and hypotheses needed to abstract a dataset exhibits a lack of genericity. Moreover, the shape of a dataset, which structures its contained information and uncertainties, is rarely considered. Inspired by state-of-the-art manifold learning and hull estimations algorithms, we propose a novel framework, the datascape, that leverages topology and graph theory to abstract heterogeneous datasets. Built upon the combination of a nearest neighbor graph, a set of convex hulls, and a metric distance that respects the shape of the data, the datascape allows exploration of the dataset’s underlying space. We show that the datascape can uncover underlying functions from simulated datasets, build predictive algorithms with performance close to state-of-the-art algorithms, and reveal insightful geodesic paths between points. It demonstrates versatility through ecological, medical, and simulated data use cases.

Contrastive Learning Dimensionality Reduction Method Based on Manifold Learning

With the development of the times, more and more high-dimensional datasets come into people's view. In order to reduce the time complexity and space complexity of downstream tasks, data dimensionality reduction becomes the primary choice. Classical dimensionality reduction algorithms are mainly divided into linear dimensionality reduction algorithms and nonlinear dimensionality reduction algorithms. Some of the traditional dimensionality reduction methods have the problems of not considering the nonlinear structure of the original dataset and the existence of weak generalisation, which makes the dimensionality reduction effect not good or model need to be recalculated because of the addition of new samples. In order to solve these problems, the research in this paper is a comparative learning dimensionality reduction method based on manifold learning. The idea of manifold learning using geodesic distance can fully consider the nonlinear structure of the original dataset. In this paper, comparative learning is the main framework. When the neural network completes the training, it only need to take the new data as input to calculate, the result can be obtained, no need to reconstruct the model which means the generality is high. Starting from the related work, this paper briefly introduces manifold learning, comparative learning and neural network algorithms. Subsequently, an innovative model is proposed, including three modules, Isomap to extract nonlinear structure, expanding neighbourhood to make pseudo-labels, and comparative learning training. Detailed analyses are carried out through experiments, comparing with PCA and LLE algorithms with the geodetic distance retention rate as an indicator, which proves that the data dimensionality reduction method of this model is more effective and ubiquitous.

WERECE: An Unsupervised Method for Educational Concept Extraction Based on Word Embedding Refinement

The era of educational big data has sparked growing interest in extracting and organizing educational concepts from massive amounts of information. Outcomes are of the utmost importance for artificial intelligence–empowered teaching and learning. Unsupervised educational concept extraction methods based on pre-trained models continue to proliferate due to ongoing advances in semantic representation. However, it remains challenging to directly apply pre-trained large language models to extract educational concepts; pre-trained models are built on extensive corpora and do not necessarily cover all subject-specific concepts. To address this gap, we propose a novel unsupervised method for educational concept extraction based on word embedding refinement (i.e., word embedding refinement–based educational concept extraction (WERECE)). It integrates a manifold learning algorithm to adapt a pre-trained model for extracting educational concepts while accounting for the geometric information in semantic computation. We further devise a discriminant function based on semantic clustering and Box–Cox transformation to enhance WERECE’s accuracy and reliability. We evaluate its performance on two newly constructed datasets, EDU-DT and EDUTECH-DT. Experimental results show that WERECE achieves an average precision up to 85.9%, recall up to 87.0%, and F1 scores up to 86.4%, which significantly outperforms baselines (TextRank, term frequency–inverse document frequency, isolation forest, K-means, and one-class support vector machine) on educational concept extraction. Notably, when WERECE is implemented with different parameter settings, its precision and recall sensitivity remain robust. WERECE also holds broad application prospects as a foundational technology, such as for building discipline-oriented knowledge graphs, enhancing learning assessment and feedback, predicting learning interests, and recommending learning resources.

Manifold Explorer: Satellite Image Labelling and Clustering Tool with Using Deep Convolutional Autoencoders

We present a novel approach to providing greater insight into the characteristics of an unlabelled dataset, increasing the efficiency with which labelled datasets can be created. We leverage dimension-reduction techniques in combination with autoencoders to create an efficient feature representation for image tiles derived from remote sensing satellite imagery. The proposed methodology consists of two main stages. Firstly, an autoencoder network is utilised to reduce the high-dimensional image tile data into a compact and expressive latentfeature representation. Subsequently, features are further reduced to a two-dimensional embedding space using the manifold learning algorithm Uniform Manifold Approximation and Projection (UMAP) and t-distributed Stochastic Neighbour Embedding (t-SNE). This step enables the visualization of the image tile clusters in a 2D plot, providing an intuitive and interactive representation that can be used to aid rapid and geographically distributed image labelling. To facilitate the labelling process, our approach allows users to interact with the 2D visualization and label clusters based on their domain knowledge. In cases where certain classes are not effectively separated, users can re-apply dimension reduction to interactively refine subsets of clusters and achieve better class separation, enabling a comprehensively labelled dataset. We evaluate the proposed approach on real-world remote sensing satellite image datasets and demonstrate its effectiveness in achieving accurate and efficient image tile clustering and labelling. Users actively participate in the labelling process through our interactive approach, leading to enhanced relevance of the labelled data, by allowing domain experts to contribute their expertise and enrich the dataset for improved downstream analysis and applications.

Robust discriminant latent variable manifold learning for rotating machinery fault diagnosis

Fault diagnosis is an important technology for performing intelligent manufacturing. To simultaneously maintain high manufacturing quality and low failure rate for manufacturing systems, it is of great value to accurately locate the fault element, evaluate the fault severity and find the fault root cause. In order to effectively perform feature extraction for rotating machinery fault diagnosis, a robust discriminant latent variable manifold learning (RDLVML) algorithm is proposed in this paper. Specifically, RDLVML algorithm could effectively select the features of the high-dimensional fault data, and extract the low-dimensional fault features with better discrimination. Through the idea of discriminative learning, the accuracy of fault diagnosis was improved by increasing the aggregation of the same state feature samples and the difference between different classes of feature samples. A novel weighted neighborhood graph is proposed by constructing the q-Rényi kernel function, which could effectively suppress the interference of outliers and noise, and make the RDLVML algorithm more robust. Moreover, the fault diagnosis method of rotating machinery based on RDLVML is proposed in this paper. Through experimental verifications and comparisons with several classical feature selection algorithms, rotating machinery fault diagnosis could be more accurately performed by this method.

Geometry Regularized Autoencoders.

A fundamental task in data exploration is to extract low dimensional representations that capture intrinsic geometry in data, especially for faithfully visualizing data in two or three dimensions. Common approaches use kernel methods for manifold learning. However, these methods typically only provide an embedding of the input data and cannot extend naturally to new data points. Autoencoders have also become popular for representation learning. While they naturally compute feature extractors that are extendable to new data and invertible (i.e., reconstructing original features from latent representation), they often fail at representing the intrinsic data geometry compared to kernel-based manifold learning. We present a new method for integrating both approaches by incorporating a geometric regularization term in the bottleneck of the autoencoder. This regularization encourages the learned latent representation to follow the intrinsic data geometry, similar to manifold learning algorithms, while still enabling faithful extension to new data and preserving invertibility. We compare our approach to autoencoder models for manifold learning to provide qualitative and quantitative evidence of our advantages in preserving intrinsic structure, out of sample extension, and reconstruction. Our method is easily implemented for big-data applications, whereas other methods are limited in this regard.

Lithium Battery SOH Estimation Based on Manifold Learning and LightGBM

In order to accurately identify the state of health (SOH) and remaining useful life (RUL) of lithium-ion batteries, this paper proposes an SOH estimation algorithm for lithium-ion batteries based on stream learning and LightGBM. To address the problem of inconsistent data length, which makes it difficult to establish the state mapping relationship between degraded data and health state, the health factors in this paper are extracted from capacity degradation features, entropy features, and correlation coefficient features. Then, the landmark isometric mapping (L-ISOMAP) manifold learning algorithm is used to dimensionally reduce the input feature set and map the high-dimensional features to the low-dimensional space to solve the dimensional explosion problem. Finally, a LightGBM prediction model is developed to perform SOH prediction on different datasets, and the superiority of the multidimensional model is evaluated. The experimental results show that the goodness-of-fit is 0.98 and above, and the MSE values are below 4 × 10−4. Comparing several prediction models, the LightGBM model has the best performance and better results in several indexes, such as MSE and RMSE. Under different working conditions, the proposed model in this paper has a goodness-of-fit of more than 0.98 in dataset B, which proves that the proposed model has a strong generalization ability.