Manifold Embedding Research Articles

Changes in cortical manifold structure following stroke and its relation to behavioral recovery in the male macaque

Stroke, a major cause of disability, disrupts brain function and motor skills. Previous research has mainly focused on reorganization of the motor system post-stroke, but the effects on other brain areas and their influence on recovery is poorly understood. Here, we use functional neuroimaging in a nonhuman primate model (23 male Cynomolgus Macaques), we explore how ischemic stroke affects whole-brain cortical architecture and its relation to spontaneous behavioral recovery. By projecting patterns of cortical functional connectivity onto a low-dimensional manifold space, we find that several regions in both sensorimotor cortex and higher-order transmodal cortex exhibit significant shifts in their manifold embedding from pre- to post-stroke. Furthermore, we observe that changes in default mode and limbic network regions, and not preserved sensorimotor cortical regions, are associated with animal behavioral recovery post-stroke. These results establish the whole-brain functional changes associated with stroke, and suggest an important role for higher-order transmodal cortex in post-stroke outcomes.

Comparative Analysis of Manifold Learning-Based Dimension Reduction Methods: A Mathematical Perspective

Manifold learning-based approaches have emerged as prominent techniques for dimensionality reduction. Among these methods, t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) stand out as two of the most widely used and effective approaches. While both methods share similar underlying procedures, empirical observations indicate two distinctive properties: global data structure preservation and computational efficiency. However, the underlying mathematical principles behind these distinctions remain elusive. To address this gap, this study presents a comparative analysis of the subprocesses involved in these methods, aiming to elucidate the mathematical mechanisms underlying the observed distinctions. By meticulously examining the equation formulations, the mathematical mechanisms contributing to global data structure preservation and computational efficiency are elucidated. To validate the theoretical analysis, data are collected through a laboratory experiment, and an open-source dataset is utilized for validation across different datasets. The consistent alignment of results obtained from both balanced and unbalanced datasets robustly confirms the study’s findings. The insights gained from this study provide a deeper understanding of the mathematical underpinnings of t-SNE and UMAP, enabling more informed and effective use of these dimensionality reduction techniques in various applications, such as anomaly detection, natural language processing, and bioinformatics.

Global-local manifold embedding broad graph convolutional network for hyperspectral image classification

Graph convolutional neural networks (GCNs) with domain-specific feature aggregation capabilities have unique advantages in hyperspectral image (HSI) classification. However, current GCN-based approaches frequently encounter the issue of node characteristics being over-smoothed while aggregating in higher-order domains. Furthermore, GCN linear classifiers focus solely on sample separability and ignore the potential manifold information of graph features, resulting in a failure to fully investigate extracted features. To address these problems, we propose a global-local manifold embedding broad graph convolutional network (GLMBG) for HSI classification. In GLMBG, we designed two modules from feature extraction and classification perspectives: The graph convolutional edge feature fusion extractor (GEFF) and the broad classifier of global-local manifold embedding (BGLME). GEFF is designed to learn graph node and local edge features from HSI through GCN and recursive filtering, combining them in a weighted manner to construct fused graph features. BGLME is designed to replace traditional linear classifiers with broad learning classifiers through manifold regularized embedding, fully utilizing the global and local manifold discriminant information of graph node features. The combination of GEFF and BGLME effectively reduces over-smoothing of graph node features while maximizing the utilization of manifold discriminant information, hence improving model feature discriminative ability. Experimental evaluations of three commonly used hyperspectral datasets show that our method surpasses state-of-the-art methods.

Improving the Two-Color Temperature Sensing Using Machine Learning Approach: GdVO4:Sm3+ Prepared by Solution Combustion Synthesis (SCS)

The gadolinium vanadate doped with samarium (GdVO4:Sm3+) nanopowder was prepared by the solution combustion synthesis (SCS) method. After synthesis, in order to achieve full crystallinity, the material was annealed in air atmosphere at 900 °C. Phase identification in the post-annealed powder samples was performed by X-ray diffraction, and morphology was investigated by high-resolution scanning electron microscope (SEM) and transmission electron microscope (TEM). Photoluminescence characterization of emission spectrum and time resolved analysis was performed using tunable laser optical parametric oscillator excitation and streak camera. In addition to samarium emission bands, a weak broad luminescence emission band of host VO43− was also observed by the detection system. In our earlier work, we analyzed the possibility of using the host luminescence for two-color temperature sensing, improving the method by introducing the temporal dependence in line intensity ratio measurements. Here, we showed that further improvements are possible by using the machine learning approach. To facilitate the initial data assessment, we incorporated Principal Component Analysis (PCA), t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) clustering of GdVO4:Sm3+ spectra at various temperatures. Good predictions of temperature were obtained using deep neural networks. Performance of the deep learning network was enhanced by data augmentation technique.

Manifold embedding stationary subspace analysis for nonstationary process monitoring with industrial applications

Industrial processes frequently exhibit nonstationary characteristics due to factors like load fluctuations and external interference. Accurate monitoring of nonstationary industrial processes is of vital importance in ensuring production stability and safety. Unfortunately, most existing monitoring methods overlook the manifold structure presented in nonstationary data due to nonstationary features, causing the loss of critical information and poor interpretability. As a consequence, monitoring performance is compromised. To address this issue, this paper proposes a manifold embedding stationary subspace analysis (MESSA) algorithm. By embedding a neighborhood preservation term into the objective function of SSA, MESSA effectively mitigates the impact of nonstationarity on manifold structure. The extracted features incorporate both global stationarity and local manifold characteristics, facilitating a more comprehensive reconstruction of the intricate underlying mechanisms in industrial processes. This contributes to a substantial enhancement in the accuracy and reliability of process monitoring. A set of nonstationary swiss-roll dataset is designed to visually demonstrate the capability of MESSA in extracting manifold structure. Case studies including a numerical case, a continuous stirred tank reactor system and a real industrial roasting process validate the superior monitoring performance of the proposed method.

Unsupervised manifold embedding to encode molecular quantum information for supervised learning of chemical data

Molecular representation is critical in chemical machine learning. It governs the complexity of model development and the fulfillment of training data to avoid either over- or under-fitting. As electronic structures and associated attributes are the root cause for molecular interactions and their manifested properties, we have sought to examine the local electron information on a molecular manifold to understand and predict molecular interactions. Our efforts led to the development of a lower-dimensional representation of a molecular manifold, Manifold Embedding of Molecular Surface (MEMS), to embody surface electronic quantities. By treating a molecular surface as a manifold and computing its embeddings, the embedded electronic attributes retain the chemical intuition of molecular interactions. MEMS can be further featurized as input for chemical learning. Our solubility prediction with MEMS demonstrated the feasibility of both shallow and deep learning by neural networks, suggesting that MEMS is expressive and robust against dimensionality reduction.

Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis.

Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.

Research on transfer learning diagnosis method for planetary gearbox bearings based on variational embedding composite multiscale diversity entropy and manifold dynamic distribution adaptation

To improve the generalisation ability of traditional machine learning fault diagnosis model, this paper proposed a planetary gearbox bearing diagnosis method based on the variational embedded composite multiscale diversity entropy (veCMED) and manifold dynamic distribution adaptation (MDDA). Firstly, fault features are extracted using variational embedded composite multiscale diversity entropy (veCMDE). Secondly, the extracted entropy fault features are combined with manifold dynamic distribution adaptation (MDDA) of the flow shape to map the entropy values of different distributions to the space of the same distribution. Thirdly, the entropy value after feature mapping is input into the KNN classifier for fault identification. The final experimental results show that the proposed veCMDE and MDDA based transfer learning method has the best generalisation ability compared to seven transfer learning methods.

Nonlinear dimensionality reduction with q-Gaussian distribution

In recent years, the dimensionality reduction has become more important as the number of dimensions of data used in various tasks such as regression and classification has increased. As popular nonlinear dimensionality reduction methods, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) have been proposed. However, the former outputs only one low-dimensional space determined by the t-distribution and the latter is difficult to control the distribution of distance between each pair of samples in low-dimensional space. To tackle these issues, we propose novel t-SNE and UMAP extended by q-Gaussian distribution, called q-Gaussian-distributed stochastic neighbor embedding (q-SNE) and q-Gaussian-distributed uniform manifold approximation and projection (q-UMAP). The q-Gaussian distribution is a probability distribution derived by maximizing the tsallis entropy by escort distribution with mean and variance, and a generalized version of Gaussian distribution with a hyperparameter q. Since the shape of the q-Gaussian distribution can be tuned smoothly by the hyperparameter q, q-SNE and q-UMAP can in- tuitively derive different embedding spaces. To show the quality of the proposed method, we compared the visualization of the low-dimensional embedding space and the classification accuracy by k-NN in the low-dimensional space. Empirical results on MNIST, COIL-20, OliverttiFaces and FashionMNIST demonstrate that the q-SNE and q-UMAP can derive better embedding spaces than t-SNE and UMAP.

Statistical method scDEED for detecting dubious 2D single-cell embeddings and optimizing t-SNE and UMAP hyperparameters

Two-dimensional (2D) embedding methods are crucial for single-cell data visualization. Popular methods such as t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) are commonly used for visualizing cell clusters; however, it is well known that t-SNE and UMAP’s 2D embeddings might not reliably inform the similarities among cell clusters. Motivated by this challenge, we present a statistical method, scDEED, for detecting dubious cell embeddings output by a 2D-embedding method. By calculating a reliability score for every cell embedding based on the similarity between the cell’s 2D-embedding neighbors and pre-embedding neighbors, scDEED identifies the cell embeddings with low reliability scores as dubious and those with high reliability scores as trustworthy. Moreover, by minimizing the number of dubious cell embeddings, scDEED provides intuitive guidance for optimizing the hyperparameters of an embedding method. We show the effectiveness of scDEED on multiple datasets for detecting dubious cell embeddings and optimizing the hyperparameters of t-SNE and UMAP.

Is Unsupervised Dimensionality Reduction Sufficient to Decode the Complexities of Electrochemical Impedance Spectra?

AbstractAs electrochemical research undergoes rapid technological progression, the acquisition of substantial amounts of electrochemical impedance spectra (EIS) becomes increasingly feasible. Yet, this advancement introduces intricate challenges in data processing, automation, and interpretation. This paper delves into the sufficiency of unsupervised machine learning (ML) and in particular dimensionality reduction methods in decoding EIS complexities, examining its strengths, limitations, and potential pathways for optimization. As we navigated the intricacies of non‐linear dimensionality reduction, spotlighting t‐distributed stochastic neighbor embedding (t‐SNE) and uniform manifold approximation and projection (UMAP) algorithms, a pattern emerged: these techniques excel at categorizing divergent impedance spectra but show limitations when faced with analogous circuit configurations, especially those substituting a capacitor with a constant phase element. This observation not only underscores a limitation but also accentuates that unsupervised ML approaches, alone, may not fully unravel the nuances of EIS spectra. In the concluding section of our manuscript, we discuss the implications of this finding from a practical standpoint, particularly for electrochemists seeking to apply these methods in their work.

Population encoding of stimulus features along the visual hierarchy

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to the mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

Deep carbonate reservoir characterization using multiseismic attributes: A comparison of unsupervised machine-learning approaches

Seismic reservoir characterization is of great interest for sweet spot identification, reservoir quality assessment, and geologic model building. The sparsity of the labeled samples often limits the application of supervised machine learning (ML) for seismic reservoir characterization. Unsupervised learning methods, in contrast, explore the internal structure of data and extract low-dimensional features of geologic interest from seismic data without the need for labels. We compare various unsupervised learning approaches, including the linear method of principal component analysis (PCA), the manifold learning methods of t-distributed stochastic neighbor embedding and uniform manifold approximation and projection (UMAP), and the convolutional autoencoder (CAE), on the 3D synthetic and field seismic data of a deep carbonate reservoir in southwest China. On the synthetic data, the low-dimensional features extracted by UMAP and CAE provide a better indication of porosity and gas saturation than traditional seismic attributes. In particular, UMAP better preserves the global structure of geologic features and indicates the potential of decoupling the gas saturation and porosity effects from seismic responses. We demonstrate that joint use of several types of seismic attributes, instead of using a single type of seismic attributes, can better delineate the reservoir structures using unsupervised ML. On the field seismic data, UMAP can effectively characterize the sedimentary facies distribution, which is consistent with the geologic understanding. Nevertheless, the porosity and saturation can not be reliably identified from field seismic data using unsupervised ML, which is likely caused by the complex pore structures in carbonates complicating the mapping relationship between seismic responses and reservoir parameters.

Robust Decoding of Upper-Limb Movement Direction Under Cognitive Distraction With Invariant Patterns in Embedding Manifold.

Motor brain-computer interfaces (BCIs) have gained growing research interest in motor rehabilitation, restoration, and prostheses control. Decoding upper-limb movement direction with noninvasive BCIs has been extensively investigated. However, few of them address the intervention of cognitive distraction that impairs decoding performance in practice. In this study, we propose a novel decoding model with invariant patterns in embedding manifold on a mixed dataset pooled from electroencephalograph (EEG) signals under different attentional states. We reconstruct an embedding low-dimensional manifold that intrinsically characterizes movements of the upper limb and transfer patterns of neural activities decomposed from brain functional connectivity (FC) to the manifold subspace to further preserve movement-related information. Experimental results showed that the proposed decoding model had higher robustness on the mixed dataset of attentive and distracted states compared to the baseline method. Our research provides insights into modeling a uniform underlying mechanism of movement-related EEG signals and can help enhance the practicability of BCI systems under real-world situations.

A novel methodology for emotion recognition through 62-lead EEG signals: multilevel heterogeneous recurrence analysis.

Recognizing emotions from electroencephalography (EEG) signals is a challenging task due to the complex, nonlinear, and nonstationary characteristics of brain activity. Traditional methods often fail to capture these subtle dynamics, while deep learning approaches lack explainability. In this research, we introduce a novel three-phase methodology integrating manifold embedding, multilevel heterogeneous recurrence analysis (MHRA), and ensemble learning to address these limitations in EEG-based emotion recognition. The proposed methodology was evaluated using the SJTU-SEED IV database. We first applied uniform manifold approximation and projection (UMAP) for manifold embedding of the 62-lead EEG signals into a lower-dimensional space. We then developed MHRA to characterize the complex recurrence dynamics of brain activity across multiple transition levels. Finally, we employed tree-based ensemble learning methods to classify four emotions (neutral, sad, fear, happy) based on the extracted MHRA features. Our approach achieved high performance, with an accuracy of 0.7885 and an AUC of 0.7552, outperforming existing methods on the same dataset. Additionally, our methodology provided the most consistent recognition performance across different emotions. Sensitivity analysis revealed specific MHRA metrics that were strongly associated with each emotion, offering valuable insights into the underlying neural dynamics. This study presents a novel framework for EEG-based emotion recognition that effectively captures the complex nonlinear and nonstationary dynamics of brain activity while maintaining explainability. The proposed methodology offers significant potential for advancing our understanding of emotional processing and developing more reliable emotion recognition systems with broad applications in healthcare and beyond.

K* Distribution: Evaluating the Latent Space of Deep Neural Networks Using Local Neighborhood Analysis.

Most examinations of neural networks' learned latent spaces typically employ dimensionality reduction techniques such as t-distributed stochastic neighbor embedding (t-SNE) or uniform manifold approximation and projection (UMAP). These methods distort the local neighborhood in the visualization, making it hard to distinguish the structure of a subset of samples in the latent space. In response to this challenge, we introduce the k* distribution and its corresponding visualization technique. This method uses local neighborhood analysis to guarantee the preservation of the structure of sample distributions for individual classes within the subset of the learned latent space. This facilitates easy comparison of different k* distributions, enabling analysis of how various classes are processed by the same neural network. Our study reveals three distinct distributions of samples within the learned latent space subset: 1) fractured; 2) overlapped; and 3) clustered, providing a more profound understanding of the existing contemporary visualizations. Experiments show that the distribution of samples within the network's learned latent space significantly varies depending on the class. Furthermore, we illustrate that our analysis can be applied to explore the latent space of diverse neural network architectures, various layers within neural networks, transformations applied to input samples, and the distribution of training and testing data for neural networks. Thus, the k* distribution should aid in visualizing the structure inside neural networks and further foster their understanding.

A Nonlinear Manifold Embedding Extreme Learning Machine to Improve the Gas Recognition of Electronic Noses

A Nonlinear Manifold Embedding Extreme Learning Machine to Improve the Gas Recognition of Electronic Noses

Image Matching via Ranking and Neighborhood Manifold Embedding

Image Matching via Ranking and Neighborhood Manifold Embedding

Reconstruction of manifold embeddings into Euclidean spaces via intrinsic distances

We consider the problem of reconstructing an embedding of a compact connected Riemannian manifold in a Euclidean space up to an almost isometry, given the information on intrinsic distances between points from its “sufficiently large” subset. This is one of the classical manifold learning problems. It happens that the most popular methods to deal with such a problem, with a long history in data science, namely, the classical Multidimensional scaling (MDS) and the Maximum variance unfolding (MVU), actually miss the point and may provide results very far from an isometry; moreover, they may even give no bi-Lipshitz embedding. We will provide an easy variational formulation of this problem, which leads to an algorithm always providing an almost isometric embedding with the distortion of original distances as small as desired (the parameter regulating the upper bound for the desired distortion is an input parameter of this algorithm).

ANALYSIS OF RELATIONSHIPS BETWEEN HETEROGENEOUS RADIOMIC FEATURES OF THROMBUS IN ACUTE ISCHEMIC STROKE

Ischemic stroke is one of the most widely recognised cerebral pathologies. Its diagnosis starts with computed tomography (CT). However, a comprehensive understanding of the characteristics of the thrombus is necessary to establish an appropriate and less hazardous treatment. This study focusses on the analysis of thrombus heterogeneity in CT images of patients with acute ischemic stroke (AIS). The radiomic features of the thrombus were used to obtain the voxel distribution in the new feature space. It was then reduced using Principal Component Analysis (PCA) and subjected to visualisation techniques such as t-distributed Stochastic Neighbour Embedding (tSNE) and Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP). By evaluating the morphological structure of the clusters created within patients, it was possible to determine the number of thrombus components. This information could help the physician predict the burden on the patient during thrombectomy, such as the number of attempts required for recanalization.