Low-dimensional Manifold Research Articles

Latent learning drives sleep-dependent plasticity in distinct CA1 subpopulations

Latent learning is a process that enables the brain to transform experiences into "cognitive maps," a form of implicit memory, without requiring reinforced training. To investigate its neural mechanisms, we record from hippocampal neurons in mice during latent learning of spatial maps and observe that the high-dimensional neural state space gradually transforms into a low-dimensional manifold that closely resembles the physical environment. This transformation process is associated with the neural reactivation of navigational experiences during sleep. Additionally, we identify a subset of hippocampal neurons that, rather than forming place fields in a novel environment, maintain weak spatial tuning but gradually develop correlated activity with other neurons. The elevated correlation introduces redundancy into the ensemble code, transforming the neural state space into a low-dimensional manifold that effectively links discrete place fields of place cells into a map-like structure. These results suggest a potential mechanism for latent learning of spatial maps in the hippocampus.

Bifurcation analysis in dynamical systems through integration of machine learning and dynamical systems theory

Abstract Characterizing the nonlinear behavior of dynamical systems near the stability boundary is a critical step toward understanding, designing, and controlling systems prone to stability concerns. Traditional methods for bifurcation analysis in both experimental systems and large-dimensional models are often hindered either by the absence of an accurate model or by the analytical complexity involved. This paper presents a novel approach that combines the theoretical frameworks of nonlinear reduced-order modeling and stability analysis in dynamical systems with advanced machine learning techniques to perform bifurcation analysis in dynamical systems. By focusing on a low-dimensional nonlinear invariant manifold, this work proposes a data-driven methodology that simplifies the process of bifurcation analysis in dynamical systems. The core of our approach lies in utilizing carefully designed neural networks to identify nonlinear transformations that map observation space into reduced manifold coordinates in its normal form where bifurcation analysis can be performed. The unique integration of analytical and data-driven approaches in the proposed method enables the learning of these transformations and the performance of bifurcation analysis with a limited number of trajectories. Therefore, this approach improves bifurcation analysis in model-less experimental systems and cost-sensitive high-fidelity simulations. The effectiveness of this approach is demonstrated across several examples.

High-fidelity experimental model verification for flow in fractured porous media

Mixed-dimensional mathematical models for flow in fractured media have been prevalent in the modeling community for almost two decades, utilizing the explicit representation of fractures by lower-dimensional manifolds embedded in the surrounding porous media. In this work, for the first time, direct qualitative and quantitative comparisons of mixed-dimensional models are drawn against laboratory experiments. Dedicated displacement experiments of steady-state laminar flow in fractured media are investigated using both high-resolution PET images as well as state-of-the-art numerical simulations.

Unsupervised representation learning of Kohn–Sham states and consequences for downstream predictions of many-body effects

Representation learning for the electronic structure problem is a major challenge of machine learning in computational condensed matter and materials physics. Within quantum mechanical first principles approaches, density functional theory (DFT) is the preeminent tool for understanding electronic structure, and the high-dimensional DFT wavefunctions serve as building blocks for downstream calculations of correlated many-body excitations and related physical observables. Here, we use variational autoencoders (VAE) for the unsupervised learning of DFT wavefunctions and show that these wavefunctions lie in a low-dimensional manifold within latent space. Our model autonomously determines the optimal representation of the electronic structure, avoiding limitations due to manual feature engineering. To demonstrate the utility of the latent space representation of the DFT wavefunction, we use it for the supervised training of neural networks (NN) for downstream prediction of quasiparticle bandstructures within the GW formalism. The GW prediction achieves a low error of 0.11 eV for a combined test set of two-dimensional metals and semiconductors, suggesting that the latent space representation captures key physical information from the original data. Finally, we explore the generative ability and interpretability of the VAE representation.

Fermat's principle in general relativity via Herglotz variational formalism

New form of Fermat's principle for light propagation in arbitrary (i.e. in general neither static nor stationary) gravitational field is proposed. It is based on Herglotz extension of canonical formalism and simple relation between the dynamics described by the Lagrangians homogeneous in velocities and the reduced dynamics on lower-dimensional configuration manifold. This approach is more flexible as it allows to extend immediately the Fermat principle to the case of massive particles and to eliminate any space-time coordinate, not only x0.

Diffusion model conditioning on Gaussian mixture model and negative Gaussian mixture gradient

Diffusion models (DMs) are a type of generative model that has had a significant impact on image synthesis and beyond. They can incorporate a wide variety of conditioning inputs — such as text or bounding boxes — to guide generation. In this work, we introduce a novel conditioning mechanism that applies Gaussian mixture models (GMMs) for feature conditioning, which helps steer the denoising process in DMs. Drawing on set theory, our comprehensive theoretical analysis reveals that the conditional latent distribution based on features differs markedly from that based on classes. Consequently, feature-based conditioning tends to generate fewer defects than class-based conditioning. Experiments are designed and carried out and the experimental results support our theoretical findings as well as effectiveness of proposed feature conditioning mechanism. Additionally, we propose a new gradient function named the Negative Gaussian Mixture Gradient (NGMG) and incorporate it into the training of diffusion models alongside an auxiliary classifier. We theoretically demonstrate that NGMG offers comparable advantages to the Wasserstein distance, serving as a more effective cost function when learning distributions supported by low-dimensional manifolds, especially in contrast to many likelihood-based cost functions, such as KL divergences.

An extended two-parameter mixed-dimensional model of fractured porous media incorporating entrance flow and boundary-layer transition effects

We develop an enhanced reduced model for single-phase flow in fractured porous media capable of incorporating more realistic interface conditions at the fracture terminations. In addition to the traditional dimensional model reduction, where the elements of the discrete fracture network are treated as lower dimensional manifolds embedded in the porous matrix, we explore the microscale behavior of the boundary layer flow at the entrances of a fracture bounded by two parallel plates to construct a new set of interface conditions of Robin-type, giving rise to localized pressure jumps at the fracture edges. Within this enriched description, sharper reduced flow and tracer transport mixed-dimensional models are constructed in the asymptotic limit ruled by two small parameters related to the ratio between fracture aperture and entrance developing length and a macroscopic length scale. The discrete flow/transport mixed-dimensional model is discretized by a new discontinuous Galerkin(dG)-based formulation. An adequate version of the Galerkin-Newton method is developed for the numerical treatment of the non-linear Robin interface condition. Considering several fracture arrangements, numerical results illustrate the sharper description of the model proposed herein in predicting flow and tracer transport patterns in fractured media.

Changes in cortical manifold structure following stroke and its relation to behavioral recovery in the male macaque

Stroke, a major cause of disability, disrupts brain function and motor skills. Previous research has mainly focused on reorganization of the motor system post-stroke, but the effects on other brain areas and their influence on recovery is poorly understood. Here, we use functional neuroimaging in a nonhuman primate model (23 male Cynomolgus Macaques), we explore how ischemic stroke affects whole-brain cortical architecture and its relation to spontaneous behavioral recovery. By projecting patterns of cortical functional connectivity onto a low-dimensional manifold space, we find that several regions in both sensorimotor cortex and higher-order transmodal cortex exhibit significant shifts in their manifold embedding from pre- to post-stroke. Furthermore, we observe that changes in default mode and limbic network regions, and not preserved sensorimotor cortical regions, are associated with animal behavioral recovery post-stroke. These results establish the whole-brain functional changes associated with stroke, and suggest an important role for higher-order transmodal cortex in post-stroke outcomes.

Generative learning for forecasting the dynamics of high-dimensional complex systems

We introduce generative models for accelerating simulations of high-dimensional systems through learning and evolving their effective dynamics. In the proposed Generative Learning of Effective Dynamics (G-LED), instances of high dimensional data are down sampled to a lower dimensional manifold that is evolved through an auto-regressive attention mechanism. In turn, Bayesian diffusion models, that map this low-dimensional manifold onto its corresponding high-dimensional space, operate on batches of physics correlated, time sequences of data and capture the statistics of the system dynamics. We demonstrate the capabilities and drawbacks of G-LED in simulations of several benchmark systems, including the Kuramoto-Sivashinsky (KS) equation, two-dimensional high Reynolds number flow over a backward-facing step, and simulations of three-dimensional turbulent channel flow. The results demonstrate that generative learning offers new frontiers for the accurate forecasting of the statistical properties of high-dimensional systems at a reduced computational cost.

Online landmark replacement for out-of-sample dimensionality reduction methods

A strategy to assist visualization and analysis of large and complex datasets is dimensionality reduction, with which one maps each data point into a low-dimensional manifold. However, various dimensionality reduction techniques are computationally infeasible for large data. Out-of-sample techniques aim to resolve this difficulty; they only apply the dimensionality reduction technique on a small portion of data, referred to as landmarks, and determine the embedding coordinates of the other points using landmarks as references. Out-of-sample techniques have been applied to online settings, or when data arrive as time series. However, existing online out-of-sample techniques use either all the previous data points as landmarks or the fixed set of landmarks and therefore are potentially not good at capturing the geometry of the entire dataset when the time series is non-stationary. To address this problem, we propose an online landmark replacement algorithm for out-of-sample techniques using geometric graphs and the minimal dominating set on them. We mathematically analyse some properties of the proposed algorithm, particularly focusing on the case of landmark multi-dimensional scaling as the out-of-sample technique, and test its performance on synthetic and empirical time-series data.

Identification method for micro–nano defects in Si3N4 ceramic bearing rollers

In response to the surface micro–nano defects of small Si3N4 ceramic bearing rollers during use, as well as the poor generalization ability and low accuracy of the object detection model trained on the micro and nano defect dataset, an enhanced recognition algorithm based on deep convolutional generative adversarial networks is proposed. Due to the limited size of the dataset, the DCGAN model is reconstructed to effectively expand the micro–nano defect dataset. In addition, noise generalization is applied to stabilize DCGAN model training, creating a low-dimensional manifold distribution to ensure significant overlap between the data and the original dataset, and activating Jensen–Shannon (JS) divergence for stable training. To verify the effectiveness of the enhanced dataset, synthetic micro–nano defects are used to improve the YOLO-v4-tiny object detection model. By comparing t-distributed stochastic neighbor embedding (t-SNE) and feature vectors, it can be found that the images generated by the optimized DCGAN have higher grayscale feature diversity and better visual consistency. After generations of enhancements, the micro–nano defect detection speed has reached 226FPS, and the accuracy has reached 97.41%.

Adaptive Learning of the Latent Space of Wasserstein Generative Adversarial Networks

Generative models based on latent variables, such as generative adversarial networks (GANs) and variational auto-encoders (VAEs), have gained lots of interests due to their impressive performance in many fields. However, many data such as natural images usually do not populate the ambient Euclidean space but instead reside in a lower-dimensional manifold. Thus an inappropriate choice of the latent dimension fails to uncover the structure of the data, possibly resulting in mismatch of latent representations and poor generative qualities. Towards addressing these problems, we propose a novel framework called the latent Wasserstein GAN (LWGAN) that fuses the Wasserstein auto-encoder and the Wasserstein GAN so that the intrinsic dimension of the data manifold can be adaptively learned by a modified informative latent distribution. We prove that there exist an encoder network and a generator network in such a way that the intrinsic dimension of the learned encoding distribution is equal to the dimension of the data manifold. We theoretically establish that our estimated intrinsic dimension is a consistent estimate of the true dimension of the data manifold. Meanwhile, we provide an upper bound on the generalization error of LWGAN, implying that we force the synthetic data distribution to be similar to the real data distribution from a population perspective. Comprehensive empirical experiments verify our framework and show that LWGAN is able to identify the correct intrinsic dimension under several scenarios, and simultaneously generate high-quality synthetic data by sampling from the learned latent distribution.

Actuation manifold from snapshot data

We propose a data-driven methodology to learn a low-dimensional manifold of controlled flows. The starting point is resolving snapshot flow data for a representative ensemble of actuations. Key enablers for the actuation manifold are isometric mapping as encoder, and a combination of a neural network and a $k$ -nearest-neighbour interpolation as decoder. This methodology is tested for the fluidic pinball, a cluster of three parallel cylinders perpendicular to the oncoming uniform flow. The centres of these cylinders are the vertices of an equilateral triangle pointing upstream. The flow is manipulated by constant rotation of the cylinders, i.e. described by three actuation parameters. The Reynolds number based on a cylinder diameter is chosen to be $30$ . The unforced flow yields statistically symmetric periodic shedding represented by a one-dimensional limit cycle. The proposed methodology yields a five-dimensional manifold describing a wide range of dynamics with small representation error. Interestingly, the manifold coordinates automatically unveil physically meaningful parameters. Two of them describe the downstream periodic vortex shedding. The other three describe the near-field actuation, i.e. the strength of boat-tailing, the Magnus effect and forward stagnation point. The manifold is shown to be a key enabler for control-oriented flow estimation.

On flat manifold bundles and the connectivity of Haefliger’s classifying spaces

We investigate a conjecture due to Haefliger and Thurston in the context of foliated manifold bundles. In this context, Haefliger-Thurston’s conjecture predicts that every M M -bundle over a manifold B B where dim ⁡ ( B ) ≤ dim ⁡ ( M ) \operatorname {dim}(B)\leq \operatorname {dim}(M) is cobordant to a flat M M -bundle. In particular, we study the bordism class of flat M M -bundles over low dimensional manifolds, comparing a finite dimensional Lie group G G with D i f f 0 ( G ) \mathrm {Diff}_0(G) .

Occipito-Ventral pathway dynamically transforms images into low-dimensional feature manifolds aligned on those supporting behavior

Occipito-Ventral pathway dynamically transforms images into low-dimensional feature manifolds aligned on those supporting behavior

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface.

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue. We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to. We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters. This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

G[formula omitted]BFNN: Generalized geodesic basis function neural network

Real-world data is typically distributed on low-dimensional manifolds embedded in high-dimensional Euclidean spaces. Accurately extracting spatial distribution features on general manifolds that reflect the intrinsic characteristics of data is crucial for effective feature representation. Therefore, we propose a generalized geodesic basis function neural network (G2BFNN) architecture. The generalized geodesic basis functions (G2BF) are defined based on generalized geodesic distances. The generalized geodesic distance metric (G2DM) is obtained by learning the manifold structure. To implement this architecture, a specific G2BFNN, named discriminative local preserving projection-based G2BFNN (DLPP-G2BFNN) is proposed. DLPP-G2BFNN mainly contains two modules, namely the manifold structure learning module (MSLM) and the network mapping module (NMM). In the MSLM module, a supervised adjacency graph matrix is constructed to constrain the learning of the manifold structure. This enables the learned features in the embedding subspace to maintain the manifold structure while enhancing the discriminability. The features and G2DM learned in the MSLM are fed into the NMM. Through the G2BF in the NMM, the spatial distribution features on manifold are obtained. Finally, the output of the network is obtained through the fully connected layer. Compared with the local response neural network based on Euclidean distance, the proposed network can reveal more essential spatial structure characteristics of the data. Meanwhile, the proposed G2BFNN is a generalized network architecture that can be combined with any manifold learning method, showcasing high scalability. The experimental results demonstrate that the proposed DLPP-G2BFNN outperforms existing methods by utilizing fewer kernels while achieving higher recognition performance.

Improved diffusion mapping combined with procrustes analysis for capturing local-global data structures in industrial process monitoring

BackgroundProcess monitoring, by providing early warnings of abnormal operating states resulting from process faults, facilitates the maintenance of normal production and ensures process safety. In the domain of industrial process monitoring, capturing the local-global structural features of data and acquiring an explicit mapping relationship for dimensionality reduction projection holds significant importance for online fault detection in industrial processes. MethodsThis study introduces an Improved Diffusion Mapping and Procrustes analysis (IDM-P) method for this purpose. Initially, considering the multiscale and correlation among industrial data features, the Mahalanobis distance is incorporated to improve the diffusion mapping algorithm. Utilizing this method allows for the concurrent capture of both local and global data structures, leading to a more efficient extraction of data-representative features, which enhances the accuracy of fault detection. Procrustes analysis is then used to obtain an explicit mapping matrix between high-dimensional data and low-dimensional manifolds, improving the efficiency of the key feature extraction of the new samples. Finally, this matrix is utilized to construct process monitoring statistics for fault detection. Significant FindingsThe method's effectiveness was validated through experiments on the TEP dataset and actual industrial data, demonstrating that IDM-P maintains higher accuracy and achieves optimal fault detection compared to other methods.

Cross-domain manifold structure preservation for transferable and cross-machine fault diagnosis

To address the decline or failure in the autonomous learning capability of traditional transfer learning methods when training and test samples come from different machines, resulting in low cross-machine fault diagnosis rates, we propose a cross-domain manifold structure preservation (CDMSP) method for diagnosing rolling bearing faults across machines. The CDMSP method can induce the manifold space projection matrices of the source and target domains more effectively. This method maps high-dimensional features into a low-dimensional manifold, preserving non-linear relationships and aligning distribution differences while maintaining cross-domain manifold structure consistency. Additionally, highly confidently labeled target domain samples are selected from each mapping result and added to the training dataset to enhance subspace learning in subsequent iterations. The CDMSP method is both simple and effective at capturing the underlying structures and patterns in the data. The CWRU dataset and our self-built test platform dataset were used to validate this method. Experimental results show that CDMSP, as a non-deep domain adaptation method of transfer learning, outperforms similar methods in cross-machine fault identification, achieving a maximum fault identification accuracy of 100 % with excellent convergence performance. Furthermore, simulated diagnostic experiments under noise interference indicate that CDMSP maintains high fault identification accuracy, even in noisy environments. Overall, CDMSP is an efficient and reliable new method for diagnosing cross-machine bearing faults.

Euclidean mirrors and dynamics in network time series

– Analyzing changes in network evolution is central to statistical network inference. We consider a dynamic network model in which each node has an associated time-varying low-dimensional latent vector of feature data, and connection probabilities are functions of these vectors. Under mild assumptions, the evolution of latent vectors exhibits low-dimensional manifold structure under a suitable distance. This distance can be approximated by a measure of separation between the observed networks themselves, and there exist Euclidean representations for underlying network structure, as characterized by this distance. These Euclidean representations, called Euclidean mirrors, permit the visualization of network dynamics and lead to methods for change point and anomaly detection in networks. We illustrate our methodology with real and synthetic data, and identify change points corresponding to massive shifts in pandemic policies in a communication network of a large organization.