Low-dimensional Manifold Research Articles

Domain Adaptation as Optimal Transport on Grassmann Manifolds.

Domain adaptation in the Euclidean space is a challenging task on which researchers recently have made great progress. However, in practice, there are rich data representations that are not Euclidean. For example, many high-dimensional data in computer vision are in general modeled by a low-dimensional manifold. This prompts the demand of exploring domain adaptation between non-Euclidean manifold spaces. This article is concerned with domain adaption over the classic Grassmann manifolds. An optimal transport-based domain adaptation model on Grassmann manifolds has been proposed. The model implements the adaption between datasets by minimizing the Wasserstein distances between the projected source data and the target data on Grassmann manifolds. Four regularization terms are introduced to keep task-related consistency in the adaptation process. Furthermore, to reduce the computational cost, a simplified model preserving the necessary adaption property and its efficient algorithm is proposed and tested. The experiments on several publicly available datasets prove the proposed model outperforms several relevant baseline domain adaptation methods.

A manifold neural population code for space in hippocampal coactivity dynamics independent of place fields

Hippocampus place cell discharge is temporally unreliable across seconds and days, and place fields are multimodal, suggesting an "ensemble cofiring" spatial coding hypothesis with manifold dynamics that does not require reliable spatial tuning, in contrast to hypotheses based on place field (spatial tuning) stability. We imaged mouse CA1 (cornu ammonis 1) ensembles in two environments across three weeks to evaluate these coding hypotheses. While place fields "remap," being more distinct between than within environments, coactivity relationships generally change less. Decoding location and environment from 1-s ensemble location-specific activity is effective and improves with experience. Decoding environment from cell-pair coactivity relationships is also effective and improves with experience, even after removing place tuning. Discriminating environments from 1-s ensemble coactivity relies crucially on the cells with the most anti-coactive cell-pair relationships because activity is internally organized on a low-dimensional manifold of non-linear coactivity relationships that intermittently reregisters to environments according to the anti-cofiring subpopulation activity.

Lagrangian configurations and Hamiltonian maps

We study configurations of disjoint Lagrangian submanifolds in certain low-dimensional symplectic manifolds from the perspective of the geometry of Hamiltonian maps. We detect infinite-dimensional flats in the Hamiltonian group of the two-sphere equipped with Hofer's metric, prove constraints on Lagrangian packing, find instances of Lagrangian Poincaré recurrence, and present a new hierarchy of normal subgroups of area-preserving homeomorphisms of the two-sphere. The technology involves Lagrangian spectral invariants with Hamiltonian term in symmetric product orbifolds.

Dynamic functional connectivity MEG features of Alzheimer’s disease

Dynamic resting state functional connectivity (RSFC) characterizes time-varying fluctuations of functional brain network activity. While many studies have investigated static functional connectivity, it has been unclear whether features of dynamic functional connectivity are associated with neurodegenerative diseases. Popular sliding-window and clustering methods for extracting dynamic RSFC have various limitations that prevent extracting reliable features to address this question. Here, we use a novel and robust time-varying dynamic network (TVDN) approach to extract the dynamic RSFC features from high resolution magnetoencephalography (MEG) data of participants with Alzheimer’s disease (AD) and matched controls. The TVDN algorithm automatically and adaptively learns the low-dimensional spatiotemporal manifold of dynamic RSFC and detects dynamic state transitions in data. We show that amongst all the functional features we investigated, the dynamic manifold features are the most predictive of AD. These include: the temporal complexity of the brain network, given by the number of state transitions and their dwell times, and the spatial complexity of the brain network, given by the number of eigenmodes. These dynamic features have higher sensitivity and specificity in distinguishing AD from healthy subjects than the existing benchmarks do. Intriguingly, we found that AD patients generally have higher spatial complexity but lower temporal complexity compared with healthy controls. We also show that graph theoretic metrics of dynamic component of TVDN are significantly different in AD versus controls, while static graph metrics are not statistically different. These results indicate that dynamic RSFC features are impacted in neurodegenerative disease like Alzheimer’s disease, and may be crucial to understanding the pathophysiological trajectory of these diseases.

Methodology for estimation of intrinsic dimensions and state variables of microstructures.

According to the manifold hypothesis, real data can be compressed to lie on a low-dimensional manifold. This paper explores the estimation of the dimensionality of this manifold with an interest in identifying independent degrees of freedom and possibly identifying state variables that would govern materials systems. The challenges identified that are specific to materials science are (i) accurate estimation of the number of dimensions of the data, (ii) coping with the intrinsic random and low-bit-depth nature of microstructure samples, and (iii) linking noncompressed domains such as processing to microstructure. Dimensionality estimates are made with the maximum-likelihood-estimation method with the Minkowski p-norms being used as a measure of the distance between microstructural images. It is found that, where dimensionality estimates are required to be accurate, it is necessary to use the Minkowski 1-norm (also known as the L_{1}-norm or Manhattan distance). This effect is found to be due to image quantification and proofs are given regarding the distortion produced by quantization. It is also found that homogenization is an effective way of estimating the dimension of random microstructure image sets. An estimate of 40 dimensions for the fibers of a SiC/SiC fiber composite is obtained. It is also found that, with images generated from a sparse domain (surrogate to the process domain), it is possible to infer the nature of the process manifold from images alone.

Extraction and analysis of flow features in planar synthetic jets using different machine learning techniques

Synthetic jets are useful fluid devices with several industrial applications. In this study, we use the flow fields generated by two synchronously operating synthetic jets and simulated using direct numerical simulations. These flow fields are characterized by a jet Reynolds number, Re=100, 150, and 200, and a Strouhal number, St=0.03. We benchmark four different dimensionality reduction techniques: (1) higher-order dynamic mode decomposition (HODMD), (2) proper orthogonal decomposition, (3) vector quantization via principal component analysis (VQPCA), and (4) linear autoencoders. These techniques are often used in generating reduced-order models (ROMs). The performances of these techniques are compared (i) in terms of their ability to accurately reconstruct the high-dimensional flow fields from their low-dimensional manifolds and (ii) in terms of their ability to extract meaningful low-dimensional patterns/features/structures that best describe the main dynamics of the synthetic jets. The similarity between the extracted features is also quantitatively assessed with the help of Procrustes analysis, showing how manifolds from different techniques become more similar when a larger number of modes are retained. Accurate reconstruction and model complexity (or interpretability) are often two counter-balancing objectives. In this comparative study, we found that among the four techniques, VQPCA has clear advantages for developing accurate ROMs, while HODMD is useful for understanding the dynamics of synthetic jets, providing additional information that is not readily available with other methods.

Equivariant Bismut Laplacian and spectral Einstein functional

This paper aims to provide an explicit computation of the equivariant noncommutative residue density of which yield the metric and Einstein tensors on even-dimensional Riemannian manifolds. A considerable contribution of this paper is the development of the spectral Einstein functionals by two vector fields and the equivariant Bismut Laplacian over spinor bundles. We prove the equivariant Dabrowski–Sitarz–Zalecki-type theorems for lower dimensional spin manifolds with (or without) boundary.

Using adversarial networks to extend brain computer interface decoding accuracy over time.

Existing intracortical brain computer interfaces (iBCIs) transform neural activity into control signals capable of restoring movement to persons with paralysis. However, the accuracy of the 'decoder' at the heart of the iBCI typically degrades over time due to turnover of recorded neurons. To compensate, decoders can be recalibrated, but this requires the user to spend extra time and effort to provide the necessary data, then learn the new dynamics. As the recorded neurons change, one can think of the underlying movement intent signal being expressed in changing coordinates. If a mapping can be computed between the different coordinate systems, it may be possible to stabilize the original decoder's mapping from brain to behavior without recalibration. We previously proposed a method based on Generalized Adversarial Networks (GANs), called 'Adversarial Domain Adaptation Network' (ADAN), which aligns the distributions of latent signals within underlying low-dimensional neural manifolds. However, we tested ADAN on only a very limited dataset. Here we propose a method based on Cycle-Consistent Adversarial Networks (Cycle-GAN), which aligns the distributions of the full-dimensional neural recordings. We tested both Cycle-GAN and ADAN on data from multiple monkeys and behaviors and compared them to a third, quite different method based on Procrustes alignment of axes provided by Factor Analysis. All three methods are unsupervised and require little data, making them practical in real life. Overall, Cycle-GAN had the best performance and was easier to train and more robust than ADAN, making it ideal for stabilizing iBCI systems over time.

TwP: Two-stage projection framework with manifold constraint for image reconstruction

As an extensively studied task in computer vision, image reconstruction aims to reconstruct original data from acquired measurements. It is a highly ill-posedness inverse problem since a countably infinite number of feasible solutions exist. To achieve high-quality reconstruction performance, we propose a two-stage projection (TwP) framework built upon the projection technique and manifold constraint. We first exploit the generalized non-convex alternating projection method for image reconstruction in the pre-reconstruction stage, that incorporates the image manifold constraint implicitly imposed by deep denoiser. It provides a unified method to solve linear/non-linear inverse problems in a plug-and-play manner and deliver a good estimation. In the post-processing stage, a feed-forward network fine-tuned for a specific task is utilized to reproject the pre-reconstruction results onto the low-dimensional image manifold, which enables the outcomes to be in or closest to the manifold. Extensive experiments across various image reconstruction tasks, such as single-photon imaging, image compressive sensing, and image deblurring, demonstrate that the proposed TwP delivers promising results against some existing state-of-the-art algorithms in terms of objective and visual perceptions.

Multifidelity uncertainty quantification with models based on dissimilar parameters

Multifidelity uncertainty quantification (MF UQ) sampling approaches have been shown to significantly reduce the variance of statistical estimators while preserving the bias of the highest-fidelity model, provided that the low-fidelity models are well correlated. However, maintaining a high level of correlation can be challenging, especially when models depend on different input uncertain parameters, which drastically reduces the correlation. Existing MF UQ approaches do not adequately address this issue. In this work, we propose a new sampling strategy that exploits a shared space to improve the correlation among models with dissimilar parameterization. We achieve this by transforming the original coordinates onto an auxiliary manifold using the adaptive basis (AB) method (Tipireddy and Ghanem, 2014). The AB method has two main benefits: (1) it provides an effective tool to identify the low-dimensional manifold on which each model can be represented, and (2) it enables easy transformation of polynomial chaos representations from high- to low-dimensional spaces. This latter feature is used to identify a shared manifold among models without requiring additional evaluations. We present two algorithmic flavors of the new estimator to cover different analysis scenarios, including those with legacy and non-legacy high-fidelity (HF) data. We provide numerical results for analytical examples, a direct field acoustic test, and a finite element model of a nuclear fuel assembly. For all examples, we compare the proposed strategy against both single-fidelity and MF estimators based on the original model parameterization.

Wavelet Transforms and Machine Learning Methods for the Study of Turbulence

This article investigates the applications of wavelet transforms and machine learning methods in studying turbulent flows. The wavelet-based hierarchical eddy-capturing framework is built upon first principle physical models. Specifically, the coherent vortex simulation method is based on the Taylor hypothesis, which suggests that the energy cascade occurs through vortex stretching. In contrast, the adaptive wavelet collocation method relies on the Richardson hypothesis, where the self-amplification of the strain field and a hierarchical breakdown of large eddies drive the energy cascade. Wavelet transforms are computational learning architectures that propagate the input data across a sequence of linear operators to learn the underlying nonlinearity and coherent structure. Machine learning offers a wealth of data-driven algorithms that can heavily use statistical concepts to extract valuable insights into turbulent flows. Supervised machine learning needs “perfect” turbulent flow data to train data-driven turbulence models. The current advancement of artificial intelligence in turbulence modeling primarily focuses on accelerating turbulent flow simulations by learning the underlying coherence over a low-dimensional manifold. Physics-informed neural networks offer a fertile ground for augmenting first principle physics to automate specific learning tasks, e.g., via wavelet transforms. Besides machine learning, there is room for developing a common computational framework to provide a rich cross-fertilization between learning the data coherence and the first principles of multiscale physics.

Manifold Interpolation for Large-Scale Multiobjective Optimization via Generative Adversarial Networks.

Large-scale multiobjective optimization problems (LSMOPs) are characterized as optimization problems involving hundreds or even thousands of decision variables and multiple conflicting objectives. To solve LSMOPs, some algorithms designed a variety of strategies to track Pareto-optimal solutions (POSs) by assuming that the distribution of POSs follows a low-dimensional manifold. However, traditional genetic operators for solving LSMOPs have some deficiencies in dealing with the manifold, which often results in poor diversity, local optima, and inefficient searches. In this work, a generative adversarial network (GAN)-based manifold interpolation framework is proposed to learn the manifold and generate high-quality solutions on the manifold, thereby improving the optimization performance of evolutionary algorithms. We compare the proposed approach with several state-of-the-art algorithms on various large-scale multiobjective benchmark functions. The experimental results demonstrate that significant improvements have been achieved by the proposed framework in solving LSMOPs.

Learning to Locate: Adaptive Fingerprint-Based Localization With Few-Shot Relation Learning in Dynamic Indoor Environments

WiFi fingerprint-based localization has been intensive studies as a promising technology of ubiquitous location-based services. Two main concerns for its wide spread applications are to tackle with the cumbersome efforts of site survey and to combat vulnerable environment changes. To address these issues comprehensively, we propose a novel approach on adaptive fingerprint-based localization with less site survey, named as LESS, by exploring a new paradigm of radio map construction and adaptation with few-shot relation learning. Firstly, we extend sparsely collected fingerprints with the fingerprint augmentation method which produces new related data and derives their location information based on local proximity property in a low-dimensional manifold space. Then, LESS designs deep relation networks to learn not only the appropriate features but also a transferable deep-distance metric for modeling the fundamental relationships of the neighborhood fingerprints. Finally, once trained, LESS can quickly establish the neighborhood relationships among new fingerprints in the changed surroundings to realize adaptive location estimations, even without the network updating. The extensive experimental results demonstrate that LESS can achieve an attractive trade-off between the system overhead and the location performance with the superiorities over others in dynamic indoor environments.

SchröWave: Realistic voice generation by solving two-stage conditional Schrödinger bridge problems

In recent years, methods based on discrete diffusion models have achieved state-of-the-art performances in voice generation [1,2]. In theory, the voice data can be transformed into the exact Gaussian prior distributions only when the diffusion time tends to infinity. But in real applications, the Gaussian prior distribution can only be achieved approximately in a limited time duration run by these diffusion-based methods, thus resulting in sub-optimal sound quality. In this paper, we present the SchröWave to realize the continuous transformation from exact Dirac's deltas to the target voice data distribution in finite time duration, conditioned on middle voice representation with different sizes. At the same time, in order to overcome the difficulty in calculating the score on the low-dimensional manifold of voice data during the generation process, we propose to use a two-stage diffusion and generation method, while each stage implemented by solving a conditional Schrödinger bridge problem. Our experiments on the public data set LJSpeech show that the effect is significant in both objective and subjective evaluation, and achieve the new state-of-the-art MOS of 4.53. Audio samples are available at https://schrowave.github.io.

Manifold learning for fMRI time-varying functional connectivity.

Whole-brain functional connectivity (FC) measured with functional MRI (fMRI) evolves over time in meaningful ways at temporal scales going from years (e.g., development) to seconds [e.g., within-scan time-varying FC (tvFC)]. Yet, our ability to explore tvFC is severely constrained by its large dimensionality (several thousands). To overcome this difficulty, researchers often seek to generate low dimensional representations (e.g., 2D and 3D scatter plots) hoping those will retain important aspects of the data (e.g., relationships to behavior and disease progression). Limited prior empirical work suggests that manifold learning techniques (MLTs)-namely those seeking to infer a low dimensional non-linear surface (i.e., the manifold) where most of the data lies-are good candidates for accomplishing this task. Here we explore this possibility in detail. First, we discuss why one should expect tvFC data to lie on a low dimensional manifold. Second, we estimate what is the intrinsic dimension (ID; i.e., minimum number of latent dimensions) of tvFC data manifolds. Third, we describe the inner workings of three state-of-the-art MLTs: Laplacian Eigenmaps (LEs), T-distributed Stochastic Neighbor Embedding (T-SNE), and Uniform Manifold Approximation and Projection (UMAP). For each method, we empirically evaluate its ability to generate neuro-biologically meaningful representations of tvFC data, as well as their robustness against hyper-parameter selection. Our results show that tvFC data has an ID that ranges between 4 and 26, and that ID varies significantly between rest and task states. We also show how all three methods can effectively capture subject identity and task being performed: UMAP and T-SNE can capture these two levels of detail concurrently, but LE could only capture one at a time. We observed substantial variability in embedding quality across MLTs, and within-MLT as a function of hyper-parameter selection. To help alleviate this issue, we provide heuristics that can inform future studies. Finally, we also demonstrate the importance of feature normalization when combining data across subjects and the role that temporal autocorrelation plays in the application of MLTs to tvFC data. Overall, we conclude that while MLTs can be useful to generate summary views of labeled tvFC data, their application to unlabeled data such as resting-state remains challenging.

On Manifold Learning in Plato's Cave: Remarks on Manifold Learning and Physical Phenomena.

Many techniques in machine learning attempt explicitly or implicitly to infer a low-dimensional manifold structure of an underlying physical phenomenon from measurements without an explicit model of the phenomenon or the measurement apparatus. This paper presents a cautionary tale regarding the discrepancy between the geometry of measurements and the geometry of the underlying phenomenon in a benign setting. The deformation in the metric illustrated in this paper is mathematically straightforward and unavoidable in the general case, and it is only one of several similar effects. While this is not always problematic, we provide an example of an arguably standard and harmless data processing procedure where this effect leads to an incorrect answer to a seemingly simple question. Although we focus on manifold learning, these issues apply broadly to dimensionality reduction and unsupervised learning.

PCAfold 2.0—Novel tools and algorithms for low-dimensional manifold assessment and optimization

We describe an update to our open-source Python package, PCAfold, designed to help researchers generate, analyze and improve low-dimensional data manifolds. In the current version, PCAfold 2.0, we introduce novel tools and algorithms for assessing and optimizing low-dimensional manifolds. This includes a method that generates a “map” of local feature sizes that can help pinpoint researchers to problematic regions on a manifold. We introduce a novel cost function that characterizes the quality of a manifold topology with a single number. We develop two algorithms for feature selection based on principal component analysis (PCA) that use the cost function as an objective function to minimize. We introduce a quantity of interest (QoI)-aware dimensionality reduction strategy where data projections are computed using an artificial neural network and are directly optimized towards representing various projection-independent and projection-dependent QoIs. We also introduce an implementation of partition of unity networks (POUnets) for efficient reconstruction of QoIs from low-dimensional manifolds based on combining neural network classification with localized polynomial regression. Our software can be broadly applicable in all domains of science and engineering that aim to reduce data dimensionality, as well as in the fundamental research on representation learning.

Low-dimensional data-based surrogate model of a continuum-mechanical musculoskeletal system based on non-intrusive model order reduction

Over the last decades, computer modeling has evolved from a supporting tool for engineering prototype design to an ubiquitous instrument in non-traditional fields such as medical rehabilitation. This area comes with unique challenges, e.g. the complex modeling of soft tissue or the analysis of musculoskeletal systems. Conventional modeling approaches like the finite element (FE) method are computationally costly when dealing with such models, limiting their usability for real-time simulation or deployment on low-end hardware, if the model at hand cannot be simplified without losing its expressiveness. Non-traditional approaches such as surrogate modeling using data-driven model order reduction are used to make complex high-fidelity models more widely available regardless. They often involve a dimensionality reduction step, in which the high-dimensional system state is transformed onto a low-dimensional subspace or manifold, and a regression approach to capture the reduced system behavior. While most publications focus on one dimensionality reduction, such as principal component analysis (PCA) (linear) or autoencoder (nonlinear), we consider and compare PCA, kernel PCA, autoencoders, as well as variational autoencoders for the approximation of a continuum-mechanical system. In detail, we demonstrate the benefits of the surrogate modeling approach on a complex musculoskeletal system of a human upper-arm with severe nonlinearities and physiological geometry. We consider both, the model’s deformation and the internal stress as the two main quantities of interest in a FE context. By doing so we are able to create computationally low-cost surrogate models which capture the system behavior with high approximation quality and fast evaluations.

An Alzheimer’s disease category progression sub-grouping analysis using manifold learning on ADNI

Many current statistical and machine learning methods have been used to explore Alzheimer’s disease (AD) and its associated patterns that contribute to the disease. However, there has been limited success in understanding the relationship between cognitive tests, biomarker data, and patient AD category progressions. In this work, we perform exploratory data analysis of AD health record data by analyzing various learned lower dimensional manifolds to separate early-stage AD categories further. Specifically, we used Spectral embedding, Multidimensional scaling, Isomap, t-Distributed Stochastic Neighbour Embedding, Uniform Manifold Approximation and Projection, and sparse denoising autoencoder based manifolds on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. We then determine the clustering potential of the learned embeddings and then determine if category sub-groupings or sub-categories can be found. We then used a Kruskal–sWallis H test to determine the statistical significance of the discovered AD subcategories. Our results show that the existing AD categories do exhibit sub-groupings, especially in mild cognitive impairment transitions in many of the tested manifolds, showing there may be a need for further subcategories to describe AD progression.

A global perspective on the intrinsic dimensionality of COVID-19 data

We develop a novel global perspective of the complexity of the relationships between three COVID-19 datasets, the standardised per-capita growth rate of COVID-19 cases and deaths, and the Oxford Coronavirus Government Response Tracker COVID-19 Stringency Index (CSI) which is a measure describing a country’s stringency of lockdown policies. We use a state-of-the-art heterogeneous intrinsic dimension estimator implemented as a Bayesian mixture model, called Hidalgo. Our findings suggest that these highly popular COVID-19 statistics may project onto two low-dimensional manifolds without significant information loss, suggesting that COVID-19 data dynamics are generated from a latent mechanism characterised by a few important variables. The low dimensionality imply a strong dependency among the standardised growth rates of cases and deaths per capita and the CSI for countries over 2020–2021. Importantly, we identify spatial autocorrelation in the intrinsic dimension distribution worldwide. The results show how high-income countries are more prone to lie on low-dimensional manifolds, likely arising from aging populations, comorbidities, and increased per capita mortality burden from COVID-19. Finally, the temporal stratification of the dataset allows the examination of the intrinsic dimension at a more granular level throughout the pandemic.