Nonlinear Manifold Research Articles

A nonlinear manifold model order reduction approach for thermo‐mechanically coupled plasticity

AbstractThis study employs a nonlinear manifold mode order reduction (MOR) approach to address the high computational cost inherent in multi‐physical nonlinear models, which are discretized by the finite element method (FEM), with a particular focus on thermo‐mechanically coupled plasticity under finite strain conditions. Utilizing nodal snapshots obtained from a full‐order simulation, a projection matrix is constructed through singular value decomposition (SVD), facilitating the construction of a reduced system within a lower‐dimensional space. This reduction process entails the multiplication of the discretized residual vector and stiffness matrix by a linear projection matrix, complemented by the incorporation of a nonlinear projection component to ensure accurate reconstruction within the nonlinear part. Subsequently, a comprehensive three‐dimensional benchmark test is conducted to assess the accuracy and efficacy of the reduced‐order model.

Boosting Barlow Twins Reduced Order Modeling for Machine Learning‐Based Surrogate Models in Multiphase Flow Problems

AbstractWe present an innovative approach called boosting Barlow Twins reduced order modeling (BBT‐ROM) to enhance the reliability of machine learning surrogate models for multiphase flow problems. BBT‐ROM builds upon Barlow Twins reduced order modeling that leverages self‐supervised learning to effectively handle linear and nonlinear manifolds by constructing well‐structured latent spaces of input parameters and output quantities. To address the challenge of high contrast data in multiphase flow problems due to injection wells and faults, we employ a boosting algorithm within BBT‐ROM. This algorithm sequentially trains a set of weak models (i.e., inaccurate models), improving prediction accuracy through ensemble learning. To evaluate the performance of BBT‐ROM, we conduct three three‐dimensional multiphase flow problems, including waterflooding and geologic carbon storage (GCS), with varying numbers of input parameter cases and model domain features. The results demonstrate that BBT‐ROM excels at predicting non‐wetting phase saturation (e.g., oil or saturation) and fluid pressure, with average relative errors ranging from 0.5% to 3%. Importantly, BBT‐ROM showcases robustness when faced with limited input parameter space during GCS testing.

Fast Terminal Synergetic Speed Control for SMPMSM Drive System

For the surface‐mounted permanent magnet synchronous motor (SMPMSM) drive system, a fast terminal synergetic speed control (FTSSC) is proposed to improve its speed dynamic performance and robustness. In the proposed control scheme, the ultra‐local model of SMPMSM drive system, which considers motor parameter uncertainties and unknown disturbances, is set up based on the algebraic parameter identification method. Then, the nonlinear terminal manifold is designed to achieve speed rapid convergence in finite time, moreover, the fast synergetic speed control law that satisfies the voltage and current constraints of SMPMSM drive system is derived. Meanwhile, the Lyapunov function is designed to prove the stability of the proposed fast terminal synergetic speed‐controlled SMPMSM drive system. Finally, the effectiveness and feasibility of proposed control is verified through the experimental research of SMPMSM drive system. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Learning the latent dynamics of fluid flows from high-fidelity numerical simulations using parsimonious diffusion maps

We use parsimonious diffusion maps (PDMs) to discover the latent dynamics of high-fidelity Navier–Stokes simulations with a focus on the two-dimensional (2D) fluidic pinball problem. By varying the Reynolds number Re, different flow regimes emerge, ranging from steady symmetric flows to quasi-periodic asymmetric and chaos. The proposed non-linear manifold learning scheme identifies in a crisp manner the expected intrinsic dimension of the underlying emerging dynamics over the parameter space. In particular, PDMs estimate that the emergent dynamics in the oscillatory regime can be captured by just two variables, while in the chaotic regime, the dominant modes are three as anticipated by the normal form theory. On the other hand, proper orthogonal decomposition/principal component analysis (POD/PCA), most commonly used for dimensionality reduction in fluid mechanics, does not provide such a crisp separation between the dominant modes. To validate the performance of PDMs, we also compute the reconstruction error, by constructing a decoder using geometric harmonics (GHs). We show that the proposed scheme outperforms the POD/PCA over the whole Re number range. Thus, we believe that the proposed scheme will allow for the development of more accurate reduced order models for high-fidelity fluid dynamics simulators, relaxing the curse of dimensionality in numerical analysis tasks such as bifurcation analysis, optimization, and control.

Tensor multi-view clustering method for natural image segmentation

Image segmentation methods based on clustering have become a advancement in the field of computer vision, and the performance of these methods mainly depends on the performance of the clustering algorithm. However, these methods have some shortcomings. First, a single feature cannot accurately obtain the complete information of the superpixels. Second, the commonly used Euclidean distance cannot extract the nonlinear manifold structure between superpixels. Third, the high-dimensional information between superpixels is not considered. Therefore, this article proposes a Tensor Multi-view Clustering Method for Natural Image Segmentation (TMCNIS), in which firstly the multiclass features of the superpixels are extracted to obtain complete information. Secondly, the Jensen–Shannon divergence is used to delineate the relationship between superpixels to obtain a more complete nonlinear structure. Thirdly, an adaptively weighted tensor Schatten-p norm is proposed to better approximate the target rank of the tensor. It also automatically shrinks the singular values appropriately to construct a finer tensor, thus fully capturing the spatial structure in the tensor. Experimental results on four clustering datasets and two image segmentation datasets demonstrate the superiority of TMCNIS compared to existing methods.

Unsupervised Anomaly Detection via Nonlinear Manifold Learning

Abstract Anomalies are samples that significantly deviate from the rest of the data and their detection plays a major role in building machine learning models that can be reliably used in applications such as data-driven design and novelty detection. The majority of existing anomaly detection methods either are exclusively developed for (semi) supervised settings, or provide poor performance in unsupervised applications where there are no training data with labeled anomalous samples. To bridge this research gap, we introduce a robust, efficient, and interpretable methodology based on nonlinear manifold learning to detect anomalies in unsupervised settings. The essence of our approach is to learn a low-dimensional and interpretable latent representation (aka manifold) for all the data points such that normal samples are automatically clustered together and hence can be easily and robustly identified. We learn this low-dimensional manifold by designing a learning algorithm that leverages either a latent map Gaussian process (LMGP) or a deep autoencoder (AE). Our LMGP-based approach, in particular, provides a probabilistic perspective on the learning task and is ideal for high-dimensional applications with scarce data. We demonstrate the superior performance of our approach over existing technologies via multiple analytic examples and real-world datasets.

GP+: A Python library for kernel-based learning via Gaussian processes

In this paper we introduce GP+, an open-source library for kernel-based learning via Gaussian processes (GPs) which are powerful statistical models that are completely characterized by their parametric covariance and mean functions. GP+ is built on PyTorch and provides a user-friendly and object-oriented tool for probabilistic learning and inference. As we demonstrate with a host of examples, GP+ has a few unique advantages over other GP modeling libraries. We achieve these advantages primarily by integrating nonlinear manifold learning techniques with GPs’ covariance and mean functions. As part of introducing GP+, in this paper we also make methodological contributions that (1) enable probabilistic data fusion and inverse parameter estimation, and (2) equip GPs with parsimonious parametric mean functions which span mixed feature spaces that have both categorical and quantitative variables. We demonstrate the impact of these contributions in the context of Bayesian optimization, multi-fidelity modeling, sensitivity analysis, and calibration of computer models.

Geometric tracking on 𝒮3 based on sliding mode control

AbstractAttitude tracking on the unit sphere of dimension 3 based on sliding mode is considered in this article. The tangent bundle of Lagrangian dynamics that describes the rotational motion of a rigid body is first equipped with the structure of a Lie group, then a sliding subgroup emerged on it is defined. Next, a sliding‐mode controller is designed for attitude tracking that relies on an intrinsic error defined on the Lie group. Almost global asymptotic stability of the closed loop is demonstrated using the Lyapunov analysis. Comparisons of the proposed geometric sliding mode controller designed on the nonlinear manifold with one designed in the Euclidean space showed that under the same working conditions the former is more energy efficient, more robust to measurement noises, and has a shorter and smoother transient response.

Analysis of Manifold and its Application

Manifold learning is a field of study in machine learning and statistics that is closely associated with dimensionality reduction algorithmic techniques is gaining popularity these days. There are two types of manifold learning approaches: linear and nonlinear. Principal component analysis (PCA) and multidimensional scaling (MDS) are two examples of linear techniques that have long been staples in the statistician's arsenal for evaluating multivariate data. Nonlinear manifold learning, which encompasses diffusion maps, Laplacian Eigenmaps, Hessian Eigenmaps, Isomap, and local linear embedding, has seen a surge in research effort recently. A few of these methods are nonlinear extensions of linear approaches. A nearest search, the definition of distances or affinities between points (a crucial component of these methods' effectiveness), and an Eigen problem for embedding high-dimensional points into a lower dimensional space make up the algorithmic process of the majority of these techniques. The strengths and weaknesses of the new method are briefly reviewed in this article. In the field of computer graphics, we utilize a particular manifold learning method was first presented in statistics and machine learning to create a global, Spectral-based shape descriptor.

Robust locally nonlinear embedding (RLNE) for dimensionality reduction of high-dimensional data with noise

Local Linear Embedding (LLE) is a nonlinear manifold learning method for dimensionality reduction in high-dimensional data. However, when the data is distorted by noise, efficiency of LLE significantly diminishes. This paper proposes a robust locally nonlinear embedding (RLNE) method to alleviate the impact of noise. This is achieved by constructing nonlinear functions between data neighbors in high-dimensional space, and then mapping the relationships to low manifolds. The constrained least squares method is used to obtain more uniform weights to ensure that the neighborhood is approximately located on the local nonlinear patches of the manifold. Theoretical analysis is conducted on the reasons underlying RLNE's robustness to noise. Experimental results on synthetic and real-world data highlight RLNE's ability to preserve the intrinsic structure of data, showcasing robustness across various types data with various levels of noise, as well as with a larger number of nearest neighbors.

Diagnosis-Guided Deep Subspace Clustering Association Study for Pathogenetic Markers Identification of Alzheimer's Disease Based on Comparative Atlases.

The roles of brain region activities and genotypic functions in the pathogenesis of Alzheimer's disease (AD) remain unclear. Meanwhile, current imaging genetics methods are difficult to identify potential pathogenetic markers by correlation analysis between brain network and genetic variation. To discover disease-related brain connectome from the specific brain structure and the fine-grained level, based on the Automated Anatomical Labeling (AAL) and human Brainnetome atlases, the functional brain network is first constructed for each subject. Specifically, the upper triangle elements of the functional connectivity matrix are extracted as connectivity features. The clustering coefficient and the average weighted node degree are developed to assess the significance of every brain area. Since the constructed brain network and genetic data are characterized by non-linearity, high-dimensionality, and few subjects, the deep subspace clustering algorithm is proposed to reconstruct the original data. Our multilayer neural network helps capture the non-linear manifolds, and subspace clustering learns pairwise affinities between samples. Moreover, most approaches in neuroimaging genetics are unsupervised learning, neglecting the diagnostic information related to diseases. We presented a label constraint with diagnostic status to instruct the imaging genetics correlation analysis. To this end, a diagnosis-guided deep subspace clustering association (DDSCA) method is developed to discover brain connectome and risk genetic factors by integrating genotypes with functional network phenotypes. Extensive experiments prove that DDSCA achieves superior performance to most association methods and effectively selects disease-relevant genetic markers and brain connectome at the coarse-grained and fine-grained levels.

A fast and accurate domain decomposition nonlinear manifold reduced order model

This paper integrates nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD). NM-ROMs approximate the full order model (FOM) state in a nonlinear-manifold by training a shallow, sparse autoencoder using FOM snapshot data. These NM-ROMs can be advantageous over linear-subspace ROMs (LS-ROMs) for problems with slowly decaying Kolmogorov n-width. However, the number of NM-ROM parameters that need to be trained scales with the size of the FOM. Moreover, for “extreme-scale” problems, the storage of high-dimensional FOM snapshots alone can make ROM training expensive. To alleviate the training cost, this paper applies DD to the FOM, computes NM-ROMs on each subdomain, and couples them to obtain a global NM-ROM. This approach has several advantages: Subdomain NM-ROMs can be trained in parallel, involve fewer parameters to be trained than global NM-ROMs, require smaller subdomain FOM dimensional training data, and can be tailored to subdomain-specific features of the FOM. The shallow, sparse architecture of the autoencoder used in each subdomain NM-ROM allows application of hyper-reduction (HR), reducing the complexity caused by nonlinearity and yielding computational speedup of the NM-ROM. This paper provides the first application of NM-ROM (with HR) to a DD problem. In particular, this paper details an algebraic DD reformulation of the FOM, training a NM-ROM with HR for each subdomain, and a sequential quadratic programming (SQP) solver to evaluate the coupled global NM-ROM. Theoretical convergence results for the SQP method and a priori and a posteriori error estimates for the DD NM-ROM with HR are provided. The proposed DD NM-ROM with HR approach is numerically compared to a DD LS-ROM with HR on the 2D steady-state Burgers’ equation, showing an order of magnitude improvement in accuracy of the proposed DD NM-ROM over the DD LS-ROM.

Ricci flow-based brain surface covariance descriptors for diagnosing Alzheimer’s disease

Automated feature extraction from MRI brain scans and diagnosis of Alzheimer’s disease are ongoing challenges. With advances in 3D imaging technology, 3D data acquisition is becoming more viable and efficient than its 2D counterpart. Rather than using feature-based vectors, in this paper, for the first time, we suggest a pipeline to extract novel covariance-based descriptors from the cortical surface using the Ricci energy optimization. The covariance descriptors are components of the nonlinear manifold of symmetric positive-definite matrices, thus we focus on using the Gaussian radial basis function to apply manifold-based classification to the 3D shape problem. Applying this novel signature to the analysis of abnormal cortical brain morphometry allows for diagnosing Alzheimer’s disease. Experimental studies performed on about two hundred 3D MRI brain models, gathered from Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our descriptors in achieving remarkable classification accuracy.

Manifold-Based Verbalizer Space Re-embedding for Tuning-Free Prompt-Based Classification

Prompt-based classification adapts tasks to a cloze question format utilizing the [MASK] token and the filled tokens are then mapped to labels through pre-defined verbalizers. Recent studies have explored the use of verbalizer embeddings to reduce labor in this process. However, all existing studies require a tuning process for either the pre-trained models or additional trainable embeddings. Meanwhile, the distance between high-dimensional verbalizer embeddings should not be measured by Euclidean distance due to the potential for non-linear manifolds in the representation space. In this study, we propose a tuning-free manifold-based space re-embedding method called Locally Linear Embedding with Intra-class Neighborhood Constraint (LLE-INC) for verbalizer embeddings, which preserves local properties within the same class as guidance for classification. Experimental results indicate that even without tuning any parameters, our LLE-INC is on par with automated verbalizers with parameter tuning. And with the parameter updating, our approach further enhances prompt-based tuning by up to 3.2%. Furthermore, experiments with the LLaMA-7B&13B indicate that LLE-INC is an efficient tuning-free classification approach for the hyper-scale language models.

Reduced dynamics and geometric optimal control of nonequilibrium thermodynamics: Gaussian case

This paper introduces a control-theoretic formulation of nonequilibrium thermodynamic systems with Gibbs states of Gaussian distributions, i.e., thermodynamic systems characterized by Gaussian probability densities. The distinct features of the paper are three-fold. First, the dynamics of a thermodynamic system is studied by transferring the original state variable from the non-Euclidean and nonlinear manifold state space to a Lie group, and further to the dual space of a Lie algebra, which is endowed with vector space structures. Consequently, the resulting reduced dynamics significantly reduces the high nonlinearity appearing in the original non-Euclidean state space of a thermal process. Second, the obtained equations of motion interestingly indicate that thermodynamics can be naturally viewed as a generalization of rigid body motions, and this bridges control theory, thermodynamics, information theory, and rigid body dynamics. Third, the optimality conditions for the energy-minimum optimal control problem of probability densities are derived via geometric Pontryagin’s principle by regarding the reduced dynamics as dynamical constraints, and a unified control algorithm is developed. Finally, the proposed approach is applied to three different scenarios including two benchmark examples to demonstrate the applicability and effectiveness. The purpose of the paper is to provide a deeper understanding of thermodynamics from a control perspective, and also to draw intrinsic connections between control theory, thermodynamics, information theory, and rigid body dynamics.

Intelligent fault diagnosis of ultrasonic motors based on graph-regularized CNN-BiLSTM

The ultrasonic motor (USM) is peculiarly prone to failure due to continuous high-frequency friction-related power transfer, whose failure mechanisms are remarkably different from traditional induction motors. Intelligent fault diagnosis provides a way to alarm and avoid catastrophic losses proactively. However, previous studies using deep learning usually ignore the inherent geometric structure of the signal distribution. This paper proposes an intelligent multi-signal fault diagnosis framework for USMs to restore the linear or nonlinear manifold structure by preserving the internal structure by integrating graph regularization with deep neural networks. Firstly, the one-dimensional CNN to learn spatial correlations and BiLSTM to exploit temporal dependencies are coalesced to build the deep neural network. Then, an improved k-nearest neighbor graph is proposed to protect the geometric structure information and force the latent features to be more concentrated within their classes. Moreover, the layer in the deep architecture to integrate graph regularization is designed to reduce computation cost, and an adaptive decay strategy is considered to adjust the coefficient of graph regularized automatically. A two-stage training algorithm is developed by considering the time to calculate the graph regularization term. Finally, the proposed multi-signal fault diagnosis framework is validated using datasets from the fault injection experiment of similar USMs in China’s Yutu rover of Chang’e lunar probe. Experimental results show that the proposed method can effectively discriminate different fault types.

Learning physics-based reduced-order models from data using nonlinear manifolds.

We present a novel method for learning reduced-order models of dynamical systems using nonlinear manifolds. First, we learn the manifold by identifying nonlinear structure in the data through a general representation learning problem. The proposed approach is driven by embeddings of low-order polynomial form. A projection onto the nonlinear manifold reveals the algebraic structure of the reduced-space system that governs the problem of interest. The matrix operators of the reduced-order model are then inferred from the data using operator inference. Numerical experiments on a number of nonlinear problems demonstrate the generalizability of the methodology and the increase in accuracy that can be obtained over reduced-order modeling methods that employ a linear subspace approximation.

Model order reduction for the 1D Boltzmann-BGK equation: identifying intrinsic variables using neural networks

Kinetic equations are crucial for modeling non-equilibrium phenomena, but their computational complexity is a challenge. This paper presents a data-driven approach using reduced order models (ROM) to efficiently model non-equilibrium flows in kinetic equations by comparing two ROM approaches: proper orthogonal decomposition (POD) and autoencoder neural networks (AE). While AE initially demonstrate higher accuracy, POD’s precision improves as more modes are considered. Notably, our work recognizes that the classical POD model order reduction approach, although capable of accurately representing the non-linear solution manifold of the kinetic equation, may not provide a parsimonious model of the data due to the inherently non-linear nature of the data manifold. We demonstrate how AEs are used in finding the intrinsic dimension of a system and to allow correlating the intrinsic quantities with macroscopic quantities that have a physical interpretation.

Enhanced video clustering using multiple riemannian manifold-valued descriptors and audio-visual information

Videos inherently blend multiple modalities in real-world scenarios, primarily visual and auditory cues. When synergized, these cues foster enhanced data representations. Standard clustering techniques, primarily designed for managing vectorial data in Euclidean spaces, struggle to handle multidimensional data with nonlinear manifold structures, such as video or image sets. While recent subspace clustering methods using Riemannian manifold representation tackle this issue, they often sideline auditory information, overlooking the potential harmony between visual and auditory modalities. This paper presents an innovative approach that crafts multiple Riemannian manifold-valued descriptors to bridge this gap, encapsulating multimodal video information in a unified structure. We architect a single-modality Riemannian subspace clustering for individual modal data and extend it to a multi-modality framework, leveraging the interplay of audio-visual data. Detailed optimization and convergence analysis are also provided. The proposed approach significantly outperforms the existing state-of-the-art methods, improving accuracy by 4%, 1%, and 2% on UCF-101, UCF-sport, and AVE datasets, respectively.

A Nonlinear Manifold Embedding Extreme Learning Machine to Improve the Gas Recognition of Electronic Noses

A Nonlinear Manifold Embedding Extreme Learning Machine to Improve the Gas Recognition of Electronic Noses