Kolmogorov N-width Research Articles

The Approximation Characteristics of Weighted Band-Limited Function Space

This article primarily investigates the width problem within weighted band-limited function space in a uniform setting. Through an analysis of the properties of s-numbers, we establish a connection between the widths of weighted band-limited function spaces and the s-numbers of infinite-dimensional diagonal operators. Furthermore, employing the discretization method, we estimate the exact asymptotic orders of Kolmogorov n-width and linear n-width in the weighted band-limited function space, which is characterized by the weight ω=ωk=|k|rk∈Z0.

Positional embeddings for solving PDEs with evolutional deep neural networks

This work extends the paradigm of evolutional deep neural networks (EDNNs) to solving parametric time-dependent partial differential equations (PDEs) on domains with geometric structure. By introducing positional embeddings based on eigenfunctions of the Laplace-Beltrami operator, geometric properties are encoded intrinsically and Dirichlet, Neumann and periodic boundary conditions of the PDE solution are enforced directly through the neural network architecture. The proposed embeddings lead to improved error convergence for static PDEs and extend EDNNs towards computational domains of realistic complexity.Several steps are taken to improve performance of EDNNs: Solving the EDNN update equation with a Krylov solver avoids the explicit assembly of Jacobians and enables scaling to larger neural networks. Computational efficiency is further improved by an ad-hoc active sampling scheme that uses the PDE dynamics to effectively sample collocation points. A modified linearly implicit Rosenbrock method is proposed to alleviate the time step requirements of stiff PDEs. Lastly, a completely training-free approach, which automatically enforces initial conditions and only requires time integration, is compared against EDNNs that are trained on the initial conditions. We report results for the Korteweg-de Vries equation, a nonlinear heat equation and (nonlinear) advection-diffusion problems on domains with and without holes and various boundary conditions, to demonstrate the effectiveness of the method. The numerical results highlight EDNNs as a promising surrogate model for parametrized PDEs with slow decaying Kolmogorov n-width.

A fast and accurate domain decomposition nonlinear manifold reduced order model

This paper integrates nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD). NM-ROMs approximate the full order model (FOM) state in a nonlinear-manifold by training a shallow, sparse autoencoder using FOM snapshot data. These NM-ROMs can be advantageous over linear-subspace ROMs (LS-ROMs) for problems with slowly decaying Kolmogorov n-width. However, the number of NM-ROM parameters that need to be trained scales with the size of the FOM. Moreover, for “extreme-scale” problems, the storage of high-dimensional FOM snapshots alone can make ROM training expensive. To alleviate the training cost, this paper applies DD to the FOM, computes NM-ROMs on each subdomain, and couples them to obtain a global NM-ROM. This approach has several advantages: Subdomain NM-ROMs can be trained in parallel, involve fewer parameters to be trained than global NM-ROMs, require smaller subdomain FOM dimensional training data, and can be tailored to subdomain-specific features of the FOM. The shallow, sparse architecture of the autoencoder used in each subdomain NM-ROM allows application of hyper-reduction (HR), reducing the complexity caused by nonlinearity and yielding computational speedup of the NM-ROM. This paper provides the first application of NM-ROM (with HR) to a DD problem. In particular, this paper details an algebraic DD reformulation of the FOM, training a NM-ROM with HR for each subdomain, and a sequential quadratic programming (SQP) solver to evaluate the coupled global NM-ROM. Theoretical convergence results for the SQP method and a priori and a posteriori error estimates for the DD NM-ROM with HR are provided. The proposed DD NM-ROM with HR approach is numerically compared to a DD LS-ROM with HR on the 2D steady-state Burgers’ equation, showing an order of magnitude improvement in accuracy of the proposed DD NM-ROM over the DD LS-ROM.

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real-time and many-query evaluations of parametric Partial Differential Equations (PDEs). Linear techniques such as Proper Orthogonal Decomposition (POD) and Greedy algorithms have been analyzed thoroughly, but they are more suitable when dealing with linear and affine models showing a fast decay of the Kolmogorov n-width. On one hand, the autoencoder architecture represents a nonlinear generalization of the POD compression procedure, allowing one to encode the main information in a latent set of variables while extracting their main features. On the other hand, Graph Neural Networks (GNNs) constitute a natural framework for studying PDE solutions defined on unstructured meshes. Here, we develop a non-intrusive and data-driven nonlinear reduction approach, exploiting GNNs to encode the reduced manifold and enable fast evaluations of parametrized PDEs. We show the capabilities of the methodology for several models: linear/nonlinear and scalar/vector problems with fast/slow decay in the physically and geometrically parametrized setting. The main properties of our approach consist of (i) high generalizability in the low-data regime even for complex behaviors, (ii) physical compliance with general unstructured grids, and (iii) exploitation of pooling and un-pooling operations to learn from scattered data.

A nonlinear-manifold reduced-order model and operator learning for partial differential equations with sharp solution gradients

Traditional linear subspace-based reduced order models (LS-ROMs) can be used to significantly accelerate simulations in which the solution space of the discretized system has a small dimension (with a fast decaying Kolmogorov n-width). However, LS-ROMs struggle to achieve speed-ups in problems whose solution space has a large dimension, such as highly nonlinear problems whose solutions have large gradients. Such an issue can be alleviated by combining nonlinear model reduction with operator learning. Over the past decade, many nonlinear manifold-based reduced order models (NM-ROM) have been proposed. In particular, NM-ROMs based on deep neural networks (DNN) have received increasing interest. This work takes inspiration from adaptive basis methods and specifically focuses on developing an NM-ROM based on Convolutional Neural Network-based autoencoders (CNNAE) with iteration-dependent trainable kernels. Additionally, we investigate DNN-based and quadratic operator inference strategies between latent spaces. A strategy to perform vectorized implicit time integration is also proposed. We demonstrate that the proposed CNN-based NM-ROM, combined with DNN-based operator inference, generally performs better than commonly employed strategies (in terms of prediction accuracy) on a benchmark advection-dominated problem. The method also presents substantial gain in terms of training speed per epoch, with a training time about one order of magnitude smaller than the one associated with a state-of-the-art technique performing with the same level of accuracy.

Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds

This work presents two novel approaches for the symplectic model reduction of high-dimensional Hamiltonian systems using data-driven quadratic manifolds. Classical symplectic model reduction approaches employ linear symplectic subspaces for representing the high-dimensional system states in a reduced-dimensional coordinate system. While these approximations respect the symplectic nature of Hamiltonian systems, linear basis approximations can suffer from slowly decaying Kolmogorov N-width, especially in wave-type problems, which then requires a large basis size. We propose two different model reduction methods based on recently developed quadratic manifolds, each presenting its own advantages and limitations. The addition of quadratic terms to the state approximation, which sits at the heart of the proposed methodologies, enables us to better represent intrinsic low-dimensionality in the problem at hand. Both approaches are effective for issuing predictions in settings well outside the range of their training data while providing more accurate solutions than the linear symplectic reduced-order models.

The Fuzzy Width Theory in the Finite-Dimensional Space and Sobolev Space

This paper aims to fuzzify the width problem of classical approximation theory. New concepts of fuzzy Kolmogorov n-width and fuzzy linear n-width are introduced on the basis of α-fuzzy distance which is induced by the fuzzy norm. Furthermore, the relationship between the classical widths in linear normed space and the fuzzy widths in fuzzy linear normed space is discussed. Finally, the exact asymptotic orders of the fuzzy Kolmogorov n-width and fuzzy linear n-width corresponding to a given fuzzy norm in finite-dimensional space and Sobolev space are estimated.

Exponentially Convergent Multiscale Finite Element Method

We provide a concise review of the exponentially convergent multiscale finite element method (ExpMsFEM) for efficient model reduction of PDEs in heterogeneous media without scale separation and in high-frequency wave propagation. The ExpMsFEM is built on the non-overlapped domain decomposition in the classical MsFEM while enriching the approximation space systematically to achieve a nearly exponential convergence rate regarding the number of basis functions. Unlike most generalizations of the MsFEM in the literature, the ExpMsFEM does not rely on any partition of unity functions. In general, it is necessary to use function representations dependent on the right-hand side to break the algebraic Kolmogorov n-width barrier to achieve exponential convergence. Indeed, there are online and offline parts in the function representation provided by the ExpMsFEM. The online part depends on the right-hand side locally and can be computed in parallel efficiently. The offline part contains basis functions that are used in the Galerkin method to assemble the stiffness matrix; they are all independent of the right-hand side, so the stiffness matrix can be used repeatedly in multi-query scenarios.

Non-linear Manifold Reduced-Order Models with Convolutional Autoencoders and Reduced Over-Collocation Method

Non-affine parametric dependencies, nonlinearities and advection-dominated regimes of the model of interest can result in a slow Kolmogorov n-width decay, which precludes the realization of efficient reduced-order models based on linear subspace approximations. Among the possible solutions, there are purely data-driven methods that leverage autoencoders and their variants to learn a latent representation of the dynamical system, and then evolve it in time with another architecture. Despite their success in many applications where standard linear techniques fail, more has to be done to increase the interpretability of the results, especially outside the training range and not in regimes characterized by an abundance of data. Not to mention that none of the knowledge on the physics of the model is exploited during the predictive phase. In order to overcome these weaknesses, we implement the non-linear manifold method introduced by Lee and Carlberg (J Comput Phys 404:108973, 2020) with hyper-reduction achieved through reduced over-collocation and teacher–student training of a reduced decoder. We test the methodology on a 2d non-linear conservation law and a 2d shallow water models, and compare the results obtained with a purely data-driven method for which the dynamics is evolved in time with a long-short term memory network.

Kolmogorov n–width and Lagrangian physics-informed neural networks: A causality-conforming manifold for convection-dominated PDEs

We make connections between complexity of training of physics-informed neural networks (PINNs) and Kolmogorov n-width of the solution. Leveraging this connection, we then propose Lagrangian PINNs (LPINNs) as a partial differential equation (PDE)-informed solution for convection-dominated problems. PINNs employ neural-networks to find the solutions of PDE-constrained optimization problems with initial conditions and boundary conditions as soft or hard constraints. These soft constraints are often blamed to be the sources of the complexity in the training phase of PINNs. Here, we demonstrate that the complexity of training (i) is closely related to the Kolmogorov n-width associated with problems demonstrating transport, convection, traveling waves, or moving fronts, and therefore becomes apparent in convection-dominated flows, and (ii) persists even when the boundary conditions are strictly enforced. Given this realization, we describe the mechanism underlying the training schemes such as those used in eXtended PINNs (XPINN), curriculum learning, and sequence-to-sequence learning. For an important category of PDEs, i.e., governed by non-linear convection–diffusion equation, we propose reformulating PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed solution. A parallel architecture with two branches is proposed. One branch solves for the state variables on the characteristics, and the second branch solves for the low-dimensional characteristics curves. The proposed architecture conforms to the causality innate to the convection, and leverages the direction of travel of the information in the domain, i.e., on the characteristics. This approach is unique as it reduces the complexity of convection-dominated PINNs at the PDE level, instead of optimization strategies and/or schedulers. Finally, we demonstrate that the loss landscapes of LPINNs are less sensitive to the so-called “complexity” of the problems, i.e., convection, compared to those in the traditional PINNs in the Eulerian framework.

Reprint of: Robust and globally efficient reduction of parametric, highly nonlinear computational models and real time online performance

This paper considers the problem of parametric, nonlinear, projection-based model order reduction (PMOR) and makes three contributions to advance its state of the art. First, it presents a computational strategy for reducing the total offline cost of nonlinear PMOR by enabling hyperreduction to be performed adaptively within the typical greedy sampling procedure, rather than uniformly at each of its iteration. Then, it presents a more accurate and computationally efficient approach for training hyperreduction based on residual Jacobians rather than residuals. Finally, it demonstrates for a large-scale, industrial-grade, parametric, nonlinear application, that the two aforementioned contributions make both offline and online stages of nonlinear PMOR practical in today’s engineering environments. The paper also shows that despite the so-called Kolmogorov n-width barrier for the model reduction of convection-dominated transport problems, the current state of the art of nonlinear, Petrov–Galerkin PMOR equipped with the advances proposed in this paper is capable of accelerating by orders of magnitude the accurate solution of shape-parametric, high Reynolds number, viscous flow problems represented by the Reynolds-averaged Navier–Stokes equations augmented with turbulence modeling. All three contributions are presented and discussed in the contexts of steady-state CFD problems; however, they are equally applicable to unsteady CFD problems as well as problems in solid mechanics and structural dynamics.

Model reduction of convection-dominated partial differential equations via optimization-based implicit feature tracking

This work introduces a new approach to reduce the computational cost of solving partial differential equations (PDEs) with convection-dominated solutions: model reduction with implicit feature tracking. Traditional model reduction techniques use an affine subspace to reduce the dimensionality of the solution manifold and, as a result, yield limited reduction and require extensive training due to the slowly decaying Kolmogorov n-width of convection-dominated problems. The proposed approach circumvents the slowly decaying n-width limitation by using a nonlinear approximation manifold systematically defined by composing a low-dimensional affine space with a space of bijections of the underlying domain. Central to the implicit feature tracking approach is a residual minimization problem over the reduced nonlinear manifold that simultaneously determines the reduced coordinates in the affine space and the domain mapping that minimize the residual of the unreduced PDE discretization. This is analogous to standard minimum-residual reduced-order models, except instead of only minimizing the residual over the affine subspace of PDE states, our method enriches the optimization space to also include admissible domain mappings. The nonlinear trial manifold is constructed by using the proposed residual minimization formulation to determine domain mappings that cause parametrized features to align in a reference domain for a set of training parameters. Because the feature is stationary in the reference domain, i.e., the convective nature of solution removed, the snapshots are effectively compressed to define an affine subspace. The space of domain mappings, originally constructed using high-order finite elements, is also compressed in a way that ensures the boundaries of the original domain are maintained. Several numerical experiments are provided, including transonic and supersonic, inviscid, compressible flows, to demonstrate the potential of the method to yield accurate approximations to convection-dominated problems with limited training.

Robust and globally efficient reduction of parametric, highly nonlinear computational models and real time online performance

This paper considers the problem of parametric, nonlinear, projection-based model order reduction (PMOR) and makes three contributions to advance its state of the art. First, it presents a computational strategy for reducing the total offline cost of nonlinear PMOR by enabling hyperreduction to be performed adaptively within the typical greedy sampling procedure, rather than uniformly at each of its iteration. Then, it presents a more accurate and computationally efficient approach for training hyperreduction based on residual Jacobians rather than residuals. Finally, it demonstrates for a large-scale, industrial-grade, parametric, nonlinear application, that the two aforementioned contributions make both offline and online stages of nonlinear PMOR practical in today’s engineering environments. The paper also shows that despite the so-called Kolmogorov n-width barrier for the model reduction of convection-dominated transport problems, the current state of the art of nonlinear, Petrov–Galerkin PMOR equipped with the advances proposed in this paper is capable of accelerating by orders of magnitude the accurate solution of shape-parametric, high Reynolds number, viscous flow problems represented by the Reynolds-averaged Navier–Stokes equations augmented with turbulence modeling. All three contributions are presented and discussed in the contexts of steady-state CFD problems; however, they are equally applicable to unsteady CFD problems as well as problems in solid mechanics and structural dynamics.

A learning-based projection method for model order reduction of transport problems

The Kolmogorov n-width of the solution manifolds of transport-dominated problems can decay slowly. As a result, it can be challenging to design efficient and accurate reduced order models (ROMs) for such problems. To address this issue, we propose a new learning-based projection method to construct nonlinear adaptive ROMs for transport problems. The construction follows the offline–online decomposition. In the offline stage, we train a neural network to construct adaptive reduced basis dependent on time and model parameters. In the online stage, we project the solution to the learned reduced manifold. Inheriting the merits from both deep learning and the projection method, the proposed method is more efficient than the conventional linear projection-based methods, and may reduce the generalization error of a solely learning-based ROM. Unlike some learning-based projection methods, the proposed method does not need to take derivatives of the neural network in the online stage.

Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases

Nonlinear model order reduction has opened the door to parameter optimization and uncertainty quantification in complex physics problems governed by nonlinear equations. In particular, the computational cost of solving these equations can be reduced by means of local reduced-order bases. This article examines the benefits of a physics-informed cluster analysis for the construction of cluster-specific reduced-order bases. We illustrate that the choice of the dissimilarity measure for clustering is fundamental and highly affects the performances of the local reduced-order bases. It is shown that clustering with an angle-based dissimilarity on simulation data efficiently decreases the intra-cluster Kolmogorov N-width. Additionally, an a priori efficiency criterion is introduced to assess the relevance of a ROM-net, a methodology for the reduction of nonlinear physics problems introduced in our previous work in Daniel et al. (2020) [48]. This criterion also provides engineers with a very practical method for ROM-nets' hyperparameters calibration under constrained computational costs for the training phase. On five different physics problems, our physics-informed clustering strategy significantly outperforms classic strategies for the construction of local reduced-order bases in terms of projection errors.

A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder

Traditional linear subspace reduced order models (LS-ROMs) are able to accelerate physical simulations in which the intrinsic solution space falls into a subspace with a small dimension, i.e., the solution space has a small Kolmogorov n-width. However, for physical phenomena not of this type, e.g., any advection-dominated flow phenomena such as in traffic flow, atmospheric flows, and air flow over vehicles, a low-dimensional linear subspace poorly approximates the solution. To address cases such as these, we have developed a fast and accurate physics-informed neural network ROM, namely nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs. Our method takes advantage of the existing numerical methods that are used to solve the corresponding full order models. The efficiency is achieved by developing a hyper-reduction technique in the context of the NM-ROM. Numerical results show that neural networks can learn a more efficient latent space representation on advection-dominated data from 1D and 2D Burgers' equations. A speedup of up to 2.6 for 1D Burgers' and a speedup of 11.7 for 2D Burgers' equations are achieved with an appropriate treatment of the nonlinear terms through a hyper-reduction technique. Finally, a posteriori error bounds for the NM-ROMs are derived that take account of the hyper-reduced operators.

Optimal pointwise sampling for L2 approximation

Given a function u∈L2=L2(D,μ), where μ is a measure on a set D, and a linear subspace Vn⊂L2 of dimension n, we show that near-best approximation of u in Vn can be computed from a near-optimal budget of Cn pointwise evaluations of u, with C>1 a universal constant. The sampling points are drawn according to some random distribution, the approximation is computed by a weighted least-squares method, and the error is assessed in expected L2 norm. This result improves on the results in [6,8] which require a sampling budget that is sub-optimal by a logarithmic factor, thanks to a sparsification strategy introduced in [17,18]. As a consequence, we obtain for any compact class K⊂L2 that the sampling number ρCnrand(K)L2 in the randomized setting is dominated by the Kolmogorov n-width dn(K)L2. While our result shows the existence of a randomized sampling with such near-optimal properties, we discuss remaining issues concerning its generation by a computationally efficient algorithm.

Model reduction of traveling-wave problems via Radon cumulative distribution transform

Due to their sizeable Kolmogorov n-width, travel-wave problems have brought critical challenges to conventional model reduction techniques. This study aims to provide new insights into this problem by exploiting the Radon cumulative distribution transform (R-CDT) that emerged in the sector of computer vision science. By virtue of the unique property that nonlinear invertible R-CDT renders both traveling and scaling components into amplitude modulations, a substantial model-reduction is achieved in the R-CDT space, while sustaining high accuracy. The method is parameter-free and data-driven, which lends itself to problems regardless of the dimensions or boundary conditions.

Low-Rank Registration Based Manifolds for Convection-Dominated PDEs

We develop an auto-encoder-type nonlinear dimensionality reduction algorithm to enable the construction of reduced order models of systems governed by convection-dominated nonlinear partial differential equations (PDEs), i.e. snapshots of solutions with large Kolmogorov n-width. Although several existing nonlinear manifold learning methods, such as LLE, ISOMAP, MDS, etc., appear as compelling candidates to reduce the dimensionality of such data, most are not applicable to reduced order modeling of PDEs, because: (i) they typically lack a straightforward mapping from the latent space to the high-dimensional physical space, and (ii) the identified latent variables are often difficult to interpret. In our proposed method, these limitations are overcome by training a low-rank diffeomorphic spatio-temporal grid that registers the output sequence of the PDEs on a non-uniform parameter/time-varying grid, such that the Kolmogorov n-width of the mapped data on the learned grid is minimized. We demonstrate the efficacy and interpretability of our proposed approach on several challenging manufactured computer vision-inspired tasks and physical systems.

A nudged hybrid analysis and modeling approach for realtime wake-vortex transport and decay prediction

We put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements for air traffic improvements. Toward emerging applications of digital twins in aviation, the proposed approach allows for constructing a realtime predictive tool for wake-vortex transport and decay systems. We build on the fact that in realistic application, there are uncertainties in initial and boundary conditions, model parameters, as well as measurements. Moreover, conventional nonlinear ROMs based on Galerkin projection (GROMs) suffer from imperfection and solution instabilities, especially for advection-dominated flows with slow decay in the Kolmogorov n-width. In the presented LSTM nudging (LSTM-N) approach, we fuse forecasts from a combination of imperfect GROM and uncertain state estimates, with sparse Eulerian sensor measurements to provide more reliable predictions in a dynamical data assimilation framework. We illustrate our concept by solving the two-dimensional vorticity transport equation. We investigate the effects of measurements noise and state estimate uncertainty on the performance of the LSTM-N behavior. We also demonstrate that it can sufficiently handle different levels of temporal and spatial measurement sparsity, and offer a huge potential in developing next-generation digital twin technologies for aerospace applications.