Kolmogorov N-width Research Articles

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

We propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

Estimates for [formula omitted]-widths of sets of smooth functions on complex spheres

In this work we investigate n-widths of multiplier operators Λ∗ and Λ, defined for functions on the complex sphere Ωd of ℂd, associated with sequences of multipliers of the type {λm,n∗}m,n∈N, λm,n∗=λ(m+n) and {λm,n}m,n∈N, λm,n=λ(max{m,n}), respectively, for a bounded function λ defined on [0,∞). If the operators Λ∗ and Λ are bounded from Lp(Ωd) into Lq(Ωd), 1≤p,q≤∞, and Up is the closed unit ball of Lp(Ωd), we study lower and upper estimates for the n-widths of Kolmogorov, linear, of Gelfand and of Bernstein, of the sets Λ∗Up and ΛUp in Lq(Ωd). As application we obtain, in particular, estimates for the Kolmogorov n-width of classes of Sobolev, of finitely differentiable, infinitely differentiable and analytic functions on the complex sphere, in Lq(Ωd), which are order sharp in various important situations.

Reduced Basis Greedy Selection Using Random Training Sets

Reduced bases have been introduced for the approximation of parametrized PDEs in applications where many online queries are required. Their numerical efficiency for such problems has been theoretically confirmed in Binev et al. (SIAM J. Math. Anal. 43 (2011) 1457–1472) and DeVore et al. (Constructive Approximation 37 (2013) 455–466), where it is shown that the reduced basis space Vn of dimension n, constructed by a certain greedy strategy, has approximation error similar to that of the optimal space associated to the Kolmogorov n-width of the solution manifold. The greedy construction of the reduced basis space is performed in an offline stage which requires at each step a maximization of the current error over the parameter space. For the purpose of numerical computation, this maximization is performed over a finite training set obtained through a discretization of the parameter domain. To guarantee a final approximation error ε for the space generated by the greedy algorithm requires in principle that the snapshots associated to this training set constitute an approximation net for the solution manifold with accuracy of order ε. Hence, the size of the training set is the ε covering number for M and this covering number typically behaves like exp(Cε−1/s) for some C > 0 when the solution manifold has n-width decay O(n−s). Thus, the shear size of the training set prohibits implementation of the algorithm when ε is small. The main result of this paper shows that, if one is willing to accept results which hold with high probability, rather than with certainty, then for a large class of relevant problems one may replace the fine discretization by a random training set of size polynomial in ε−1. Our proof of this fact is established by using inverse inequalities for polynomials in high dimensions.

Optimal Subspaces for Mean Square Approximation of Classes of Differentiable Functions on a Segment

In this paper, a set of optimal subspaces is specified for L2 approximation of three classes of functions in the Sobolev spaces $$W_{2}^{{(r)}}$$ defined on a segment and subject to certain boundary conditions. A subspace X of a dimension not exceeding n is called optimal for a function class A if the best approximation of A by X is equal to the Kolmogorov n-width of A. These boundary conditions correspond to subspaces of periodically extended functions with symmetry properties. All approximating subspaces are generated by equidistant shifts of a single function. The conditions of optimality are given in terms of Fourier coefficients of a generating function. In particular, we indicate optimal spline spaces of all degrees d $$ \geqslant $$ r – 1 with equidistant knots of several different types.

A long short-term memory embedding for hybrid uplifted reduced order models

In this paper, we introduce an uplifted reduced order modeling (UROM) approach through the integration of standard projection based methods with long short-term memory (LSTM) embedding. Our approach has three modeling layers or components. In the first layer, we utilize an intrusive projection approach to model the dynamics represented by the largest modes. The second layer consists of an LSTM model to account for residuals beyond this truncation. This closure layer refers to the process of including the residual effect of the discarded modes into the dynamics of the largest scales. However, the feasibility of generating a low rank approximation tails off for higher Kolmogorov n-width systems due to the underlying nonlinear processes. The third uplifting layer, called super-resolution, addresses this limited representation issue by expanding the span into a larger number of modes utilizing the versatility of LSTM. Therefore, our model integrates a physics-based projection model with a memory embedded LSTM closure and an LSTM based super-resolution model. In several applications, we exploit the use of Grassmann manifold to construct UROM for unseen conditions. We perform numerical experiments by using the Burgers and Navier–Stokes equations with quadratic nonlinearity. Our results show the robustness of the proposed approach in building reduced order models for parameterized systems and confirm the improved trade-off between accuracy and efficiency.

Optimal subspaces for mean square approximation of classes of differentiable functions on a segment

In this paper, a set of optimal subspaces is specified for L2 approximation of three classes of functions in the Sobolev spaces $$W_{2}^{{(r)}}$$ defined on a segment and subject to certain boundary conditions. A subspace X of a dimension not exceeding n is called optimal for a function class A if the best approximation of A by X is equal to the Kolmogorov n-width of A. These boundary conditions correspond to subspaces of periodically extended functions with symmetry properties. All approximating subspaces are generated by equidistant shifts of a single function. The conditions of optimality are given in terms of Fourier coefficients of a generating function. In particular, we indicate optimal spline spaces of all degrees d $$ \geqslant $$ r – 1 with equidistant knots of several different types.

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) proper orthogonal decomposition (POD). Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov n-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov n-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov–Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Finally, we demonstrate the ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic n-width limitations of linear subspaces.

Approximation tools and decay rates for eigenvalues of integral operators on a general setting

We provide the characterization of the Peetre-type K-functional on a compact two-point homogeneous space in terms of the rate of approximation of a family of multipliers operators. This extends the well known results on the spherical setting. The characterization is employed to prove that an abstract Holder condition or finite order of differentiability assumption on generating positive kernels of integral operators implies a sharp decay rates for their eigenvalues sequences. Consequently, sharp upper bounds for the Kolmogorov n-width of unit balls in reproducing kernel Hilbert space are obtained.

Kolmogorov n-widths for linear dynamical systems

Kolmogorov $n$-widths and Hankel singular values are two commonly used concepts in model reduction. Here we show that for the special case of linear time-invariant dynamical (LTI) systems, these two concepts are directly connected. More specifically, the greedy search applied to the Hankel operator of an LTI system resembles the minimizing subspace for the Kolmogorov n-width and the Kolmogorov $n$-width of an LTI system equals its $(n+1)st$ Hankel singular value once the subspaces are appropriately defined. We also establish a lower bound for the Kolmorogov $n$-width for parametric LTI systems and illustrate that the method of active subspaces can be viewed as the dual concept to the minimizing subspace for the Kolmogorov $n$-width.

Decay of the Kolmogorov [formula omitted]-width for wave problems

The Kolmogorov N-width dN(M) describes the rate of the worst-case error (w.r.t. a subset M⊂H of a normed space H) arising from a projection onto the best-possible linear subspace of H of dimension N∈N. Thus, dN(M) sets a limit to any projection-based approximation such as determined by the reduced basis method. While it is known that dN(M) decays exponentially fast for many linear coercive parameterized partial differential equations, i.e., dN(M)=O(e−βN), we show in this note, that only dN(M)=O(N−1∕2) for initial–boundary-value problems of the hyperbolic wave equation with discontinuous initial conditions. This is aligned with the known slow decay of dN(M) for the linear transport problem.

On some estimates for best approximations of bivariate functions by Fourier–Jacobi sums in the mean

Some problems in computational mathematics and mathematical physics lead to Fourier series expansions of functions (solutions) in terms of special functions, i.e., to approximate representations of functions (solutions) by partial sums of corresponding expansions. However, the errors of these approximations are rarely estimated or minimized in certain classes of functions. In this paper, the convergence rate (of best approximations) of a Fourier series in terms of Jacobi polynomials is estimated in classes of bivariate functions characterized by a generalized modulus of continuity. An approximation method based on “spherical” partial sums of series is substantiated, and the introduction of a corresponding class of functions is justified. A two-sided estimate of the Kolmogorov N-width for bivariate functions is given.

Hyperbolic geometrical approach to model reduction

Model reduction of large scale systems is an actively researched area of modelling and control. The problem is more involved if uncertainties are also present and a computational tractable nominal model is needed for the design. Based on results of the Kolmogorov n-width theory the paper provides useful bounds for the worst case approximation error - both H2 and H∞ – in terms of the hyperbolic distance related to the sets of uncertain poles. A related model reduction strategy that uses only this a priori pole information is also proposed. The method is illustrated through numerical examples.

Linear and nonlinear approximation of spherical radial basis function networks

In this paper, the center-selection strategy of spherical radial basis function networks (SRBFNs) is considered. To approximate functions in the Bessel-potential Sobolev classes, we provide two lower bounds of nonlinear SRBFN approximation. In the first one, we prove that, up to a logarithmic factor, the lower bound of SRBFN approximation coincides with the Kolmogorov n-width. In the other one, we prove that if a pseudo-dimension assumption is imposed on the activation function, then the logarithmic factor can even be omitted. These results together with the well known Jackson-type inequality of SRBFN approximation imply that the center-selection strategy does not affect the approximation capability of SRBFNs very much, provided the target function belongs to the Bessel-potential Sobolev classes. Thus, we can choose centers only for the algorithmic factor. Hence, a linear SRBFN approximant whose centers are specified before the training is recommended.

Kolmogorov n-Widths of Function Classes Induced by a Non-Degenerate Differential Operator: A Convex Duality Approach

The problem of computing the asymptotic order of the Kolmogorov n-width of the unit ball of the space of multivariate periodic functions induced by a differential operator associated with a polynomial in the general case when the ball is compactly embedded into L 2 has been open for a long time. In the present paper, we use convex analytical tools to solve it in the case when the differential operator is non-degenerate.

Sharp estimates for the convergence rate of Fourier—Bessel series

Sharp estimates are derived for the convergence rate of Fourier series in terms of Bessel functions of the first kind for some classes of functions characterized by a generalized modulus of continuity. The Kolmogorov N-width of these classes of functions are also estimated.

Optimal Local Approximation Spaces for Generalized Finite Element Methods with Application to Multiscale Problems

The paper addresses a numerical method for solving second order elliptic partial differential equations that describe fields inside heterogeneous media. The scope is general and treats the case of rough coefficients, i.e., coefficients with values in $L^{\infty}(\Omega)$. This class of coefficients includes as examples media with microstructure as well as media with multiple nonseparated length scales. The approach taken here is based on the the generalized finite element method (GFEM) introduced in [I. Babuska, G. Caloz, and J. E. Osborn, SIAM J. Numer. Anal., 31 (1994), pp. 945–981] and elaborated in [I. Babuska, U. Banerjee, and J. Osborn, Int. J. Comput. Methods, 1 (2004), pp. 67–103], [I. Babuska and J. Melenk, Internat. J. Numer. Methods Engrg., 40 (1997), pp. 727–758], and [T. Strouboulis, I. Babuska, and K. Copps, Comput. Methods Appl. Mech. Engrg., 181 (2001), pp. 43–69]. The GFEM is constructed by partitioning the computational domain $\Omega$ into a collection of preselected subsets $\omega_{i},i=1,2,\ldots m$, and constructing finite-dimensional approximation spaces $\Psi_{i}$ over each subset using local information. The notion of the Kolmogorov n-width is used to identify the optimal local approximation spaces. These spaces deliver local approximations with errors that decay almost exponentially with the degrees of freedom $N_{i}$ in the energy norm over $\omega_{i}$. The local spaces $\Psi_{i}$ are used within the GFEM scheme to produce a finite-dimensional subspace $S^{N}$ of $H^{1}(\Omega)$, which is then employed in the Galerkin method. It is shown that the error in the Galerkin approximation decays in the energy norm almost exponentially (i.e., superalgebraically) with respect to the degrees of freedom N. When length scales “separate” and the microstructure is sufficiently fine with respect to the length scale of the domain $\omega_{i}$, it is shown that homogenization theory can be used to construct local approximation spaces with exponentially decreasing error in the preasymtotic regime.

N-Widths, sup–infs, and optimality ratios for the k-version of the isogeometric finite element method

We begin the mathematical study of the k-method utilizing the theory of Kolmogorov n-widths. The k-method is a finite element technique where spline basis functions of higher-order continuity are employed. It is a fundamental feature of the new field of isogeometric analysis. In previous works, it has been shown that using the k-method has many advantages over the classical finite element method in application areas such as structural dynamics, wave propagation, and turbulence. The Kolmogorov n-width and sup–inf were introduced as tools to assess the effectiveness of approximating functions. In this paper, we investigate the approximation properties of the k-method with these tools. Following a review of theoretical results, we conduct a numerical study in which we compute the n-width and sup–inf for a number of one-dimensional cases. This study sheds further light on the approximation properties of the k-method. We finish this paper with a comparison study of the k-method and the classical finite element method and an analysis of the robustness of polynomial approximation.

Relative widths of smooth functions determined by linear differential operator

Let W and V be centrally symmetric sets in a normed space X. The relative Kolmogorov n-width of W relative to V in X is defined by K n ( W , V , X ) : = inf L n sup f ∈ W inf g ∈ V ∩ L n ‖ f − g ‖ X , where the infimum is taken over all n-dimensional subspaces L n of X. Let P r ( t ) = t σ ∏ j = 1 l ( t 2 − t j 2 ) , t j ⩾ 0 , j = 1 , 2 , … , l , l ⩾ 1 , σ = 0 or 1, r = 2 l + σ . Denote by P r ( D ) ( D = d d t ) the self-conjugate differential operator induced by P r ( t ) , and by K q ( P r ) the generalized Sobolev class of 2 π-periodic smooth functions defined by K q ( P r ) = { f ∈ L ˜ q , 2 π ( r ) : ‖ P r ( D ) f ‖ q ⩽ 1 } , where L ˜ q , 2 π ( r ) = { f ∈ L q ( T ) : f ( r − 1 ) is absolutely continuous on T = [ 0 , 2 π ] and f ( r ) ∈ L q ( T ) } . In this paper, we consider the relative Kolmogorov n-width of K p ( P r ) relative to K p ( P r ) in the space L q ( T ) , and obtain the asymptotic orders of relative widths K n ( K ∞ ( P r ) , K ∞ ( P r ) , L q ( T ) ) and K n ( K 1 ( P r ) , K 1 ( P r ) , L q ( T ) ) for 1 ⩽ q ⩽ ∞ .

RELATIVE WIDTH OF SMOOTH CLASSES OF MULTIVARIATE PERIODIC FUNCTIONS WITH RESTRICTIONS ON ITERATED LAPLACE DERIVATIVES IN THE L2-METRIC

For two subsets W and V of a Banach space X, let K n ( W, V, X) denote the relative Kolmogorov n-width of W relative to V defined by K n ( W , V , X ) = inf L n sup f ∈ W inf g ∈ V ∩ L n ∥ f - g ∥ x , where the infimum is taken over all n-dimensional linear subspaces L n of X. Let W 2(Δ r ) denote the class of 2π-periodic functions f with d-variables satisfying ∫ [ - π , π ] d | Δ r f ( x ) | 2 d x ≤ 1 , while Δ r is the r-iterate of Laplace operator Δ. This article discusses the relative Kolmogorov n-width of W 2(Δ r ) relative to W 2(Δ r ) in L q ([−π, π] d ) (1 ≤ q ≤ ∞), and obtain its weak asymptotic result.

Estimates for n-widths of the Hardy-type operators (Addendum to “Improved estimates for the approximation numbers of the Hardy-type operators”)

Consider the Hardy-type operator T : L p ( a , b ) → L p ( a , b ) ,- ∞ ⩽ a < b ⩽ ∞ , which is defined by ( Tf ) ( x ) = v ( x ) ∫ a x u ( t ) f ( t ) dt . It is shown that ρ n ( T ) = 1 n α p ∫ a b u ( x ) v ( x ) + O ( n - 2 ) , where ρ n ( T ) stands for any of the following: the Kolmogorov n-width, the Gel’fand n-width, the Bernstein n-width or the nth approximation number of T.