Multilevel Approximations Research Articles

Three-Level Schemes with Double Change in the Time Step

Nonstationary problems are solved numerically by applying multilevel (with more than two levels) time approximations. They are easy to construct and relatively easy to study in the case of uniform grids. However, the numerical study of application-oriented problems often involves approximations with a variable time step. The construction of multilevel schemes on nonuniform grids is associated with maintaining the prescribed accuracy and ensuring the stability of the approximate solution. In this paper, three-level schemes for the approximate solution of the Cauchy problem for a second-order evolution equation are constructed in the special case of a doubled or halved step size. The focus is on the approximation features in the transition between different step sizes. The study is based on general results of the stability (well-posedness) theory of operator-difference schemes in a finite-dimensional Hilbert space. Estimates for stability with respect to initial data and the right-hand side are obtained in the case of a doubled or halved time step.

Benchmark coupled-cluster lattice energy of crystalline benzene and assessment of multi-level approximations in the many-body expansion.

The many-body expansion (MBE) is promising for the efficient, parallel computation of lattice energies in organic crystals. Very high accuracy should be achievable by employing coupled-cluster singles, doubles, and perturbative triples at the complete basis set limit [CCSD(T)/CBS] for the dimers, trimers, and potentially tetramers resulting from the MBE, but such a brute-force approach seems impractical for crystals of all but the smallest molecules. Here, we investigate hybrid or multi-level approaches that employ CCSD(T)/CBS only for the closest dimers and trimers and utilize much faster methods like Møller-Plesset perturbation theory (MP2) for more distant dimers and trimers. For trimers, MP2 is supplemented with the Axilrod-Teller-Muto (ATM) model of three-body dispersion. MP2(+ATM) is shown to be a very effective replacement for CCSD(T)/CBS for all but the closest dimers and trimers. A limited investigation of tetramers using CCSD(T)/CBS suggests that the four-body contribution is entirely negligible. The large set of CCSD(T)/CBS dimer and trimer data should be valuable in benchmarking approximate methods for molecular crystals and allows us to see that a literature estimate of the core-valence contribution of the closest dimers to the lattice energy using just MP2 was overbinding by 0.5 kJ mol-1, and an estimate of the three-body contribution from the closest trimers using the T0 approximation in local CCSD(T) was underbinding by 0.7 kJ mol-1. Our CCSD(T)/CBS best estimate of the 0K lattice energy is -54.01 kJ mol-1, compared to an estimated experimental value of -55.3 ± 2.2 kJ mol-1.

Multilevel Picard approximations for McKean-Vlasov stochastic differential equations

In the literature there exist approximation methods for McKean-Vlasov stochastic differential equations which have a computational effort of order 3. In this article we introduce full-history recursive multilevel Picard approximations for McKean-Vlasov stochastic differential equations. We prove that these MLP approximations have computational effort of order 2+ which is essentially optimal in high dimensions.

Convergence and rate optimality of adaptive multilevel stochastic Galerkin FEM

Abstract We analyze an adaptive algorithm for the numerical solution of parametric elliptic partial differential equations in two-dimensional physical domains, with coefficients and right-hand-side functions depending on infinitely many (stochastic) parameters. The algorithm generates multilevel stochastic Galerkin approximations; these are represented in terms of a sparse generalized polynomial chaos expansion with coefficients residing in finite element spaces associated with different locally refined meshes. Adaptivity is driven by a two-level a posteriori error estimator and employs a Dörfler-type marking on the joint set of spatial and parametric error indicators. We show that, under an appropriate saturation assumption, the proposed adaptive strategy yields optimal convergence rates with respect to the overall dimension of the underlying multilevel approximation spaces.

Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations

Abstract One of the most challenging problems in applied mathematics is the approximate solution of nonlinear partial differential equations (PDEs) in high dimensions. Standard deterministic approximation methods like finite differences or finite elements suffer from the curse of dimensionality in the sense that the computational effort grows exponentially in the dimension. In this work we overcome this difficulty in the case of reaction–diffusion type PDEs with a locally Lipschitz continuous coervice nonlinearity (such as Allen–Cahn PDEs) by introducing and analyzing truncated variants of the recently introduced full-history recursive multilevel Picard approximation schemes.

Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations.

For a long time it has been well-known that high-dimensional linear parabolic partial differential equations (PDEs) can be approximated by Monte Carlo methods with a computational effort which grows polynomially both in the dimension and in the reciprocal of the prescribed accuracy. In other words, linear PDEs do not suffer from the curse of dimensionality. For general semilinear PDEs with Lipschitz coefficients, however, it remained an open question whether these suffer from the curse of dimensionality. In this paper we partially solve this open problem. More precisely, we prove in the case of semilinear heat equations with gradient-independent and globally Lipschitz continuous nonlinearities that the computational effort of a variant of the recently introduced multilevel Picard approximations grows at most polynomially both in the dimension and in the reciprocal of the required accuracy.

A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations

Deep neural networks and other deep learning methods have very successfully been applied to the numerical approximation of high-dimensional nonlinear parabolic partial differential equations (PDEs), which are widely used in finance, engineering, and natural sciences. In particular, simulations indicate that algorithms based on deep learning overcome the curse of dimensionality in the numerical approximation of solutions of semilinear PDEs. For certain linear PDEs it has also been proved mathematically that deep neural networks overcome the curse of dimensionality in the numerical approximation of solutions of such linear PDEs. The key contribution of this article is to rigorously prove this for the first time for a class of nonlinear PDEs. More precisely, we prove in the case of semilinear heat equations with gradient-independent nonlinearities that the numbers of parameters of the employed deep neural networks grow at most polynomially in both the PDE dimension and the reciprocal of the prescribed approximation accuracy. Our proof relies on recently introduced full history recursive multilevel Picard approximations for semilinear PDEs.

The MRCC program system: Accurate quantum chemistry from water to proteins.

MRCC is a package of ab initio and density functional quantum chemistry programs for accurate electronic structure calculations. The suite has efficient implementations of both low- and high-level correlation methods, such as second-order Møller-Plesset (MP2), random-phase approximation (RPA), second-order algebraic-diagrammatic construction [ADC(2)], coupled-cluster (CC), configuration interaction (CI), and related techniques. It has a state-of-the-art CC singles and doubles with perturbative triples [CCSD(T)] code, and its specialties, the arbitrary-order iterative and perturbative CC methods developed by automated programming tools, enable achieving convergence with regard to the level of correlation. The package also offers a collection of multi-reference CC and CI approaches. Efficient implementations of density functional theory (DFT) and more advanced combined DFT-wave function approaches are also available. Its other special features, the highly competitive linear-scaling local correlation schemes, allow for MP2, RPA, ADC(2), CCSD(T), and higher-order CC calculations for extended systems. Local correlation calculations can be considerably accelerated by multi-level approximations and DFT-embedding techniques, and an interface to molecular dynamics software is provided for quantum mechanics/molecular mechanics calculations. All components of MRCC support shared-memory parallelism, and multi-node parallelization is also available for various methods. For academic purposes, the package is available free of charge.

Multilevel approximation of parametric and stochastic PDES

We analyze the complexity of the sparse-grid interpolation and sparse-grid quadrature of countably-parametric functions which take values in separable Banach spaces with unconditional bases. Assuming a suitably quantified holomorphic dependence on the parameters, we establish dimension-independent convergence rate bounds for sparse-grid approximation schemes. Analogous results are shown in the case that the parametric families are obtained as approximate solutions of corresponding parametric-holomorphic, nonlinear operator equations as considered in [A. Cohen and A. Chkifa and Ch. Schwab: Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs, J. Math. Pures Appl. 103 (2015) 400–428], for example by means of stable, finite-dimensional approximations. We discuss in detail nonlinear Petrov–Galerkin projections. Error and convergence rate bounds for constructive and explicit multilevel, sparse tensor approximation schemes combining sparse-grid interpolation in the parameter space and general, multilevel discretization schemes in the physical domain are proved. The present results unify and generalize earlier works in terms of the admissible multilevel approximations in the physical domain (comprising general stable Petrov–Galerkin and discrete Petrov–Galerkin schemes, collocation and stable domain approximations) and in terms of the admissible operator equations (comprising smooth, nonlinear locally well-posed operator equations). Additionally, a novel computational strategy to localize sequences of nested index sets for the anisotropic Smolyak interpolation in parameter space is developed which realizes best [Formula: see text]-term benchmark convergence rates. We also consider Smolyak-type quadratures in this general setting, for which we establish improved convergence rates based on cancellations in the integrands’ gpc expansions by symmetries of quadratures and the probability measure [J.Z̃ech and Ch.S̃chwab: Convergence rates of high dimensional Smolyak quadrature, Report 2017-27, SAM ETH Zürich (2017)]. Several examples illustrating the abstract theory include domain uncertainty quantification (UQ) for general, linear, second-order, elliptic advection–reaction–diffusion equations on polygonal domains, where optimal convergence rates of FEM are known to require local mesh refinement near corners. Further applications of the presently developed theory comprise evaluations of posterior expectations in Bayesian inverse problems.

Bridging the Gap Between Flat and Hierarchical Low-Rank Matrix Formats: The Multilevel Block Low-Rank Format

Matrices possessing a low-rank property arise in numerous scientific applications. This property can be exploited to provide a substantial reduction of the complexity of their LU or LDL T factorization. Among the possible low-rank formats, the flat Block Low-Rank (BLR) format is easy to use but achieves superlinear complexity. Alternatively, the hierarchical formats achieve linear complexity at the price of a much more complex, hierarchical matrix representation. In this paper, we propose a new format based on multilevel BLR approximations: the matrix is recursively defined as a BLR matrix whose full-rank blocks are themselves represented by BLR matrices. We call this format multilevel BLR (MBLR). Contrarily to hierarchical matrices, the number of levels in the block hierarchy is fixed to a given constant; while this format can still be represented within the H formalism, we show that applying the H theory to it leads to very pessimistic complexity bounds. We therefore extend the theory to prove better bounds, and show that the MBLR format provides a simple way to finely control the desired complexity of dense factorizations. By striking a balance between the simplicity of the BLR format and the low complexity of the hierarchical ones, the MBLR format bridges the gap between flat and hierarchical low-rank matrix formats. The MBLR format is of particular relevance in the context of sparse direct solvers, for which it is able to trade off the optimal dense complexity of the hierarchical formats to benefit from the simplicity and flexibility of the BLR format while still achieving O(n) sparse complexity. We finally compare our MBLR format with the related BLR-H (or Lattice-H) format; our theoretical analysis shows that both formats achieve the same asymptotic complexity for a given top level block size.

On the Algebraic Construction of Sparse Multilevel Approximations of Elliptic Tensor Product Problems

We consider the solution of elliptic problems on the tensor product of two physical domains as for example present in the approximation of the solution covariance of elliptic partial differential equations with random input. Previous sparse approximation approaches used a geometrically constructed multilevel hierarchy. Instead, we construct this hierarchy for a given discretized problem by means of the algebraic multigrid method. Thereby, we are able to apply the sparse grid combination technique to problems given on complex geometries and for discretizations arising from unstructured grids, which was not feasible before. Numerical results show that our algebraic construction exhibits the same convergence behaviour as the geometric construction, while being applicable even in black-box type PDE solvers.

Multilevel higher-order quasi-Monte Carlo Bayesian estimation

We propose and analyze deterministic multilevel (ML) approximations for Bayesian inversion of operator equations with uncertain distributed parameters, subject to additive Gaussian measurement data. The algorithms use a ML approach based on deterministic, higher-order quasi-Monte Carlo (HoQMC) quadrature for approximating the high-dimensional expectations, which arise in the Bayesian estimators, and a Petrov–Galerkin (PG) method for approximating the solution to the underlying partial differential equation (PDE). This extends the previous single-level (SL) approach from [J. Dick, R. N. Gantner, Q. T. Le Gia and Ch. Schwab, Higher order quasi-Monte Carlo integration for Bayesian estimation, Report 2016-13, Seminar for Applied Mathematics, ETH Zürich (in review)]. Compared to the SL approach, the present convergence analysis of the ML method requires stronger assumptions on holomorphy and regularity of the countably-parametric uncertainty-to-observation maps of the forward problem. As in the SL case and in the affine-parametric case analyzed in [J. Dick, F. Y. Kuo, Q. T. Le Gia and Ch. Schwab, Multi-level higher order QMC Galerkin discretization for affine parametric operator equations, SIAM J. Numer. Anal. 54 (2016) 2541–2568], we obtain sufficient conditions which allow us to achieve arbitrarily high, algebraic convergence rates in terms of work, which are independent of the dimension of the parameter space. The convergence rates are limited only by the spatial regularity of the forward problem, the discretization order achieved by the PG discretization, and by the sparsity of the uncertainty parametrization. We provide detailed numerical experiments for linear elliptic problems in two space dimensions, with [Formula: see text] parameters characterizing the uncertain input, confirming the theory and showing that the ML HoQMC algorithms can outperform, in terms of error versus computational work, both multilevel Monte Carlo (MLMC) methods and SL HoQMC methods, provided the parametric solution maps of the forward problems afford sufficient smoothness and sparsity of the high-dimensional parameter spaces.

A Multilevel Approach for Computing the Limited-Memory Hessian and its Inverse in Variational Data Assimilation

Use of data assimilation techniques is becoming increasingly common across many application areas. The inverse Hessian (and its square root) plays an important role in several different aspects of these processes. In geophysical and engineering applications, the Hessian-vector product is typically defined by sequential solution of a tangent linear and adjoint problem; for the inverse Hessian, however, no such definition is possible. Frequently, the requirement to work in a matrix-free environment means that compact representation schemes are employed. In this paper, we propose an enhanced approach based on a new algorithm for constructing a multilevel eigenvalue decomposition of a given operator, which results in a much more efficient compact representation of the inverse Hessian (and its square root). After introducing these multilevel approximations, we investigate their accuracy and demonstrate their efficiency (in terms of reducing memory requirements and/or computational time) using the example of preconditioning a Gauss-Newton minimization procedure.

Multiscale decomposition for heterogeneous land‐atmosphere systems

AbstractThe land‐atmosphere system is characterized by pronounced land surface heterogeneity and vigorous atmospheric turbulence both covering a wide range of scales. The multiscale surface heterogeneities and multiscale turbulent eddies interact nonlinearly with each other. Understanding these multiscale processes quantitatively is essential to the subgrid parameterizations for weather and climate models. In this paper, we propose a method for surface heterogeneity quantification and turbulence structure identification. The first part of the method is an orthogonal transform in the probability density function (PDF) domain, in contrast to the orthogonal wavelet transforms which are performed in the physical space. As the basis of the whole method, the orthogonal PDF transform (OPT) is used to asymptotically reconstruct the original signals by representing the signal values with multilevel approximations. The “patch” idea is then applied to these reconstructed fields in order to recognize areas at the land surface or in turbulent flows that are of the same characteristics. A patch here is a connected area with the same approximation. For each recognized patch, a length scale is then defined to build the energy spectrum. The OPT and related energy spectrum analysis, as a whole referred to as the orthogonal PDF decomposition (OPD), is applied to two‐dimensional heterogeneous land surfaces and atmospheric turbulence fields for test. The results show that compared to the wavelet transforms, the OPD can reconstruct the original signal more effectively, and accordingly, its energy spectrum represents the signal's multiscale variation more accurately. The method we propose in this paper is of general nature and therefore can be of interest for problems of multiscale process description in other geophysical disciplines.

High-order methods as an alternative to using sparse tensor products for stochastic Galerkin FEM

Solutions of random elliptic boundary value problems admit efficient approximations by polynomials on the parameter domain. Each coefficient in such an expansion is a spatially dependent function, and can be approximated within a hierarchy of finite element spaces. If the finite elements are of sufficiently high order, using just a single spatial mesh is predicted to achieve the same convergence rate with respect to the total number of degrees of freedom as sparse tensor product constructions and other multilevel stochastic Galerkin approximations. Numerical computations for an elliptic two-point boundary value problem confirm this and indicate no loss of accuracy for a single-level method compared to using a sparse tensor product with the same total number of degrees of freedom.

Convergence Rates of Multilevel and Sparse Tensor Approximations for a Random Elliptic PDE

Partial differential equations with random coefficients can be cast as parametric problems with a potentially infinite-dimensional parameter domain. For a class of elliptic equations, we derive convergence rates of approximations based on a tensorized polynomial basis on the parameter domain and a sequence of spatial discretizations. We prove that a sparse tensor product construction achieves essentially the same convergence rate as a multilevel approximation in which the spatial discretization level is chosen separately for each coefficient of the solution with respect to the polynomial basis. In some cases, the same rate is attained also by a single level approximation, using just one spatial discretization for all coefficients. We suggest an adaptive algorithm that reaches the optimal convergence rate with respect to the total number of degrees of freedom in a multilevel setting, without prior knowledge of this rate. Numerical computations confirm theoretical results.

Multi-level Low-rank Approximation-based Spectral Clustering for image segmentation

Spectral clustering is a well-known graph-theoretic approach of finding natural groupings in a given dataset, and has been broadly used in image segmentation. Nowadays, High-Definition (HD) images are widely used in television broadcasting and movies. Segmenting these high resolution images presents a grand challenge to the current spectral clustering techniques. In this paper, we propose an efficient spectral method, Multi-level Low-rank Approximation-based Spectral Clustering (MLASC), to segment high resolution images. By integrating multi-level low-rank matrix approximations, i.e., the approximations to the affinity matrix and its subspace, as well as those for the Laplacian matrix and the Laplacian subspace, MLASC gains great computational and spacial efficiency. In addition, the proposed fast sampling strategy make it possible to select sufficient data samples in MLASC, leading to accurate approximation and segmentation. From a theoretical perspective, we mathematically prove the correctness of MLASC, and provide detailed analysis on its computational complexity. Through experiments performed on both synthetic and real datasets, we demonstrate the superior performance of MLASC.

Multilevel approximations in sample-based inversion from the Dirichlet-to-Neumann map

In 2005, Christen and Fox introduced a delayed acceptance Metropolis-Hastings (DAMH) algorithm that improved computational efficiency in sample-based imaging of electrical conductivity (EIT). That work used a linear approximation to the forward map in the first step of the algorithm. In this paper, we develop an alternative approximation for use in DAMH, namely a multilevel approximation developed from the hierarchy of coarse-scale models obtained by variational coarsening. This approach builds on two important strengths of robust multigrid solvers. First, the cost of a fine-scale solution of the forward map scales linearly with the degrees of freedom, and hence, it is provides better efficiency for algorithms performing sample-based inference. Second, the homogenization implicit in robust variational multigrid methods gives better solutions at coarse scales than homogenization by averaging of coefficients. We report results from a stylized example in electrical impedance imaging where data is a noisy and incomplete measurement of the Dirichlet-to-Neumann map.

Partitioned, Multilevel Response Surfaces for Modeling Complex Systems

The most prevalent type of approximating functions employed for efficient engineering analysis and design integration are polynomial response surfaces. However, the construction of response surface approximations has been limited to problems with only a few variables, due to the number of analyses necessary to fit sufficiently accurate models. An approach is presented for partitioning response surfaces and constructing multilevel approximations for problems with larger numbers of variables. Using this approach, the (computer) experimentation necessary for fitting response surface models is reduced tremendously. A modified composite experimental design is also presented for the construction of response models that provide more consistently accurate predictions across the range of the design variables. The multilevel, partitioned response surface modeling and modified composite design approaches are demonstrated for the preliminary design of a commercial turbofan engine, an example problem defined in collaboration with Allison Engine Company, Rolls-Royce Aerospace Group.

Adaptive multilevel finite element approximations of semilinear elliptic boundary value problems

In this paper we consider two aspects of the problem of designing efficient numerical methods for the approximation of semilinear boundary value problems. First we consider the use of two and multilevel algorithms for approximating the discrete solution. Secondly we consider adaptive mesh refinement based on feedback information from coarse level approximations. The algorithms are based on an a posteriori error estimate, where the error is estimated in terms of computable quantities only. The a posteriori error estimate is used for choosing appropriate spaces in the multilevel algorithms, mesh refinements, as a stopping criterion and finally it gives an estimate of the total error.