Empirical Convergence Rates Research Articles

Adaptive hybrid high-order method for guaranteed lower eigenvalue bounds

The higher-order guaranteed lower eigenvalue bounds of the Laplacian in the recent work by Carstensen et al. (Numer Math 149(2):273–304, 2021) require a parameter Cst,1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C_{\ ext {st},1}$$\\end{document} that is found not robust as the polynomial degree p increases. This is related to the H1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$H^1$$\\end{document} stability bound of the L2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^{2}$$\\end{document} projection onto polynomials of degree at most p and its growth Cst, 1∝(p+1)1/2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C_{\ extrm{st, 1}}\\propto (p+1)^{1/2}$$\\end{document} as p→∞\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p \\rightarrow \\infty $$\\end{document}. A similar estimate for the Galerkin projection holds with a p-robust constant Cst,2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C_{\ ext {st},2}$$\\end{document} and Cst,2≤2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C_{\ ext {st},2} \\le 2$$\\end{document} for right-isosceles triangles. This paper utilizes the new inequality with the constant Cst,2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C_{\ ext {st},2}$$\\end{document} to design a modified hybrid high-order eigensolver that directly computes guaranteed lower eigenvalue bounds under the idealized hypothesis of exact solve of the generalized algebraic eigenvalue problem and a mild explicit condition on the maximal mesh-size in the simplicial mesh. A key advance is a p-robust parameter selection. The analysis of the new method with a different fine-tuned volume stabilization allows for a priori quasi-best approximation and improved L2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^{2}$$\\end{document} error estimates as well as a stabilization-free reliable and efficient a posteriori error control. The associated adaptive mesh-refining algorithm performs superior in computer benchmarks with striking numerical evidence for optimal higher empirical convergence rates.

Numerical analysis for electromagnetic scattering with nonlinear boundary conditions

This work studies time-dependent electromagnetic scattering from obstacles whose interaction with the wave is fully determined by a nonlinear boundary condition. In particular, the boundary condition studied in this work enforces a power law type relation between the electric and magnetic fields along the boundary. Based on time-dependent jump relations of classical boundary operators, we derive a nonlinear system of time-dependent boundary integral equations that determines the tangential traces of the scattered electric and magnetic fields. These fields can subsequently be computed at arbitrary points in the exterior domain by evaluating a time-dependent representation formula. Fully discrete schemes are obtained by discretizing the nonlinear system of boundary integral equations with Runge–Kutta based convolution quadrature in time and Raviart–Thomas boundary elements in space. Error bounds with explicitly stated convergence rates are proven, under the assumption of sufficient regularity of the exact solution. The error analysis is conducted through novel techniques based on time-discrete transmission problems and the use of a new discrete partial integration inequality. Numerical experiments illustrate the use of the proposed method and provide empirical convergence rates.

Direct Guaranteed Lower Eigenvalue Bounds with Optimal a Priori Convergence Rates for the Bi-Laplacian

An extra-stabilised Morley finite element method (FEM) directly computes guaranteed lower eigenvalue bounds with optimal a priori convergence rates for the bi-Laplace Dirichlet eigenvalues. The smallness assumption $\min\{\lambda_h,\lambda\}h_{\max}^{4}$ $\le 15.0864$ on the maximal mesh-size $h_{\max}$ makes the computed $k$-th discrete eigenvalue $\lambda_h\le \lambda$ a lower eigenvalue bound for the $k$-th Dirichlet eigenvalue $\lambda$. This holds for multiple and clusters of eigenvalues and serves for the localisation of the bi-Laplacian Dirichlet eigenvalues in particular for coarse meshes. The analysis requires interpolation error estimates for the Morley FEM with explicit constants in any space dimension $n\ge 2$, which are of independent interest. The convergence analysis in $3$D follows the Babuska-Osborn theory and relies on a companion operator for the Morley finite element method. This is based on the Worsey-Farin 3D version of the Hsieh-Clough-Tocher macro element with a careful selection of center points in a further decomposition of each tetrahedron into 12 sub-tetrahedra. Numerical experiments in 2D support the optimal convergence rates of the extra-stabilised Morley FEM and suggest an adaptive algorithm with optimal empirical convergence rates.

Sample Complexity of the Sign-Perturbed Sums Identification Method: Scalar Case*

Sign-Perturbed Sum (SPS) is a powerful finite-sample system identification algorithm which can construct confidence regions for the true data generating system with exact coverage probabilities, for any finite sample size. SPS was developed in a series of papers and it has a wide range of applications, from general linear systems, even in a closed-loop setup, to nonlinear and nonparametric approaches. Although several theoretical properties of SPS were proven in the literature, the sample complexity of the method was not analysed so far. This paper aims to fill this gap and provides the first results on the sample complexity of SPS. Here, we focus on scalar linear regression problems, that is we study the behaviour of SPS confidence intervals. We provide high probability upper bounds, under three different sets of assumptions, showing that the sizes of SPS confidence intervals shrink at a geometric rate around the true parameter, if the observation noises are subgaussian. We also show that similar bounds hold for the previously proposed outer approximation of the confidence region. Finally, we present simulation experiments comparing the theoretical and the empirical convergence rates.

Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling

We consider the problem of minimizing a high-dimensional objective function, which may include a regularization term, using only (possibly noisy) evaluations of the function. Such optimization is also called derivative-free, zeroth-order, or black-box optimization. We propose a new zeroth-order regularized optimization method, dubbed ZORO. When the underlying gradient is approximately sparse at an iterate, ZORO needs very few objective function evaluations to obtain a new iterate that decreases the objective function. We achieve this with an adaptive, randomized gradient estimator, followed by an inexact proximal-gradient scheme. Under a novel approximately sparse gradient assumption and various different convex settings, we show that the (theoretical and empirical) convergence rate of ZORO is only logarithmically dependent on the problem dimension. Numerical experiments show that ZORO outperforms existing methods with similar assumptions, on both synthetic and real datasets.

Convergent adaptive hybrid higher-order schemes for convex minimization

This paper proposes two convergent adaptive mesh-refining algorithms for the hybrid high-order method in convex minimization problems with two-sided p-growth. Examples include the p-Laplacian, an optimal design problem in topology optimization, and the convexified double-well problem. The hybrid high-order method utilizes a gradient reconstruction in the space of piecewise Raviart–Thomas finite element functions without stabilization on triangulations into simplices or in the space of piecewise polynomials with stabilization on polytopal meshes. The main results imply the convergence of the energy and, under further convexity properties, of the approximations of the primal resp. dual variable. Numerical experiments illustrate an efficient approximation of singular minimizers and improved convergence rates for higher polynomial degrees. Computer simulations provide striking numerical evidence that an adopted adaptive HHO algorithm can overcome the Lavrentiev gap phenomenon even with empirical higher convergence rates.

A universal centred high-order method based on implicit Taylor series expansion with fast second order evolution of spatial derivatives

In this paper, a centred universal high-order finite volume method for solving hyperbolic balance laws is presented. The scheme belongs to the family of ADER methods where the Generalized Riemann Problems (GRP) is a building block. The solution to these problems is carried through an implicit Taylor series expansion, which allows the scheme to work very well for stiff source terms. The series expansion is high order for the state and requires the evolution in time of spatial derivatives. A Taylor expansion of second order based on a linearization around the resolved state, is proposed for evolving spatial derivatives. A von Neumann stability analysis is carried out to investigate the range of CFL values for which stability and accuracy are balanced. The scheme implements a centred, low dissipation approach for dealing with the advective part of the system which profits from small CFL values. Numerical tests demonstrate that the present scheme can solve, efficiently, hyperbolic balance laws in both conservative and non-conservative form. The scheme is proven to be well-balanced, the C-property, a classical assessment of well-balancing of non-conservative schemes, is numerically demonstrated. An empirical convergence rate assessment shows that the expected theoretical orders of accuracy are achieved up to the fifth order.

Learning GANs in Simultaneous Game Using Sinkhorn With Positive Features

Entropy regularized optimal transport (EOT) distance and its symmetric normalization, known as the Sinkhorn divergence, offer smooth and continuous metrized weak-convergence distance metrics. They have excellent geometric properties and are useful to compare probability distributions in some generative adversarial network (GAN) models. Computing them using the original Sinkhorn matrix scaling algorithm is still expensive. The running time is quadratic at $\mathcal {O}(n^{2})$ in the size $n$ of the training dataset. This work investigates the problem of accelerating the GAN training when Sinkhorn divergence is used as a minimax objective. Let $\mathcal {G}$ be a Gaussian map from the ground space onto the positive orthant $\mathbb {R}_{+}^{r}$ with $r \ll n $ . To speed up the divergence computation, we propose the use of $c(x,y)= - \varepsilon \log \left \langle{ \mathcal {G}(x),\mathcal {G}(y) }\right \rangle $ as the ground cost. This approximation, known as Sinkhorn with positive features, brings down the running time of the Sinkhorn matrix scaling algorithm to $\mathcal {O}(r \, n)$ , which is linear in $n$ . To solve the minimax optimization in GAN, we put forward a more efficient simultaneous stochastic gradient descent-ascent (SimSGDA) algorithm in place of the standard sequential gradient techniques. Empirical evidence shows that our model, trained using SimSGDA on the DCGAN neural architecture on tiny-coloured Cats and CelebA datasets, converges to stationary points. These are the local Nash equilibrium points. We carried out numerical experiments to confirm that our model is computationally stable. It generates samples of comparable quality to those produced by prior Sinkhorn and Wasserstein GANs. Further simulations, assessed on the similarity index measures (SSIM), show that our model’s empirical convergence rate is comparable to that of WGAN-GP.

Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling

Recurrent neural networks have shown remarkable success in modeling sequences. However low resource situations still adversely affect the generalizability of these models. We introduce a new family of models, called Lattice Recurrent Units (LRU), to address the challenge of learning deep multi-layer recurrent models with limited resources. LRU models achieve this goal by creating distinct (but coupled) flow of information inside the units: a first flow along time dimension and a second flow along depth dimension. It also offers a symmetry in how information can flow horizontally and vertically. We analyze the effects of decoupling three different components of our LRU model: Reset Gate, Update Gate and Projected State. We evaluate this family of new LRU models on computational convergence rates and statistical efficiency.Our experiments are performed on four publicly-available datasets, comparing with Grid-LSTM and Recurrent Highway networks. Our results show that LRU has better empirical computational convergence rates and statistical efficiency values, along with learning more accurate language models.

A priori and a posteriori error control of discontinuous Galerkin finite element methods for the von Kármán equations

This paper analyses discontinuous Galerkin finite element methods (DGFEM) to approximate a regular solution to the von Karman equations defined on a polygonal domain. A discrete inf–sup condition sufficient for the stability of the discontinuous Galerkin discretization of a well-posed linear problem is established, and this allows the proof of local existence and uniqueness of a discrete solution to the nonlinear problem with a Banach fixed point theorem. The Newton scheme is locally second-order convergent and appears to be a robust solution strategy up to machine precision. A comprehensive a priori and a posteriori energy-norm error analysis relies on one sufficiently large stabilization parameter and sufficiently fine triangulations. In case the other stabilization parameter degenerates towards infinity, the DGFEM reduces to a novel C0-interior penalty method (IPDG). In contrast to the known C0-IPDG dueto Brenner et al., (2016, A C0 interior penalty method for a von Karman plate. Numer. Math., 1–30), the overall discrete formulation maintains symmetry of the trilinear form in the first two components—despite the general nonsymmetry of the discrete nonlinear problems. Moreover, a reliable and efficient a posteriori error analysis immediately follows for the DGFEM of this paper, while the different norms in the known C0-IPDG lead to complications with some nonresidual-type remaining terms. Numerical experiments confirm the best-approximation results and the equivalence of the error and the error estimators. A related adaptive mesh-refining algorithm leads to optimal empirical convergence rates for a nonconvex domain.

Design and analysis of ADER-type schemes for model advection–diffusion–reaction equations

We construct, analyze and assess various schemes of second order of accuracy in space and time for model advection–diffusion–reaction differential equations. The constructed schemes are meant to be of practical use in solving industrial problems and are derived following two related approaches, namely ADER and MUSCL-Hancock. Detailed analysis of linear stability and local truncation error are carried out. In addition, the schemes are implemented and assessed for various test problems. Empirical convergence rate studies confirm the theoretically expected accuracy in both space and time.

An ADER-type scheme for a class of equations arising from the water-wave theory

In this work we propose a numerical strategy to solve a family of partial differential equations arising from the water-wave theory. These problems may contain four terms; a source which is an algebraic function of the solution, a convective part involving first order spatial derivatives of the solution, a diffusive part involving second order spatial derivatives and the transient part. Unlike partial differential equations of hyperbolic or parabolic type, where the transient part is the time derivative of the solution, here the transient part can contain mixed time and space derivatives.In [Zambra et al. International Journal for Numerical Methods in Engineering 89(2):227-240, 2012], the authors proposed a globally implicit strategy to solve the Richards equation. In that case, transient terms consisted of algebraic expressions of the solution. Motivated by this work, we propose a one-step finite volume method to deal with problems in which transient terms are differential operators. Here, a locally implicit formulation is investigated, which is based on the ADER philosophy. The scheme is divided in three steps: i) a polynomial reconstruction of the data; ii) solutions to Generalized Riemann Problems (GRP); iii) the solution of differential problems. Note that steps i) and ii), are those of conventional ADER schemes for conservation laws. Advantages of the present approach include the possibility to construct high-order approximations in both space and time, for which existing methodologies for hyperbolic problems can be applied. The differential problems associated to the transient term can be non-linear and numerical strategies can be adopted to deal wit it. Convergence of the scheme is proved rigorously and an empirical convergence rates assessment is carried out in order to illustrate the high space and time accuracy of the present scheme.

Implicit, semi-analytical solution of the generalized Riemann problem for stiff hyperbolic balance laws

We present a semi-analytical, implicit solution to the generalized Riemann problem (GRP) for non-linear systems of hyperbolic balance laws with stiff source terms. The solution method is based on an implicit, time Taylor series expansion and the Cauchy–Kowalewskaya procedure, along with the solution of a sequence of classical Riemann problems. Our new GRP solver is then used to construct locally implicit ADER methods of arbitrary accuracy in space and time for solving the general initial–boundary value problem for non-linear systems of hyperbolic balance laws with stiff source terms. Analysis of the method for model problems is carried out and empirical convergence rate studies for suitable tests problems are performed, confirming the theoretically expected high order of accuracy.

Error analysis of nonconforming and mixed FEMs for second-order linear non-selfadjoint and indefinite elliptic problems

The state-of-the art proof of a global inf-sup condition on mixed finite element schemes does not allow for an analysis of truly indefinite, second-order linear elliptic PDEs. This paper, therefore, first analyses a nonconforming finite element discretization which converges owing to some a priori$$L^2$$L2 error estimates even for reduced regularity on non-convex polygonal domains. An equivalence result of that nonconforming finite element scheme to the mixed finite element method (MFEM) leads to the well-posedness of the discrete solution and to a priori error estimates for the MFEM. The explicit residual-based a posteriori error analysis allows some reliable and efficient error control and motivates some adaptive discretization which improves the empirical convergence rates in three computational benchmarks.

High-Order Finite-Volume Transport on the Cubed Sphere: Comparison between 1D and 2D Reconstruction Schemes

Abstract This paper presents two finite-volume (FV) schemes for solving linear transport problems on the cubed-sphere grid system. The schemes are based on the central-upwind finite-volume (CUFV) method, which is a class of Godunov-type method for solving hyperbolic conservation laws, and combines the attractive features of the classical upwind and central FV methods. One of the CUFV schemes is based on a dimension-by-dimension approach and employs a fifth-order one-dimensional (1D) Weighted Essentially Nonoscillatory (WENO5) reconstruction method. The other scheme employs a fully two-dimensional (2D) fourth-order accurate reconstruction method. The cubed-sphere grid system imposes several computational challenges due to its patched-domain topology and nonorthogonal curvilinear grid structure. A high-order 1D interpolation procedure combining cubic and quadratic interpolations is developed for the FV schemes to handle the discontinuous edges of the cubed-sphere grid. The WENO5 scheme is compared against the fourth-order Kurganov–Levy (KL) scheme formulated in the CUFV framework. The performance of the schemes is compared using several benchmark problems such as the solid-body rotation and deformational-flow tests, and empirical convergence rates are reported. In addition, a bound-preserving filter combined with an optional positivity-preserving filter is tested for nonsmooth problems. The filtering techniques considered are local, inexpensive, and effective. A fourth-order strong stability preserving explicit Runge–Kutta time-stepping scheme is used for integration. The results show that schemes are competitive to other published FV schemes in the same category.

Penalization of a stochastic variational inequality modeling an elasto-plastic problem with noise

In a recent work of A.Bensoussan and J.Turi, it has been shown that the solution of a stochastic variational inequality modeling an elasto-plastic oscillator excited by a white noise has a unique invariant probability measure. The latter is useful for engineering in order to evaluate statistics of plastic deformations for large times of a certain type of mechanical structure. However, in terms of mathematics, not much is known about its regularity properties. From then on, an interesting mathematical question is to determine them. Therefore, in order to investigate this question, we introduce in this paper approximate solutions of the stochastic variational inequality by a penalization method. The idea is simple: the inequality is replaced by an equation with a nonlinear additional term depending on a parameter n penalizing the solution whenever it goes beyond a prespeci ed area. In this context, the dynamics is smoother. In a rst part, we show that the penalized process converges towards the original solution of the aforementioned inequality on any finite time interval as n goes to infinity. Then, in a second part, we justify that for each n it has at least one invariant probability measure. We conjecture that it is unique, but unfortunately we are not able to prove it. Finally, we provide numerical experiments in support of our conjecture. Moreover, we give an empirical convergence rate of the sequence of measures related to the penalized process.

Fast X-ray CT image reconstruction using a linearized augmented Lagrangian method with ordered subsets.

Augmented Lagrangian (AL) methods for solving convex optimization problems with linear constraints are attractive for imaging applications with composite cost functions due to the empirical fast convergence rate under weak conditions. However, for problems such as X-ray computed tomography (CT) image reconstruction, where the inner least-squares problem is challenging and requires iterations, AL methods can be slow. This paper focuses on solving regularized (weighted) least-squares problems using a linearized variant of AL methods that replaces the quadratic AL penalty term in the scaled augmented Lagrangian with its separable quadratic surrogate function, leading to a simpler ordered-subsets (OS) accelerable splitting-based algorithm, OS-LALM. To further accelerate the proposed algorithm, we use a second-order recursive system analysis to design a deterministic downward continuation approach that avoids tedious parameter tuning and provides fast convergence. Experimental results show that the proposed algorithm significantly accelerates the convergence of X-ray CT image reconstruction with negligible overhead and can reduce OS artifacts when using many subsets.

Hyperbolic reformulation of a 1D viscoelastic blood flow model and ADER finite volume schemes

The applicability of ADER finite volume methods to solve hyperbolic balance laws with stiff source terms in the context of well-balanced and non-conservative schemes is extended to solve a one-dimensional blood flow model for viscoelastic vessels, reformulated as a hyperbolic system, via a relaxation time. A criterion for selecting relaxation times is found and an empirical convergence rate assessment is carried out to support this result. The proposed methodology is validated by applying it to a network of viscoelastic vessels for which experimental and numerical results are available. The agreement between the results obtained in the present paper and those available in the literature is satisfactory. Key features of the present formulation and numerical methodologies, such as accuracy, efficiency and robustness, are fully discussed in the paper.

Well‐balanced high‐order solver for blood flow in networks of vessels with variable properties

We present a well-balanced, high-order non-linear numerical scheme for solving a hyperbolic system that models one-dimensional flow in blood vessels with variable mechanical and geometrical properties along their length. Using a suitable set of test problems with exact solution, we rigorously assess the performance of the scheme. In particular, we assess the well-balanced property and the effective order of accuracy through an empirical convergence rate study. Schemes of up to fifth order of accuracy in both space and time are implemented and assessed. The numerical methodology is then extended to realistic networks of elastic vessels and is validated against published state-of-the-art numerical solutions and experimental measurements. It is envisaged that the present scheme will constitute the building block for a closed, global model for the human circulation system involving arteries, veins, capillaries and cerebrospinal fluid.

Wavelet-Based Estimation of Anisotropic Spatiotemporal Long-Range Dependence

In this article, the estimation of spatiotemporal long-range dependence is formulated in the spectral wavelet domain. Sample information is provided by functional spectral data. Their high local singularity at the origin is captured by the wavelet transform. Weak consistency of the spectral wavelet estimators proposed is derived. Two functional estimation algorithms are implemented. A simulation study is developed to illustrate the efficiency of the computational methods derived. An approximation to the empirical convergence rate of the spectral wavelet periodogram is computed in some simulated examples.