Romberg Extrapolation Research Articles

New weak error bounds and expansions for optimal quantization

We propose new weak error bounds and expansion in dimension one for optimal quantization-based cubature formula for different classes of functions, such that piecewise affine functions, Lipschitz convex functions or differentiable function with piecewise-defined locally Lipschitz or α-Hölder derivatives. This new results rest on the local behaviours of optimal quantizers, the Lr-Ls distribution mismatch problem and Zador’s Theorem. This new expansion supports the definition of a Richardson–Romberg extrapolation yielding a better rate of convergence for the cubature formula. An extension of this expansion is then proposed in higher dimension for the first time. We then propose a novel variance reduction method for Monte Carlo estimators, based on one dimensional optimal quantizers.

Bridging the gap between constant step size stochastic gradient descent and Markov chains

We consider the minimization of a strongly convex objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step size. While the detailed analysis was only performed for quadratic functions, we provide an explicit asymptotic expansion of the moments of the averaged SGD iterates that outlines the dependence on initial conditions, the effect of noise and the step size, as well as the lack of convergence in the general (nonquadratic) case. For this analysis we bring tools from Markov chain theory into the analysis of stochastic gradient. We then show that Richardson–Romberg extrapolation may be used to get closer to the global optimum, and we show empirical improvements of the new extrapolation scheme.

Romberg extrapolation for Euler summation-based cubature on regular regions

Romberg extrapolation is a long-known method to improve the convergence rate of the trapezoidal rule on intervals. For simple regions such as the cube [0,1]^q it is directly transferable to cubature in q dimensions. In this paper, we formulate Romberg extrapolation for Euler summation-based cubature on arbitrary q-dimensional regular regions mathcal {G}subset mathbb {R}^q and derive an explicit representation for the remainder term.

Modified Algorithm for Solving Linear Integro-Differential Equations of the Second Kind

In this paper, a modified algorithm is proposed for solving linear integro-differential equations of the second kind. The main idea is based on applying Romberg extrapolation algorithm (REA), on Trapezoidal rule. In accordance with the computational perspective, the comparison has shown that Adomian decomposition approach is more effective to be utilized. The numerical results show that the modified algorithm has been successfully applied to the linear integro-differential equations and the comparisons with some existing methods appeared in the literature reveal that the modified algorithm is more accurate and convenient.

Błędy zaokrągleń w algorytmie Romberga

It is shown that each quadrature computed by Romberg algorithm is exact for slightly perturbed data (computed values of the integrand). For the ordinary summation algorithm the cumulation of rounding errors is proportional to N, the number of quadrature modes. For more elaborate summation the cumulation is proportional to log N. For the binary floating point arithmetic with proper rounding of the sum, the cumulation of errors can be made practically independent of N. In each case the influence of Romberg extrapolation on the cumulation of rounding errors is bounded by a constant.

Weighted multilevel Langevin simulation of invariant measures

We investigate a weighted multilevel Richardson–Romberg extrapolation for the ergodic approximation of invariant distributions of diffusions adapted from the one introduced in [Bernoulli 23 (2017) 2643–2692] for regular Monte Carlo simulation. In a first result, we prove under weak confluence assumptions on the diffusion, that for any integer $R\ge2$, the procedure allows us to attain a rate $n^{\frac{R}{2R+1}}$ whereas the original algorithm convergence is at a weak rate $n^{1/3}$. Furthermore, this is achieved without any explosion of the asymptotic variance. In a second part, under stronger confluence assumptions and with the help of some second-order expansions of the asymptotic error, we go deeper in the study by optimizing the choice of the parameters involved by the method. In particular, for a given $\varepsilon>0$, we exhibit some semi-explicit parameters for which the number of iterations of the Euler scheme required to attain a mean-squared error lower than $\varepsilon^{2}$ is about $\varepsilon^{-2}\log(\varepsilon^{-1})$. Finally, we numerically test this multilevel Langevin estimator on several examples including the simple one-dimensional Ornstein–Uhlenbeck process but also a high dimensional diffusion motivated by a statistical problem. These examples confirm the theoretical efficiency of the method.

A multi-step Richardson–Romberg extrapolation method for stochastic approximation

We obtain an expansion of the implicit weak discretization error for the target of stochastic approximation algorithms introduced and studied in Frikha (2013). This allows us to extend and develop the Richardson–Romberg extrapolation method for Monte Carlo linear estimator (introduced in Talay and Tubaro (1990) and deeply studied in Pagès (2007)) to the framework of stochastic optimization by means of stochastic approximation algorithm. We notably apply the method to the estimation of the quantile of diffusion processes. Numerical results confirm the theoretical analysis and show a significant reduction in the initial computational cost.

Invariant measure of duplicated diffusions and application to Richardson–Romberg extrapolation

With a view to numerical applications we address the following question: given an ergodic Brownian diffusion with a unique invariant distribution, what are the invariant distributions of the duplicated system consisting of two trajectories? We mainly focus on the interesting case where the two trajectories are driven by the same Brownian path. Under this assumption, we first show that uniqueness of the invariant distribution (weak confluence) of the duplicated system is essentially always true in the one-dimensional case. In the multidimensional case, we begin by exhibiting explicit counter-examples. Then, we provide a series of weak confluence criterions (of integral type) and also of a.s. pathwise confluence, depending on the drift and diffusion coefficients through a non-infinitesimal Lyapunov exponent. As examples, we apply our criterions to some non-trivially confluent settings such as classes of gradient systems with non-convex potentials or diffusions where the confluence is generated by the diffusive component. We finally establish that the weak confluence property is connected with an optimal transport problem. As a main application, we apply our results to the optimization of the Richardson-Romberg extrapolation for the numerical approximation of the invariant measure of the initial ergodic Brownian diffusion.

The asymptotic expansion and extrapolation of trapezoidal rule for integrals with fractional order singularities

This paper is aimed at deriving the error asymptotic expansion of trapezoidal rule approximation to integrals for the functions with fractional derivatives or algebraic singularities at some points and from which to design a modified Romberg extrapolation algorithm to effectively compute some singular integrals. Firstly, high-order local fractional derivatives are defined, then a general fractional Taylor's expansion is derived. Secondly, the error asymptotic expansion of trapezoidal rule for these integrals is obtained directly by using the formula of sums of non-integer powers and the general fractional Taylor's expansion. This method is different from the previous work. Thirdly, a modified Romberg extrapolation algorithm is designed to get numerical results efficiently. Finally, numerical examples are presented to verify the correctness of the theoretical analysis and the effectiveness of the extrapolation method.

A refined way of solving reactor point kinetics equations for imposed reactivity insertions

We apply the concept of convergence acceleration, also known as extrapolation, to find the solution of the reactor kinetics equations (RKEs). The method features simplicity in that an approximate finite difference formulation is constructed and converged to high accuracy from knowledge of the error term. Through the Romberg extrapolation, we demonstrate its high accuracy for a variety of imposed reactivity insertions found in the literature. The unique feature of the proposed algorithm, called RKE/R(omberg), is that no special attention is given to the stiffness of the RKEs. Finally, because of its simplicity and accuracy, the RKE/R algorithm is arguably the most efficient numerical solution of the RKEs developed to date.

An application of Romberg extrapolation on quadrature method for solving linear Volterra integral equations of the second kind

The Romberg extrapolation is applied on quadrature method solution of linear Volterra integral equations (VIE) of the second kind. An algorithm for new solution calculated by Romberg method is given. The calculated solutions show more efficiency and accuracy with less computation than solution with quadrature method over more points of integration.

Statistical Romberg extrapolation: A new variance reduction method and applications to option pricing

We study the approximation of Ef(X-T) by a Monte Carlo algorithm, where X is the solution of a stochastic differential equation and f is a given function. We introduce a new variance reduction method, which can be viewed as a statistical analogue of Romberg extrapolation method. Namely, we use two Euler schemes with steps delta and delta(beta), 0 < beta < 1. This leads to an algorithm which, for a given level of the statistical error, has a complexity significantly lower than the complexity of the standard Monte Carlo method. We analyze the asymptotic error of this algorithm in the context of general (possibly degenerate) diffusions. In order to find the optimal beta (which turns out to be beta = 1/2), we establish a central limit: type theorem, based on a result of Jacod and Protter for the asymptotic distribution of the error in the Euler scheme. We test our method on various examples. In particular, we adapt it to Asian options. In this setting, we have a CLT and, as a by-product, an explicit expansion of the discretization error.

Some stochastic particle methods for nonlinear parabolic PDEs

We introduce, on some examples, the main tools to analyze the convergence rate of sto- chastic particle methods for the numerical approximation of some nonlinear parabolic PDEs. We treat the case of Lipschitz coecients as well as the case of viscous scalar conservation laws. We also discuss some particular aspects like the case of a bounded spatial domain or the use of a Romberg extrapolation as a speed up procedure.

The law of the Euler scheme for stochastic differential equations

We study the approximation problem ofE f(X T ) byE f(X ), where (X t ) is the solution of a stochastic differential equation, (X ) is defined by the Euler discretization scheme with stepT/n, andf is a given function. For smoothf's, Talay and Tubaro have shown that the errorE f(X T ) −f(X ) can be expanded in powers of 1/n, which permits to construct Romberg extrapolation precedures to accelerate the convergence rate. Here, we prove that the expansion exists also whenf is only supposed measurable and bounded, under an additional nondegeneracy condition of Hormander type for the infinitesimal generator of (X t ): to obtain this result, we use the stochastic variations calculus. In the second part of this work, we will consider the density of the law ofX and compare it to the density of the law ofX T .

The Euler scheme for stochastic differential equations: error analysis with Malliavin calculus

We study the approximation problem of Ef(X T) by Ef(X T n) , where ( X t ) is the solution of a stochastic differential equation, ( X t n ) is defined by the Euler discretization scheme with step T n , and f is a given function. For smooth f's, Talay and Tubaro had shown that the error Ef(X T) − Ef(X T n) can be expanded in powers of T n , which permits to construct Romberg extrapolation procedures to accelerate the convergence rate. Here, we present our following recent result: the expansion exists also when f is only supposed measurable and bounded, under a nondegeneracy condition (essentially, the Hörmander condition for the infinitesimal generator of ( X t )): this is obtained with Malliavin's calculus. We also get an estimate on the difference between the density of the law of X T and the density of the law of X T n .

Schwinger-Lanczos approach to solving scattering equations

The Lanczos method, well known in linear algebra, is applied to solving Lippmann-Schwinger equation describing scattering of non-relativistic particles with atoms and molecules. It is shown that a combination of the Lanczos technique with an appropriate contracting coordinate transformation and Romberg extrapolation generates a very efficient and accurate method for calculation of various scattering quantities with a very sparse grid. The method yields accurate results for a very low number of meshpoints (typically few tens in electron atom scattering calculation). Detailed test calculations were performed for scattering of electrons with hydrogen atoms and for dissociative attachment of electrons to diatomic molecules.

THE RUNGE-KUTTA CONTRAHARMONIC MEAN METHOD WITH EXTRAPOLATION FOR THE PARALLEL SOLUTION OF O.D.E.'S

In this paper the implementation of the Runge-Kutta Contraharmonic Mean Method using Romberg extrapolation on a parallel computer is presented.

On extrapolation of Jagerman and Stetter rules

In this paper quadrature rules introduced by Jagerman [1] and Stetter [2] are considered and asymptotic expansions for the error given. This allows to make use of the Romberg extrapolation process. Such rules can be viewed as generalizations of the well-known mid-point rule. Thus, numerical examples comparing these rules are finally presented.

Extrapolation technique for solution of scattering integral equations

The Romberg extrapolation technique, widely used for the evaluation of integrals, is generalised and applied to solving integral equations in scattering theory. The proposed approach makes it possible to obtain accurate results with a very small number of mesh points and to control the error of the approximation. Typical examples of the calculation of phase shifts and singularities of the T matrix (i.e. of the energies of bound, virtual and resonance states) are presented.

Method of continued fractions for on- and off-shelltmatrix of local and nonlocal potentials

The method of continued fractions recently proposed by the authors is generalized to an off-shell t-matrix calculation for any nonlocal nonsymmetric interaction. The efficiency of the method is demonstrated for some examples in nuclear physics. The method is not only very efficient, but yields very accurate results when it is combined with the Romberg extrapolation method. A new separable approximation of a potential and the off-shell t matrix is proposed in connection with the method of continued fractions.