Multilevel Monte Carlo Method Research Articles

Complexity analysis of quasi continuous level Monte Carlo

Continuous level Monte Carlo is an unbiased, continuous version of the celebrated multilevel Monte Carlo method. The approximation level is assumed to be continuous resulting in a stochastic process describing the quantity of interest. Continuous level Monte Carlo methods allow naturally for samplewise adaptive mesh refinements, which are indicated by (goal-oriented) error estimators. The samplewise refinement levels are drawn in the estimator from an exponentially-distributed random variable. Unfortunately in practical examples this results in higher costs due to high variance in the samples. In this paper we propose a variant of continuous level Monte Carlo, where a quasi Monte Carlo sequence is utilized to ''sample'' the exponential random variable. We provide a complexity theorem for this novel estimator and show that this theoretically and practically results in a variance reduction of the whole estimator.

Multilevel Monte Carlo Methods for Stochastic Convection–Diffusion Eigenvalue Problems

We develop new multilevel Monte Carlo (MLMC) methods to estimate the expectation of the smallest eigenvalue of a stochastic convection–diffusion operator with random coefficients. The MLMC method is based on a sequence of finite element (FE) discretizations of the eigenvalue problem on a hierarchy of increasingly finer meshes. For the discretized, algebraic eigenproblems we use both the Rayleigh quotient (RQ) iteration and implicitly restarted Arnoldi (IRA), providing an analysis of the cost in each case. By studying the variance on each level and adapting classical FE error bounds to the stochastic setting, we are able to bound the total error of our MLMC estimator and provide a complexity analysis. As expected, the complexity bound for our MLMC estimator is superior to plain Monte Carlo. To improve the efficiency of the MLMC further, we exploit the hierarchy of meshes and use coarser approximations as starting values for the eigensolvers on finer ones. To improve the stability of the MLMC method for convection-dominated problems, we employ two additional strategies. First, we consider the streamline upwind Petrov–Galerkin formulation of the discrete eigenvalue problem, which allows us to start the MLMC method on coarser meshes than is possible with standard FEs. Second, we apply a homotopy method to add stability to the eigensolver for each sample. Finally, we present a multilevel quasi-Monte Carlo method that replaces Monte Carlo with a quasi-Monte Carlo (QMC) rule on each level. Due to the faster convergence of QMC, this improves the overall complexity. We provide detailed numerical results comparing our different strategies to demonstrate the practical feasibility of the MLMC method in different use cases. The results support our complexity analysis and further demonstrate the superiority over plain Monte Carlo in all cases.

Multipolynomial Monte Carlo for trace estimation in lattice QCD

Estimating the trace of the inverse of a large matrix is an important problem in lattice quantum chromodynamics. A multilevel Monte Carlo method is proposed for this problem that uses different degree polynomials for the levels. The polynomials are developed from the GMRES algorithm for solving linear equations. To reduce orthogonalization expense, the highest degree polynomial is a composite or double polynomial found with a polynomial preconditioned GMRES iteration. Added to some of the Monte Carlo pieces is deflation of eigenvalues that reduces the variance. Deflation is also used for finding a reduced degree deflated polynomial. The new Multipolynomial Monte Carlo method can significantly improve the trace computation for matrices that have a difficult spectrum due to small eigenvalues.

Uncertainty quantification in the Henry problem using the multilevel Monte Carlo method

We investigate the applicability of the well-known multilevel Monte Carlo (MLMC) method to the class of density-driven flow problems, in particular the problem of salinisation of coastal aquifers. As a test case, we solve the uncertain Henry saltwater intrusion problem. Unknown porosity, permeability and recharge parameters are modelled by using random fields. The classical deterministic Henry problem is non-linear and time-dependent, and can easily take several hours of computing time. Uncertain settings require the solution of multiple realisations of the deterministic problem, and the total computational cost increases drastically. Instead of computing of hundreds random realisations, typically the mean value and the variance are computed. The standard methods such as the Monte Carlo or surrogate-based methods are a good choice, but they compute all stochastic realisations on the same, often, very fine mesh. They also do not balance the stochastic and discretisation errors. These facts motivated us to apply the MLMC method. We demonstrate that by solving the Henry problem on multi-level spatial and temporal meshes, the MLMC method reduces the overall computational and storage costs. To reduce the computing cost further, parallelization is performed in both physical and stochastic spaces. To solve each deterministic scenario, we run the parallel multigrid solver ug4 in a black-box fashion.

Multilevel Monte Carlo methods for the Grad-Shafranov free boundary problem

The equilibrium configuration of a plasma in an axially symmetric reactor is described mathematically by a free boundary problem associated with the celebrated Grad-Shafranov equation. The presence of uncertainty in the model parameters introduces the need to quantify the variability in the predictions. This is often done by computing a large number of model solutions on a computational grid for an ensemble of parameter values and then obtaining estimates for the statistical properties of solutions. In this study, we explore the savings that can be obtained using multilevel Monte Carlo methods, which reduce costs by performing the bulk of the computations on a sequence of spatial grids that are coarser than the one that would typically be used for a simple Monte Carlo simulation. We examine this approach using both a set of uniformly refined grids and a set of adaptively refined grids guided by a discrete error estimator. Numerical experiments show that multilevel methods dramatically reduce the cost of simulation, with cost reductions typically on the order of 60 or more and possibly as large as 200. Adaptive griding results in more accurate computation of geometric quantities such as x-points associated with the model.

Efficient risk estimation via nested multilevel quasi-Monte Carlo simulation

We consider the problem of estimating the probability of a large loss from a financial portfolio, where the future loss is expressed as a conditional expectation. Since the conditional expectation is intractable in most cases, one may resort to nested simulation. To reduce the complexity of nested simulation, we present an improved multilevel Monte Carlo (MLMC) method by using quasi-Monte Carlo (QMC) to estimate the portfolio loss in each financial scenario generated via Monte Carlo. We prove that using QMC can accelerate the convergence rates in both the crude nested simulation and the multilevel nested simulation. Under certain conditions, the complexity of the proposed MLMC method can be reduced to O(ϵ−2(logϵ)2). On the other hand, we find that using QMC in MLMC encounters a high-kurtosis phenomenon due to the existence of indicator functions. To remedy this, we propose a smoothed method which uses logistic sigmoid functions to approximate indicator functions. Numerical results show that the optimal MLMC complexity O(ϵ−2) is almost attained even in moderate high dimensions.

MULTILEVEL MONTE CARLO ESTIMATORS FOR DERIVATIVE-FREE OPTIMIZATION UNDER UNCERTAINTY

Optimization is a key tool for scientific and engineering applications; however, in the presence of models affected by uncertainty, the optimization formulation needs to be extended to consider statistics of the quantity of interest. Optimization under uncertainty (OUU) deals with this endeavor and requires uncertainty quantification analyses at several design locations; i.e., its overall computational cost is proportional to the cost of performing a forward uncertainty analysis at each design location. An OUU workflow has two main components: an inner loop strategy for the computation of statistics of the quantity of interest, and an outer loop optimization strategy tasked with finding the optimal design, given a merit function based on the inner loop statistics. In this work, we propose to alleviate the cost of the inner loop uncertainty analysis by leveraging the so-called multilevel Monte Carlo (MLMC) method, which is able to allocate resources over multiple models with varying accuracy and cost. The resource allocation problem in MLMC is formulated by minimizing the computational cost given a target variance for the estimator. We consider MLMC estimators for statistics usually employed in OUU workflows and solve the corresponding allocation problem. For the outer loop, we consider a derivative-free optimization strategy implemented in the SNOWPAC library; our novel strategy is implemented and released in the Dakota software toolkit. We discuss several numerical test cases to showcase the features and performance of our approach with respect to its Monte Carlo single fidelity counterpart.

Monte Carlo convergence rates for kth moments in Banach spaces

We formulate standard and multilevel Monte Carlo methods for the kth moment Mεk[ξ] of a Banach space valued random variable ξ:Ω→E, interpreted as an element of the k-fold injective tensor product space ⊗εkE. For the standard Monte Carlo estimator of Mεk[ξ], we prove the k-independent convergence rate 1−1p in the Lq(Ω;⊗εkE)-norm, provided that (i) ξ∈Lkq(Ω;E) and (ii) q∈[p,∞), where p∈[1,2] is the Rademacher type of E. By using the fact that Rademacher averages are dominated by Gaussian sums combined with a version of Slepian's inequality for Gaussian processes due to Fernique, we moreover derive corresponding results for multilevel Monte Carlo methods, including a rigorous error estimate in the Lq(Ω;⊗εkE)-norm and the optimization of the computational cost for a given accuracy. Whenever the type of the Banach space E is p=2, our findings coincide with known results for Hilbert space valued random variables.We illustrate the abstract results by three model problems: second-order elliptic PDEs with random forcing or random coefficient, and stochastic evolution equations. In these cases, the solution processes naturally take values in non-Hilbertian Banach spaces. Further applications, where physical modeling constraints impose a setting in Banach spaces of type p<2, are indicated.

Gradient-based optimisation of the conditional-value-at-risk using the multi-level Monte Carlo method

In this work, we tackle the problem of minimising the Conditional-Value-at-Risk (CVaR) of output quantities of complex differential models with random input data, using gradient-based approaches in combination with the Multi-Level Monte Carlo (MLMC) method. In particular, we consider the framework of multi-level Monte Carlo for parametric expectations introduced in [24] and propose modifications of the MLMC estimator, error estimation procedure, and adaptive MLMC parameter selection to ensure the estimation of the CVaR and sensitivities for a given design with a prescribed accuracy. We then propose combining the MLMC framework with an alternating inexact minimisation-gradient descent algorithm, for which we prove Q-linear convergence in the optimisation iterations under the assumptions of strong convexity and Lipschitz continuity of the gradient of the objective function. We demonstrate the performance of our approach on two numerical examples of practical relevance, which evidence the same optimal asymptotic cost-tolerance behaviour as standard MLMC methods for fixed design computations of output expectations.

Approximating Inverse Cumulative Distribution Functions to Produce Approximate Random Variables

For random variables produced through the inverse transform method, approximate random variables are introduced, which are produced using approximations to a distribution’s inverse cumulative distribution function. These approximations are designed to be computationally inexpensive and much cheaper than library functions, which are exact to within machine precision and, thus, highly suitable for use in Monte Carlo simulations. The approximation errors they introduce can then be eliminated through use of the multilevel Monte Carlo method. Two approximations are presented for the Gaussian distribution: a piecewise constant on equally spaced intervals and a piecewise linear using geometrically decaying intervals. The errors of the approximations are bounded and the convergence demonstrated, and the computational savings are measured for C and C++ implementations. Implementations tailored for Intel and Arm hardware are inspected alongside hardware agnostic implementations built using OpenMP. The savings are incorporated into a nested multilevel Monte Carlo framework with the Euler-Maruyama scheme to exploit the speedups without losing accuracy, offering speed ups by a factor of 5–7. These ideas are empirically extended to the Milstein scheme and the non-central χ 2 distribution for the Cox-Ingersoll-Ross process, offering speedups of a factor of 250 or more.

Uncertainty quantification in coastal aquifers using the multilevel Monte Carlo method

AbstractWe are solving a problem of salinisation of coastal aquifers. As a test case example, we consider the Henry saltwater intrusion problem. Since porosity, permeability and recharge are unknown or only known at a few points, we model them using random fields and random variables. The Henry problem describes a two‐phase flow and is non‐linear and time‐dependent. The solution to be found is the expectation of the salt mass fraction, which is uncertain and time‐dependent. To estimate this expectation, we use the well‐known multilevel Monte Carlo (MLMC) method. The MLMC method takes just a few samples on computationally expensive (fine) meshes and more samples on cheap (coarse) meshes. Then, by building a telescoping sum, the MLMC method estimates the expected value at a much lower computational cost than the classical Monte Carlo method. The deterministic solver used here is the well‐known parallel and scalable ug4 solver.

Optimizing shift selection in multilevel Monte Carlo for disconnected diagrams in lattice QCD

The calculation of disconnected diagram contributions to physical signals is a computationally expensive task in Lattice QCD. To extract the physical signal, the trace of the inverse Lattice Dirac operator, a large sparse matrix, must be stochastically estimated. Because the variance of the stochastic estimator is typically large, variance reduction techniques must be employed. Multilevel Monte Carlo (MLMC) methods reduce the variance of the trace estimator by utilizing a telescoping sequence of estimators. Frequency Splitting is one such method that uses a sequence of inverses of shifted operators to estimate the trace of the inverse lattice Dirac operator, however there is no a priori way to select the shifts that minimize the cost of the multilevel trace estimation. In this article, we present a sampling and interpolation scheme that is able to predict the variances associated with Frequency Splitting under displacements of the underlying space time lattice. The interpolation scheme is able to predict the variances to high accuracy and therefore chooses shifts that correspond to an approximate minimum of the cost for the trace estimation. We show that Frequency Splitting with the chosen shifts displays significant speedups over multigrid deflation, and that these shifts can be used for multiple configurations within the same ensemble with no penalty to performance.

A Simple, Bias-free Approximation of Covariance Functions by the Multilevel Monte Carlo Method Having Nearly Optimal Complexity

A Simple, Bias-free Approximation of Covariance Functions by the Multilevel Monte Carlo Method Having Nearly Optimal Complexity

Reliability analysis of the stress intensity factor using multilevel Monte Carlo methods

This paper presents a reliability analysis for fracture toughness using a multilevel refinement on a hierarchy of computational models. A 2D finite element model discretized by quadrilateral elements is developed to analyze the stress intensity with the presence of an initial edge crack. The multilevel simulations are obtained considering a non-uniform sequence of mesh refinement in the vicinity of the crack tip. We set the probabilistic problem accounting for applied stress and crack size uncertainties. We analyze several error tolerances using the standard and multilevel Monte Carlo methods combined with the selective refinement procedure. The probability of failure is estimated by expanding it in a telescoping sum of an initial approximation at the coarsest mesh and a series of incremental corrections between the subsequent levels. In our analysis, we take on two common fracture problems; a single-edge notched tension to investigate the pure mode-I and an asymmetric four-points bending to consider the mixed mode-I/II. The results show significant savings in the computation cost.

On the rate of convergence of an exponential scheme for the non-linear stochastic Schrödinger equation with finite-dimensional state space

We address the numerical solution of the finite-dimensional non-linear stochastic Schrödinger equation, which is a locally Lipschitz stochastic differential equation modeling, for instance, quantum measurement processes. We study the rate of weak convergence of an exponential scheme that reproduces the norm of the desired solution by projecting onto the unit sphere. This justifies the use of the Talay-Tubaro extrapolation procedure in the numerical simulation of open quantum systems. In particular, we prove that an Euler-Exponential scheme converges with weak-order one, and we obtain the leading order term of its weak error expansion with respect to the step-size. Then, applying the Talay-Tubaro extrapolation procedure to the Euler-Exponential scheme under consideration we get a second-order method for computing the mean values of smooth functions of the solution of the non-linear stochastic Schrödinger equation. We also prove that the exponential scheme under study has order of strong convergence 1/2, which gives theoretical support to the use of the multilevel Monte Carlo method in simulating open quantum systems. We present a numerical experiment with a quantized electromagnetic field in interaction with a reservoir that illustrates the good performance of the weak second-order method, and the multilevel Monte Carlo method.

A massively parallel implementation of multilevel Monte Carlo for finite element models

The Multilevel Monte Carlo (MLMC) method has proven to be an effective variance-reduction statistical method for Uncertainty Quantification (UQ) in Partial Differential Equation (PDE) models, combining model computations at different levels to create an accurate estimate. Still, the computational complexity of the resulting method is extremely high, particularly for 3D models, which requires advanced algorithms for the efficient exploitation of High Performance Computing (HPC). In this article we present a new implementation of the MLMC in massively parallel computer architectures, exploiting parallelism within and between each level of the hierarchy. The numerical approximation of the PDE is performed using the finite element method but the algorithm is quite general and could be applied to other discretization methods. The two key ingredients of the implementation are a good processor partition scheme together with a good scheduling algorithm to assign work to different processors. We introduce a multiple partition of the set of processors that permits the simultaneous execution of different levels and we develop a dynamic scheduling algorithm to exploit it. The problem of finding the optimal scheduling of distributed tasks in a parallel computer is an NP-complete problem. We propose and analyze a new greedy scheduling algorithm to assign samples and we show that it is a 2-approximation, which is the best that may be expected under general assumptions. On top of this result we design a distributed memory implementation using the Message Passing Interface (MPI) standard. Finally we present a set of numerical experiments illustrating its scalability properties.

Urban Flood Modeling: Uncertainty Quantification and Physics‐Informed Gaussian Processes Regression Forecasting

AbstractEstimating uncertainty in flood model predictions is important for many applications, including risk assessment and flood forecasting. We focus on uncertainty in physics‐based urban flooding models. We consider the effects of the model's complexity and uncertainty in key input parameters. The effect of rainfall intensity on the uncertainty in water depth predictions is also studied. As a test study, we choose the Interconnected Channel and Pond Routing (ICPR) model of a part of the city of Minneapolis. The uncertainty in the ICPR model's predictions of the floodwater depth is quantified in terms of the ensemble variance using the multilevel Monte Carlo (MC) simulation method. Our results show that uncertainties in the studied domain are highly localized. Model simplifications, such as disregarding the groundwater flow, lead to overly confident predictions, that is, predictions that are both less accurate and uncertain than those of the more complex model. We find that for the same number of uncertain parameters, increasing the model resolution reduces uncertainty in the model predictions (and increases the MC method's computational cost). We employ the multilevel MC method to reduce the cost of estimating uncertainty in a high‐resolution ICPR model. Finally, we use the ensemble estimates of the mean and covariance of the flood depth for real‐time flood depth forecasting using the physics‐informed Gaussian process regression method. We show that even with few measurements, the proposed framework results in a more accurate forecast than that provided by the mean prediction of the ICPR model.

A Multilevel Monte Carlo Approach for a Stochastic Optimal Control Problem Based on the Gradient Projection Method

A multilevel Monte Carlo (MLMC) method is applied to simulate a stochastic optimal problem based on the gradient projection method. In the numerical simulation of the stochastic optimal control problem, the approximation of expected value is involved, and the MLMC method is used to address it. The computational cost of the MLMC method and the convergence analysis of the MLMC gradient projection algorithm are presented. Two numerical examples are carried out to verify the effectiveness of our method.

A multilevel Monte Carlo ensemble and hybridizable discontinuous Galerkin method for a stochastic parabolic problem

AbstractA second‐order, multilevel Monte Carlo ensemble, and hybridizable discontinuous Galerkin (MLMCE‐HDG) method is proposed to solve the stochastic parabolic partial differential equations (SPDEs). By introducing an ensemble average of the diffusion coefficient, the MLMCE‐HDG method results in a single discrete system with multiple right‐hand‐vectors, which can be solved more efficiently than a group of linear systems. A rigorous error estimate is obtained with a second‐order accuracy in time and optimal convergence rate in physical space. Comparing with the multilevel Monte Carlo and hybridizable discontinuous Galerkin (MLMC‐HDG) method, the MLMCE‐HDG method can reduce the computational cost. Finally, we provide several numerical experiments to illustrate the theoretical results.

Quantify uncertainty by estimating the probability density function of the output of interest using MLMC based Bayes method

&lt;p style='text-indent:20px;'&gt;In uncertainty quantification, the quantity of interest is usually the statistics of the space and/or time integration of system solution. In order to reduce the computational cost, a Bayes estimator based on multilevel Monte Carlo (MLMC) is introduced in this paper. The cumulative distribution function of the output of interest, that is, the expectation of the indicator function, is estimated by MLMC method instead of the classic Monte Carlo simulation. Then, combined with the corresponding probability density function, the quantity of interest is obtained by using some specific quadrature rules. In addition, the smoothing of indicator function and Latin hypercube sampling are used to accelerate the reduction of variance. An elliptic stochastic partial differential equation is used to provide a research context for this model. Numerical experiments are performed to verify the advantage of computational reduction and accuracy improvement of our MLMC-Bayes method.&lt;/p&gt;