Parareal Algorithm Research Articles

Investigation of Second Order Taylor Series in the Coarse Operator of Parareal Algorithm for Power System Simulation

Investigation of Second Order Taylor Series in the Coarse Operator of Parareal Algorithm for Power System Simulation

Multilevel Parareal Algorithm with Averaging for Oscillatory Problems

Multilevel Parareal Algorithm with Averaging for Oscillatory Problems

Parareal Neural Networks Emulating a Parallel-in-Time Algorithm.

As deep neural networks (DNNs) become deeper, the training time increases. In this perspective, multi-GPU parallel computing has become a key tool in accelerating the training of DNNs. In this article, we introduce a novel methodology to construct a parallel neural network that can utilize multiple GPUs simultaneously from a given DNN. We observe that layers of DNN can be interpreted as the time steps of a time-dependent problem and can be parallelized by emulating a parallel-in-time algorithm called parareal. The parareal algorithm consists of fine structures which can be implemented in parallel and a coarse structure that gives suitable approximations to the fine structures. By emulating it, the layers of DNN are torn to form a parallel structure, which is connected using a suitable coarse network. We report accelerated and accuracy-preserved results of the proposed methodology applied to VGG-16 and ResNet-1001 on several datasets.

Combining machine-learned and empirical force fields with the parareal algorithm: application to the diffusion of atomistic defects

Combining machine-learned and empirical force fields with the parareal algorithm: application to the diffusion of atomistic defects

Stabilization of parareal algorithms for long-time computation of a class of highly oscillatory Hamiltonian flows using data

Stabilization of parareal algorithms for long-time computation of a class of highly oscillatory Hamiltonian flows using data

Multi-step variant of the parareal algorithm: convergence analysis and numerics

In this paper, we consider the problem of accelerating the numerical simulation of time dependent problems involving a multi-step time scheme by the parareal algorithm. The parareal method is based on combining predictions made by a coarse and cheap propagator, with corrections computed with two propagators: the previous coarse and a precise and expensive one used in a parallel way over the time windows. A multi-step time scheme can potentially bring higher approximation orders than plain one-step methods but the initialisation of each time window needs to be appropriately chosen. Our main contribution is the design and analysis of an algorithm adapted to this type of discretisation without being too much intrusive in the coarse or fine propagators. At convergence, the parareal algorithm provides a solution that coincides with the solution of the fine solver. In the classical version of parareal, the local initial condition of each time window is corrected at every iteration. When the fine and/or coarse propagators is a multi-step time scheme, we need to choose a consistent approximation of the solutions involved in the initialisation of the fine solver at each time windows. Otherwise, the initialisation error will prevent the parareal algorithm to converge towards the solution with fine solver’s accuracy. In this paper, we develop a variant of the algorithm that overcome this obstacle. Thanks to this, the parareal algorithm is more coherent with the underlying time scheme and we recover the properties of the original version. We show both theoretically and numerically that the accuracy and convergence of the multi-step variant of parareal algorithm are preserved when we choose carefully the initialisation of each time window.

Characteristic Analysis of Transformer Loss with Subsynchronous Components by Field-Circuit Coupled Time Periodic FEM

A field-circuit coupled finite element calculation model embedded loss separation theory is proposed, which can effectively deal with the nonlinear core loss of transformer caused by subsynchronous frequency components injection. In order to deal with the increased time period due to the subsynchronous frequency components, and the problem of excessive loss current and too small parallel resistance due to zero-crossing point of voltage, the Parareal algorithm is introduced to solve the model. The algorithm is verified by the experiments of the physical transformer under the conditions of rated fundamental voltage and with subsynchronous components, respectively.

Analysis of a New Krylov subspace enhanced parareal algorithm for time-periodic problems

Analysis of a New Krylov subspace enhanced parareal algorithm for time-periodic problems

Wavelet-based Edge Multiscale Parareal Algorithm for subdiffusion equations with heterogeneous coefficients in a large time domain

We present the Wavelet-based Edge Multiscale Parareal (WEMP) Algorithm, recently proposed in Li and Hu (2021), for efficiently solving subdiffusion equations with heterogeneous coefficients in long time. This algorithm combines the benefits of multiscale methods, which can handle heterogeneity in the spatial domain, and the strength of parareal algorithms for speeding up time evolution problems when sufficient processors are available. Our algorithm overcomes the challenge posed by the nonlocality of the fractional derivative in previous parabolic problem work by constructing an auxiliary problem on each coarse temporal subdomain to completely uncouple the temporal variable. We prove the approximation properties of the correction operator and derive a new summation of exponential to generate a single-step time stepping scheme, with the number of terms of O(|logτf|2) independent of the final time, where τf is the fine-scale time step size. We establish the convergence rate of our algorithm in terms of the mesh size in the spatial domain, the level parameter used in the multiscale method, the coarse-scale time step size, and the fine-scale time step size. Finally, we present several numerical tests that demonstrate the effectiveness of our algorithm and validate our theoretical results.

A posteriori error analysis for a space‐time parallel discretization of parabolic partial differential equations

AbstractWe construct a space‐time parallel method for solving parabolic partial differential equations by coupling the parareal algorithm in time with overlapping domain decomposition in space. The goal is to obtain a discretization consisting of “local” problems that can be solved on parallel computers efficiently. However, this introduces significant sources of error that must be evaluated. Reformulating the original parareal algorithm as a variational method and implementing a finite element discretization in space enables an adjoint‐based a posteriori error analysis to be performed. Through an appropriate choice of adjoint problems and residuals the error analysis distinguishes between errors arising due to the temporal and spatial discretizations, as well as between the errors arising due to incomplete parareal iterations and incomplete iterations of the domain decomposition solver. We first develop an error analysis for the parareal method applied to parabolic partial differential equations, and then refine this analysis to the case where the associated spatial problems are solved using overlapping domain decomposition. These constitute our time parallel algorithm and space‐time parallel algorithm respectively. Numerical experiments demonstrate the accuracy of the estimator for both algorithms and the iterations between distinct components of the error.

Towards parallel time-stepping for the numerical simulation of atherosclerotic plaque growth

The numerical simulation of atherosclerotic plaque growth is computationally prohibitive, since it involves a complex cardiovascular fluid-structure interaction (FSI) problem with a characteristic time scale of milliseconds to seconds, as well as a plaque growth process governed by reaction-diffusion equations, which takes place over several months. In this work we combine a temporal homogenization approach, which separates the problem in computationally expensive FSI problems on a micro scale and a reaction-diffusion problem on the macro scale, with parallel time-stepping algorithms. It has been found in the literature that parallel time-stepping algorithms do not perform well when applied directly to the FSI problem. To circumvent this problem, a parareal algorithm is applied on the macro-scale reaction-diffusion problem instead of the micro-scale FSI problem. We investigate modifications in the coarse propagator of the parareal algorithm, in order to further reduce the number of costly micro problems to be solved. The approaches are tested in detailed numerical investigations based on serial simulations.

Robust Convergence of Parareal Algorithms with Arbitrarily High-Order Fine Propagators

Robust Convergence of Parareal Algorithms with Arbitrarily High-Order Fine Propagators

Low-rank Parareal: a low-rank parallel-in-time integrator

In this work, the Parareal algorithm is applied to evolution problems that admit good low-rank approximations and for which the dynamical low-rank approximation (DLRA) can be used as time stepper. Many discrete integrators for DLRA have recently been proposed, based on splitting the projected vector field or by applying projected Runge–Kutta methods. The cost and accuracy of these methods are mostly governed by the rank chosen for the approximation. These properties are used in a new method, called low-rank Parareal, in order to obtain a time-parallel DLRA solver for evolution problems. The algorithm is analyzed on affine linear problems and the results are illustrated numerically.

A parareal algorithm for a highly oscillating Vlasov-Poisson system with reduced models for the coarse solving

A parareal algorithm for a highly oscillating Vlasov-Poisson system with reduced models for the coarse solving

GParareal: a time-parallel ODE solver using Gaussian process emulation

Sequential numerical methods for integrating initial value problems (IVPs) can be prohibitively expensive when high numerical accuracy is required over the entire interval of integration. One remedy is to integrate in a parallel fashion, “predicting” the solution serially using a cheap (coarse) solver and “correcting” these values using an expensive (fine) solver that runs in parallel on a number of temporal subintervals. In this work, we propose a time-parallel algorithm (GParareal) that solves IVPs by modelling the correction term, i.e. the difference between fine and coarse solutions, using a Gaussian process emulator. This approach compares favourably with the classic parareal algorithm and we demonstrate, on a number of IVPs, that GParareal can converge in fewer iterations than parareal, leading to an increase in parallel speed-up. GParareal also manages to locate solutions to certain IVPs where parareal fails and has the additional advantage of being able to use archives of legacy solutions, e.g. solutions from prior runs of the IVP for different initial conditions, to further accelerate convergence of the method — something that existing time-parallel methods do not do.

Analysis of the parareal algorithm for linear parametric differential equations

This paper presents a parareal algorithm with parameterized propagators for linear parametric differential equations over a wide range of parameters. Through transforming the initial value problem into nonparametric ODEs based on Taylor series, we construct the general parameterized fine and coarse propagators for the parareal algorithm in each time subinterval. Furthermore, to accelerate the convergence of the algorithm, the coarse propagator based on the waveform relaxation method is proposed. By analysing the computational complexity of the WR propagator and the general coarse propagator, we find these two propagators are appropriate for the different situations. Finally, the convergence analysis of the parareal algorithm with these propagators is presented and our analysis is illustrated with two numerical experiments.

Numerical wave propagation aided by deep learning

We propose a deep learning approach for wave propagation in media with multiscale wave speed, using a second-order linear wave equation model. We use neural networks to enhance the accuracy of a given inaccurate coarse solver, which under-resolves a class of multiscale wave media and wave fields of interest. Our approach involves generating training data by the given computationally efficient coarse solver and another sufficiently accurate solver, applied to a class of wave media (described by their wave speed profiles) and initial wave fields. We find that the trained neural networks can approximate the nonlinear dependence of the propagation on the wave speed as long as the causality is appropriately sampled in training data. We combine the neural-network-enhanced coarse solver with the parareal algorithm and demonstrate that the coupled approach improves the stability of parareal algorithms for wave propagation and improves the accuracy of the enhanced coarse solvers.

Time Parallel Denoising Algorithm Based on P-M Equation for Real Image

This paper combines the regularized Perona and Malik model with the time parareal algorithm to denoise the real image which has the characteristics of typical Gaussian noise. This method can remove the noise stably and effectively and has high calculation efficiency. For the real color image of 6000 × 4000 pixels, the time cost can be shortened from one minute in serial computing to a few seconds based on our combination method if we do not care about the requirement of computing hardware. This provides some reference for practical engineering application.

Stochastic Parareal: An Application of Probabilistic Methods to Time-Parallelization

Parareal is a well-studied algorithm for numerically integrating systems of time-dependent differential equations by parallelising the temporal domain. Given approximate initial values at each temporal sub-interval, the algorithm locates a solution in a fixed number of iterations using a predictor-corrector, stopping once a tolerance is met. This iterative process combines solutions located by inexpensive (coarse resolution) and expensive (fine resolution) numerical integrators. In this paper, we introduce a stochastic parareal algorithm aimed at accelerating the convergence of the deterministic parareal algorithm. Instead of providing the predictor-corrector with a deterministically located set of initial values, the stochastic algorithm samples initial values from dynamically varying probability distributions in each temporal sub-interval. All samples are then propagated in parallel using the expensive integrator. The set of sampled initial values yielding the most continuous (smoothest) trajectory across consecutive sub-intervals are fed into the predictor-corrector, converging in fewer iterations than the deterministic algorithm with a given probability. The performance of the stochastic algorithm, implemented using various probability distributions, is illustrated on low-dimensional systems of ordinary differential equations (ODEs). We provide numerical evidence that when the number of sampled initial values is large enough, stochastic parareal converges almost certainly in fewer iterations than the deterministic algorithm, maintaining solution accuracy. Given its stochastic nature, we also highlight that multiple simulations of stochastic parareal return a distribution of solutions that can represent a measure of uncertainty over the ODE solution.

Coupling Parareal with Optimized Schwarz Waveform Relaxation for Parabolic Problems

We propose and analyse a parallel method, both in the time and space directions, that couples the Parareal algorithm with the Optimized Schwarz waveform relaxation (OSWR) method, with only few OSWR iterations in the fine propagator and with a simple coarse propagator deduced from the Backward Euler method. The analysis of this coupled method is presented for a one-dimensional advection-reaction-diffusion equation. We prove a general convergence result for this method via energy estimates. Numerical results for two-dimensional advection-diffusion problems and for a diffusion equation with strong heterogeneities are presented, to illustrate the performance of the coupled Parareal-OSWR algorithm.