Time-parallel Method Research Articles

On the convergence of linear and nonlinear Parareal methods for the Cahn–Hilliard equation

This paper introduces, analyses, and implements efficient time parallel methods for solving the Cahn–Hilliard (CH) equation. Efficient numerical methods for the CH equation are crucial due to its wide range of applications. In particular, simulating the CH equation often requires long computational times to obtain the solution during the phase coarsening stage. Therefore, there is a need to accelerate the computations using parallel methods in the time dimension. We propose linear and nonlinear Parareal methods for the CH equation, depending on the choice of the fine approximation. The effectiveness of our approach is demonstrated through numerical experiments.

High-order space–time parallel computing of the Navier–Stokes equations

Space–time parallel methods can leverage modern parallel computing architectures to further accelerate numerical simulations where parallelizing only in space limits concurrency. In this paper, we develop a high-order space–time parallel computing method for solving the Navier–Stokes equations. The method is based on revisionist integral deferred correction (RIDC) algorithm proposed by Christlieb, Macdonald and Ong [SIAM J. Sci. Comput., 32 (2010), pp. 818-835]. This parallel-in-time algorithm yields high-order accurate results within a wall-clock time comparable to that of a forward or backward Euler simulation. We extend the RIDC algorithm to a space–time parallel method and apply the method to a compressible fluid solver using the message passing interface (MPI) framework. The global communicator is split into space and time communicators to manage parallel computing in spatial and temporal domains, respectively. Accuracy tests show that the method is high-order accurate. Numerical results encompassing both smooth flows and flows with discontinuities demonstrate that the proposed method can accelerate the computation robustly. Additionally, a three-dimensional space–time parallel computation of flow around a sphere is performed. The results indicate that RIDC has the potential to be used in the field of computational fluid dynamics.

Diagonalization based Parallel-in-Time method for a class of fourth order time dependent PDEs

In this paper, we design, analyze and implement efficient time parallel methods for a class of fourth order time-dependent partial differential equations (PDEs), namely the biharmonic heat equation, the linearized Cahn–Hilliard (CH) equation and the nonlinear CH equation. We use a diagonalization technique on all-at-once system to develop efficient iterative time parallel methods for investigating the solution behaviour of the said equations. We present the convergence analysis of Parallel-in-Time (PinT) algorithms. We verify our findings by presenting numerical results.

Time parallelism and Newton-adaptivity of the two-derivative deferred correction discontinuous Galerkin method

In this work, we consider a high-order discretization of compressible viscous flows allowing parallelization both in space and time.The discontinuous Galerkin spectral element method, which is well-suited for massively parallel simulations, is used for spatial discretization. The main novelty in this work is the additional demonstration of time-parallel capabilities within an implicit two-derivative timestepping procedure to further increase the parallel speedup. Temporal parallelism is made possible by a predictor-corrector-type time discretization that allows to split the associated workload onto multiple processors.We identify a homogeneous load balance with respect to the linear (GMRES) iterations on each processor as a key for parallel efficiency. To homogenize the load and to enable practical simulations, an adaptive strategy for Newton’s method is introduced. It is shown that the time-parallel method provides a parallel efficiency of approx. 60−70% on 4−7 computational partitions. Moreover, the capabilities of the novel method for the simulation of large-scale problems are illustrated with a mixed temporal and spatial parallelization on more than 1000 processors.

GParareal: a time-parallel ODE solver using Gaussian process emulation

Sequential numerical methods for integrating initial value problems (IVPs) can be prohibitively expensive when high numerical accuracy is required over the entire interval of integration. One remedy is to integrate in a parallel fashion, “predicting” the solution serially using a cheap (coarse) solver and “correcting” these values using an expensive (fine) solver that runs in parallel on a number of temporal subintervals. In this work, we propose a time-parallel algorithm (GParareal) that solves IVPs by modelling the correction term, i.e. the difference between fine and coarse solutions, using a Gaussian process emulator. This approach compares favourably with the classic parareal algorithm and we demonstrate, on a number of IVPs, that GParareal can converge in fewer iterations than parareal, leading to an increase in parallel speed-up. GParareal also manages to locate solutions to certain IVPs where parareal fails and has the additional advantage of being able to use archives of legacy solutions, e.g. solutions from prior runs of the IVP for different initial conditions, to further accelerate convergence of the method — something that existing time-parallel methods do not do.

A parallel in time/spectral collocation combined with finite difference method for the time fractional differential equations

In this article, we consider the numerical solution for the time fractional differential equations (TFDEs). We propose a parallel in time method, combined with a spectral collocation scheme and the finite difference scheme for the TFDEs. The parallel in time method follows the same sprit as the domain decomposition that consists in breaking the domain of computation into subdomains and solving iteratively the sub-problems over each subdomain in a parallel way. Concretely, the iterative scheme falls in the category of the predictor-corrector scheme, where the predictor is solved by finite difference method in a sequential way, while the corrector is solved by computing the difference between spectral collocation and finite difference method in a parallel way. The solution of the iterative method converges to the solution of the spectral method with high accuracy. Some numerical tests are performed to confirm the efficiency of the method in three areas: (i) convergence behaviors with respect to the discretization parameters are tested; (ii) the overall CPU time in parallel machine is compared with that for solving the original problem by spectral method in a single processor; (iii) for the fixed precision, while the parallel elements grow larger, the iteration number of the parallel method always keep constant, which plays the key role in the efficiency of the time parallel method.

Applications of time parallelization

This review article serves to summarize the many advances in time-parallel computations since the excellent review article by Gander, “50 years of Time Parallel Integration” (Gander, in: 50 years of time parallel time integration. Multiple shooting and time domain decomposition, Springer, Berlin, 2015). We focus, when possible, on applications of time parallelism and the observed speedup and efficiency, highlighting the challenges and benefits of parallel time computations. The applications covered range from numerous PDE-based simulations (both hyperbolic and parabolic), to PDE-constrained optimization, powergrid simulations, and machine learning. The time-parallel methods covered range from various iterative schemes (multigrid, waveform, multiple shooting, domain decomposition) to direct time-parallel methods.

Asynchronous time-parallel method based on Laplace transform

Laplace transform method has proved to be very efficient and easy to parallelize for the solution of time-dependent problems. However, the synchronization delay among processors implies an upper bound on the expectable acceleration factor, which leads to a lot of wasted time. In this paper, we propose an original asynchronous Laplace transform method formalized for quasilinear problems based on the well-known Gaver–Stehfest algorithm. Parallel experiments show the convergence of our new method, as well as several interesting properties compared with the classical algorithms.

An Approximate Time-Parallel Method for the Fast and Accurate Computation of Particle Trajectories in a Magnetic Field*

A temporal multiscale hybridization method is presented that carefully couples coarse scale gyrokinetic models with exact charged particle solution trajectories (that is, with full phase information) in a magnetic field. The approach is based on the careful approximation of a sum, generally employed for time-parallel (TP) computing applications. While the hybridization method presented is highly parallelizable, a computational efficiency gain is seen from considering serial computations only. A complete numerical method is only presented for the aforementioned charged particle application, however, the general approach depicted likely has relevance to a wide swath of challenging multiscale/multiphysics problems. Additionally, the approach has obvious relevance to TP computing applications (such as variable selection on which to perform TP calculations and fine scale sampling strategies).

Parareal in Time for Fast Power System Dynamic Simulations

Recent advancements in high-performance parallel computing platforms and parallel algorithms have significantly enhanced the opportunities for real-time power system protection and control. This paper investigates application of Parareal in time algorithm for fast dynamic simulations. Parareal algorithm belongs to the class of temporal decomposition methods which divide the time interval into sub-intervals and solve them concurrently. Time-parallel algorithms face the difficulty of providing correct initial conditions for all the sub-intervals which impact the convergence rates. Parareal overcomes this difficulty by using an approximate trajectory. It has become popular in recent years for long transient simulations (e.g., molecular dynamics, fusion, reacting flows). This paper presents an approach for reliable implementation of Parareal with detailed models of power systems including saturation. Windowing approach is proposed for improving the convergence. Parareal is compared with the Newton-based time-parallel method. Effectiveness of the algorithm is analyzed by parallel emulation using extensive case studies on 10-generator 39-bus system and 327-generator 2383-bus system for various disturbances. Parareal with simulation windows of 1 s have shown convergence in 1 to 3 iterations for majority of the simulated cases, irrespective of the size of the system and nature of the disturbance. All the cases tested have converged with the proposed implementation.

A stencil-based implementation of Parareal in the C++ domain specific embedded language STELLA

In view of the rapid rise of the number of cores in modern supercomputers, time-parallel methods that introduce concurrency along the temporal axis are becoming increasingly popular. For the solution of time-dependent partial differential equations, these methods can add another direction for concurrency on top of spatial parallelization. The paper presents an implementation of the time-parallel Parareal method in a C++ domain specific language for stencil computations (STELLA). STELLA provides both an OpenMP and a CUDA backend for a shared memory parallelization, using the CPU or GPU inside a node for the spatial stencils. Here, we intertwine this node-wise spatial parallelism with the time-parallel Parareal. This is done by adding an MPI-based implementation of Parareal, which allows us to parallelize in time across nodes. The performance of Parareal with both backends is analyzed in terms of speedup, parallel efficiency and energy-to-solution for an advection–diffusion problem with a time-dependent diffusion coefficient.

Parallel Time Integration with Multigrid

We consider optimal-scaling multigrid solvers for the linear systems that arise from the discretization of problems with evolutionary behavior. Typically, solution algorithms for evolution equations are based on a time-marching approach, solving sequentially for one time step after the other. Parallelism in these traditional time-integration techniques is limited to spatial parallelism. However, current trends in computer architectures are leading toward systems with more, but not faster, processors. Therefore, faster compute speeds must come from greater parallelism. One approach to achieving parallelism in time is with multigrid, but extending classical multigrid methods for elliptic operators to this setting is not straightforward. In this paper, we present a nonintrusive, optimal-scaling time-parallel method based on multigrid reduction (MGR). We demonstrate optimality of our multigrid-reduction-in-time algorithm (MGRIT) for solving diffusion equations in two and three space dimensions in numerical ex...

Time-parallel multiscale/multiphysics framework

We introduce the time-parallel compound wavelet matrix method (tpCWM) for modeling the temporal evolution of multiscale and multiphysics systems. The method couples time parallel (TP) and CWM methods operating at different spatial and temporal scales. We demonstrate the efficiency of our approach on two examples: a chemical reaction kinetic system and a non-linear predator–prey system. Our results indicate that the tpCWM technique is capable of accelerating time-to-solution by 2–3-orders of magnitude and is amenable to efficient parallel implementation.

Listening to catalysis—a real time parallel method for high throughput product analysis

In this paper, we describe the development of two parallel real time detection systems for the analysis of the products of a heterogeneously catalyzed gas phase reaction based on the photoacoustic effect. We developed a process addressing the problem of automated catalyst preparation and prepared bulk metal oxides with high surface areas by an automated, parallelized synthesis based on activated carbon. To evaluate analytical systems and catalytic activity of our materials we performed catalytical tests in CO oxidation and ODH of ethane.

A time-parallel multigrid-extrapolation method for parabolic partial differential equations

We consider the problem of solving unsteady partial differential equations on an MIMD machine. Conventional parallel methods use a data partitioning type approach in which the solution grid at each time-step is divided amongst the available processors. The sequential nature of the time integration is, however, retained. The algorithm presented in this paper makes use of a time-parallel approach, whreby several processors may be employed to solve at several time-steps simultaneously. The time-parallel method enables the inherent parallelism of the extrapolation scheme to be efficiently exploited, allowing a significant increase both in accuracy and in the degree of parallelism. The efficiencies obtained by an implementation on a message-passing multiprocessor demonstrate the suitability of the time-parallel extrapolation method for this type of equation.