High performance computing aspects of a dimension independent semi-Lagrangian discontinuous Galerkin code

Abstract

The recently developed semi-Lagrangian discontinuous Galerkin approach is used to discretize hyperbolic partial differential equations (usually first order equations). Since these methods are conservative, local in space, and able to limit numerical diffusion, they are considered a promising alternative to more traditional semi-Lagrangian schemes (which are usually based on polynomial or spline interpolation).In this paper, we consider a parallel implementation of a semi-Lagrangian discontinuous Galerkin method for distributed memory systems (so-called clusters). Both strong and weak scaling studies are performed on the Vienna Scientific Cluster 2 (VSC-2). In the case of weak scaling we observe a parallel efficiency above 0.8 for both two and four dimensional problems and up to 8192 cores. Strong scaling results show good scalability to at least 512 cores (we consider problems that can be run on a single processor in reasonable time). In addition, we study the scaling of a two dimensional Vlasov–Poisson solver that is implemented using the framework provided. All of the simulations are conducted in the context of worst case communication overhead; i.e., in a setting where the CFL (Courant–Friedrichs–Lewy) number increases linearly with the problem size.The framework introduced in this paper facilitates a dimension independent implementation of scientific codes (based on C++ templates) using both an MPI and a hybrid approach to parallelization. We describe the essential ingredients of our implementation. Program summaryProgram title: sldgCatalogue identifier: AEZO_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEZO_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 320111No. of bytes in distributed program, including test data, etc.: 5725614Distribution format: tar.gzProgramming language: C++03.Computer: PC, HPC systems.Operating system: POSIX compatible (extensively tested on various Linux systems). Only the timing class requires POSIX routines; all other parts of the program can be run on any system where a C++ compiler, Boost, and FFTW is available.RAM: Megabytes to Terabytes (depending on the problem size)Classification: 6.5, 4.3, 19.8.External routines: Boost, FFTWNature of problem: An approximate solution of the advection equation is computed using the semi-Lagrangian discontinuous Galerkin method. This procedure is used as the fundamental building block in a splitting based Vlasov solver.Solution method: The semi-Lagrangian discontinuous Galerkin approach is employed. For the Vlasov–Poisson problem we, in addition, use a splitting approach in time and the FFT to solve Poisson’s equation.Restrictions: We assume that the hyperbolic part of the problem under consideration can be (for example, using a splitting approach) decomposed into a sequence of one-dimensional advections.Unusual features: The framework is independent of the dimension of the problem under consideration. This is accomplished using template techniques.Running time: From minutes to weeks (depending on the problem size).