Parallel high-order resolution of the Shallow-water equations on real large-scale meshes with complex bathymetries

Abstract

The resolution of the Shallow-water equations is of practical interest in the study of inundations and often requires very large and dense meshes to accurately simulate river flows. Those large meshes are often decomposed into multiple sub-domains to allow for parallel processing. When such a decomposition process is used in the context of distributed parallel computing, each sub-domain requires an exchange of one or more layers of ghost cells at each time step of the simulation due to the spatial dependency of numerical methods. In the first part of this paper, we show how the domain decomposition and ghost-layer generation process can be performed in a parallel manner for large meshes, and show a new way of storing the resulting sub-domains with all their send/receive information within a single CGNS mesh file. The performance of the ghost-layer generation process is studied both in terms of time and memory on 2D and 3D meshes containing up to 70 million cells. In the second part of the paper, the program developed in the first part is used to generate the domain decomposition of large meshes of practical interest in the study of natural free surface flows. We use our in-house multi-CPU multi-GPU (MPI+CUDA) solver to show the impact of multiple layers of ghost cells on the execution times of the first-order HLLC, and second-order WAF and MUSCL methods. Finally, the parallel solver is used to perform the second-order resolution on real large-scale meshes of rivers near Montréal, using up to 32 GPUs on meshes of 13 million cells.