URANOS: A GPU accelerated Navier-Stokes solver for compressible wall-bounded flows

Abstract

URANOS, a massively parallel GPU-accelerated compressible flow solver for high-fidelity modeling of compressible wall flows, is presented. URANOS is based on modern high-fidelity and high-resolution discretization strategies for time-accurate compressible flow predictions. The solver provides six different convective scheme implementations, a cutting-edge method for viscous terms treatment, and three different frameworks for turbulence modeling (DNS, LES, and innovative WMLES), while utilizing a high-order finite-difference approach, ranging from second to sixth order spatial accuracy. This paper details all of these numerical models and implementation issues. Computationally, URANOS combines multiple three-dimensional MPI parallelization strategies with the open standard, OpenACC, for machine wide, on node, and on GPU parallelism, tailoring the software to match the state-of-the-art HPC computing facilities. Special attention is given to the GPU porting and data management, detailing the solver scaling properties and performance for multi-node/-GPU systems on three distinct architectures. In these experiments the different MPI strategies are compared for both GPU-aware MPI/GPU (Host-based MPI) and results are provided proving, consistently, that the GPU-aware MPI outperforms the GPU (Host-based MPI) approach. Furthermore, the GPU version is compared to the CPU only demonstrating over three times speed-up for a node-to-node comparison, even at large scales, with an efficiency of about 80% while using 1024 GPUs. Finally, the manuscript presents several validation benchmarks, from simple academic comparisons to turbulent channel and boundary layer configurations. URANOS is compared with the best DNS data available in the literature across various Mach numbers, ranging from almost incompressible conditions to hypersonic regimes, illustrating the solver's capability in treating a wide range of complex wall-flows problems using DNS, wall-resolved, and wall-modeled LES approaches. Thus, using OpenACC as a paradigm, URANOS provides a straightforward and efficient platform that can efficiently exploit the most modern GPU-accelerated computing architectures in a totally open-source and non-vendor-specific way. The solver represents a framework where the CFD community can explore the limits of modeling, test new models/methods all across a broad range of Mach and Reynolds numbers flows. The solver is released under a BSD license. Program summaryProgram title: Unsteady Robust All-around Navier-StOkes Solver (URANOS)CPC Library link to program files:https://doi.org/10.17632/pw5hshn9k6.1Developer's repository link:https://gitlab.com/fralusa/uranos_gpuLicensing provisions: BSD License 2.0Programming language: Fortran 90, OpenACC, MPINature of problem: Solving the compressible Navier-Stokes system of equations in a three-dimensional Cartesian framework from low to high Mach and Reynolds numbers conditions.Solution method: Convective terms are treated with several high-order high-resolution shock-capturing schemes (WENO/TENO) in hybrid and non-hybrid versions. The system dynamics is advanced in time with a three-stage, third-order total variation diminishing Runge-Kutta method. Three-dimensional MPI parallelization is implemented in several optimized arrangements and coupling OpenACC directives in a genuinely multi-GPU logic.