Development of an unstructured mesh gyrokinetic particle-in-cell code for exascale fusion plasma simulations on GPUs

Abstract

This paper presents XGCm, a new unstructured mesh gyrokinetic Particle-in-Cell (PIC) code for modeling fusion plasma. The physical models and aspects of the numerical methods employed are the same as those used in the X-point gyrokinetic code, XGC. The core difference is that XGCm builds on an unstructured mesh-centric infrastructure that supports a distributed mesh, making it scalable in both the number of mesh elements and number of particles. A second advantage of an unstructured mesh infrastructure is its performance is not degraded when generally graded or anisotropic meshes are used. The switch from a particle-centric data infrastructure to a distributed mesh-centric infrastructure required the introduction of new methods to execute core PIC operations and substantial modifications to a number of key algorithms from those used in XGC. First, we present the methods and algorithms used in the development of XGCm, which performs all key computing steps on the GPU accelerators. GPU accelerators are providing the main computational power of current generation U.S. Department of Energy supercomputers. Secondly, we perform code validation and test using the circular geometry cyclone base case and realistic DIII-D geometry case, respectively. The turbulence growth rate shows excellent agreement with existing XGC result in the first case, while ion temperature gradient turbulence growth is further demonstrated in the second case. Finally, we present weak scaling results, using up to full system (27,648 GPUs) of the Oak Ridge National Laboratory's Summit supercomputer.