Billion degree of freedom granular dynamics simulation on commodity hardware via heterogeneous data-type representation

Abstract

We discuss modeling, algorithmic, and software aspects that allow a simulation tool called Chrono::Granular to run billion-degree-of-freedom dynamics problems on commodity hardware, i.e., a workstation with one GPU. The ability to scale the solution to large problem sizes is traced back to an adimensionalization process combined with the use of mixed-precision data types that reduce memory pressure and improve arithmetic intensity, judicious use of the memory ecosystem on GPU cards as exposed by CUDA on Nvidia architectures, and a software implementation that prioritizes execution speed over modeling generality. The simulation approach is demonstrated for 3D scenarios with up to 710 million bodies for the frictionless case (of relevance in emulsions), and up to 210 million bodies for scenarios with friction (of relevance in terradynamics, additive manufacturing, soft-matter physics). The frictional contact model used draws on the Discrete Element Method (DEM). A performance benchmark shows linear scaling with problem size up to GPU memory capacity. The implementation has an application programming interface that enables it to interact in a cosimulation framework with third-party dynamics engines. This interaction is anchored by a force–displacement data exchange protocol that brings in external bodies as geometries defined by triangle meshes. We demonstrate the cosimulation mechanism by interfacing to an open source, multiphysics simulation engine called Chrono. Therein, triangular meshes define moving boundary conditions for Chrono::Granular, which in turn provides forces and torques acting on the triangular meshes. Several tests are considered for validation and scaling analysis purposes. The limiting aspects of the current implementation are its exclusive support of monodisperse granular systems, and its lack of handling geometries beyond spheres. These limitations are addressed by ongoing work.