CPL library — A minimal framework for coupled particle and continuum simulation

Abstract

We present an open-source library for coupling particle codes, such as molecular dynamics (MD) or the discrete element method (DEM), and grid based computational fluid dynamics (CFD). The application is focused on domain decomposition coupling, where a particle and continuum software model different parts of a single simulation domain with information exchange. This focus allows a simple library to be developed, with core mapping and communication handled by just four functions. Emphasis is on scaling on supercomputers, a tested cross-language library, deployment with containers and well-documented simple examples. Building on this core, a template is provided to facilitate the user development of common features for coupling, such as averaging routines and functions to apply constraint forces. The interface code for LAMMPS and OpenFOAM is provided to both include molecular detail in a continuum solver and model fluids flowing through a granular system. Two novel development features are highlighted which will be useful in the development of the next generation of multi-scale software: (i) The division of coupled code into a smaller blocks with testing over a range of processor topologies. (ii) The use of coupled mocking to facilitate coverage of various parts of the code and allow rapid prototyping. These two features aim to help users develop coupled models in a test-driven manner and focus on the physics of the problem instead of just software development. All presented code is open-source with detailed documentation on the dedicated website (cpl-library.org) permitting useful aspects to be evaluated and adopted in other projects. Program summaryProgram Title:CPLLIBRARYProgram Files doi:http://dx.doi.org/10.17632/9dh8w97d2x.1Licensing provisions: GPLv3Programming languages: Fortran/C/C++/PythonExternal routines/libraries: Message Passing Interface (MPI)Nature of problem: Coupling of particle and continuum software to enable simulations not possible with either code alone. In particular, handling communication and interaction of computational fluid dynamics (CFD) software with either molecular dynamics (MD) or discrete element method (DEM) solvers on high performance computing (HPC) platforms.Solution method: A shared library with a minimal set of functions to enable coupling, together with an entire infrastructure to facilitate development of validated coupled software. This includes a minimal Python interface to encourage mock testing, libraries to help develop coupled tools along with pre-coupled examples including OpenFOAM, LAMMPS, Flowmol (Smith, 2014) and interactive plotting using wxPython and matplotlib.Unusual features: Minimal interface with simple setup. A CPL_Mocks framework to facilitate debugging and test driven development. Communication established between independent executables using MPI_Open_Port, reducing required changes to core source code. Coupled codes track the git repository with validation through continuous integration testing and deployment on DockerHub. All communication based on MPI_Cart and MPI_Graph so optimisation is possible through MPI implementation on supercomputers.