Next generation full-wave ultrasound simulation: Exploiting parallelism in the move toward exascale

Abstract

In recent years, the capability of performing 3D full-wave nonlinear ultrasound simulations in tissue-realistic media has been demonstrated by a number of groups. These models have since underpinned a range of interesting studies into the interaction of ultrasound fields with the human body. However, simulations have thus far been limited to domain sizes on the order of 1 billion grid points. This limitation usually means either the spatial area or highest frequency of interest is truncated. Looking toward the exascale era, where supercomputers integrating over 1M cores are predicted to appear before 2020, there is a significant opportunity to use models to gain new insight into ultrasound-tissue interaction with unprecedented detail. The challenge is to develop suitable numerical methods that map to these massively parallel architectures, in particular, that minimize data movement, allow the exploitation of co-processors, and minimize the accumulation of numerical errors. Here, we show a novel domain decomposition approach for the Fourier collocation spectral method that allows ultrasound simulations to be distributed across up to 32k CPU cores or hundreds of accelerators with reasonable efficiency. Using this model, we demonstrate for the first time the possibility of ultrasound simulations exceeding 70 billion grid points.