Tests with FALKSOL A massively parallel multi-level domain decomposing direct solver

Abstract

An implementation of a massively parallel domain decomposing direct finite element equation solver named FALKSOL is tested using the parallel version of the state of the art open-source solver MUMPS (MUltifrontal Massively Parallel Solver) as a reference. FALKSOL includes its own very advanced multi-level domain decomposition and load balancing procedure. In the elemental mode, MUMPS uses a single level domain decomposition, and then solves the interface problem. FALKSOL offers a full out-of-core functionality, while MUMPS offers only a partial one. The problem size that can be solved by MUMPS is therefore hard limited by the amount of physical memory plus swap space1 present. This is an actual bottleneck. FALKSOL can solve much larger problems. It is essentially only limited by the size of the secondary storage available. With modern so-called Non-Volatile Memory express (NVMe) on the PCI bus, enough I/O bandwidth is obtained with a couple of raided units. The results show that MUMPS is about twice as fast as FALKSOL in the limited range of problem size it copes with. The solvers are complementary. The out-of-core multi-level domain decomposition algorithm in FALKSOL makes it scale. The test indicates that there is a considerable potential gain in scalability/elasticity choosing the FALKSOL multi-level domain-decomposition out-of-core approach. Especially considering more powerful and larger compute systems. The price to be paid in speed compared with MUMPS is reasonable. The current stand-alone implementation of FALKSOL is interfaced with the 3Dhp code Demkowicz (2007) [32], Demkowicz et al. (2008) [33]. The described MUMPS-like interface can be used to select the best solver for the task.