A heuristic approach for finding a solution to the constant-degree parallelism alignment problem

Abstract

Two important aspects have to be addressed when automatically parallelizing loop nests for massively parallel distributed memory computers, namely maximizing parallelism and minimizing communication overhead due to non-local data accesses. This paper studies the problem of finding a computation mapping and data distributions that minimize the number of remote data accesses for a given degree of parallelism. This problem is called the constant-degree parallelism alignment problem. The heuristic presented uses a linear algebra framework and assumes affine data access functions. It proceeds by incrementally building a basis of the set of vectors representing the alignments between computation and data accesses that should be satisfied. The heuristic algorithm is applied to benchmark programs and shown superior to more basic mappings.