A communication+computation load balanced loop partitioning method

Abstract

We present an iteration and data partitioning approach for DOALL loops on distributed memory systems. The method first examines the highest amount of parallelism (available parallelism) which could be potentially exploited in a loop nest. It then examines the amount of communication overhead which can potentially nullify the benefits due to parallelism and attempts to maximally eliminate the communication to minimize the loop completion time by trading parallelism to a minimal extent. This is achieved by determining the directions of iteration space partitioning which result in minimum communication. Finally, in order to generate a load balanced partition with respect to computation+communication, the method uses a new larger partition owns rule to distribute the underlying data. Necessary theoretical framework has been developed and the merit of the method is shown through a performance evaluation on Cray T3D.