On improving the performance of data partitioning oriented parallel irregular reductions

Abstract

Different parallelization techniques for reductions have been classified in this paper into two classes: LPO (loop partitioning-oriented techniques) and DPO (data partitioning-oriented techniques). We have analyzed both classes in terms of a set of performance properties: data locality, memory overhead, parallelism and workload balancing. We propose several techniques to increase the exploited parallelism and to introduce load balancing into a DPO method. Regarding parallelism, the solution is based on the partial expansion of the reduction array. For load balancing, the first technique is generic, as it can deal with any kind of load unbalance present in the problem domain. The second technique handles a special case of load unbalancing appearing when there are a large number of write operations on small regions of the reduction arrays. Efficient implementations of the proposed optimizing solutions for the DWA-LIP (data write affinity-loop index prefetching) DPO method are presented, experimentally tested on static and dynamic kernel codes, and compared with other parallel reduction methods.