Compiling several classes of communication patterns on a multithreaded architecture

Abstract

Communication optimizations play a crucial role in performance of parallel applications which are compiled and executed on distributed memory machines. Multithreaded architectures can support multiple threads of execution on each processor, with low-cost thread initiation, low-overhead communication, and efficient data transfer and synchronization between threads on different processors. These mechanisms can be used for achieving an effective overlap between communication and computation, and therefore, good performance on communication intensive parallel applications. We focus on generating correct and efficient multithreaded code for array based programs that involve different classes of communication patterns. We consider producer-consumer, scalar reductions, and near-neighbor communication patterns. We describe multithreaded programming methodologies suitable for handling loops with each of these patterns. We further show how a compiler can generate threaded code for loops with such patterns. We present experimental results from two benchmark programs, CG, and Tomcatv. Our results show that: 1) the compiler generated multithreaded code achieves high performance, not previously seen from distributed memory compilers, and 2) the performance of compiler generated code is comparable to the performance of hand-written multithreaded codes.