T-NBC: Transparent Non-blocking MPI Collective Operations

Abstract

Without modifying MPI applications,transparently translating MPI collective operations into non-blocking ones in communication libraries can overlap collective communication with the computation following the operations and benefit most current applications.In applications,the following computation includes communication-unrelated computation(CURC) and communication-related computation(CRC).CURC is easier to overlap with collective communication;however,CRC need access communication data and is more difficult to overlap with collective communication.In the paper,we propose transparent non-blocking collective operations(T-NBC).It can obtain the overlap between collective communication and following communication.Besides the overlap with CURC,it improves the overlap with CRC by transmitting collective messages with different priorities according to their accessed sequence in applications.Evaluations of micro-benchmark demonstrate that a large potential overlap between collective communication and following computation can be obtained.In FT(Fourier Transform) and IS(Integer Sort) of NPB(NAS Parallel Benchmarks),even following computation dominated by CRC,a large portion of collective communication is overlapped.Their performance is respectively improved by 5% and 36%.