Hardware supports for efficient barrier synchronization on 2-D mesh networks

Abstract

In this paper, we consider a hardware scheme for supporting barrier synchronization on scalable systems with a 2D mesh network. Our design takes into account of the program execution path in such systems-from programming interfaces down to routers. The hardware router design will be based on the MPI-1 standard. A distributed algorithm is proposed to construct a collective synchronization tree (CS tree) from the nodes participating in the barrier based upon the CS tree, the status registers in the routers are set up and synchronization messages are transmitted along the paths set by the status registers. Performance evaluations show that our proposed method has better performance for barrier synchronization and is less sensitive to variations in group size and startup delay than previous approaches. However our scheme has the extra overhead of building the CS tree. Thus it is more suitable for parallel iterative computations, in which the same barrier is invoked repetitively.