Efficient Barrier Synchronization Research Articles

HyBar: high efficient barrier synchronization based on a hybrid packet-circuit switching Network-on-Chip

Realizing barrier synchronization in multi-/many-core processors with high efficiency becomes more and more challenging as the number of cores integrated in a single chip keeps growing. Quite a few barrier solutions have been proposed, while they provide limited improvements for synchronizing large amounts of cores or incur unfavorable restrictions on performing concurrent barriers. This paper presents HyBar, a hardware barrier based on a hybrid switching NoC which adopts packet switching and circuit switching methods in two sub-networks respectively. Dedicated channels in the circuit-switching sub-network are dynamically built and removed when barrier requests traverse the packet-switching sub-network according to a modified dimension-order routing algorithm. The efficiency of inter-core communication for concurrent barriers is improved by merging barrier arrival requests and broadcasting release requests along the circuit channels. The execution time of synthetic cases, benchmark kernels and parallel applications using various barrier solutions are evaluated in an RTL-based simulation platform. Experimental results show that our proposal provides about 15\%--50\% performance improvement compared to previous solutions, while the hardware overhead is marginal under SMIC 40 nm technology. Moreover, HyBar introduces a minor efficiency loss for concurrent barriers with no limitation on their layouts of participating cores in the on-chip network.

Barrier synchronization on wormhole-routed networks

In this paper, we propose an efficient barrier synchronization scheme on networks with arbitrary topologies. We first present a distributed method in building a barrier routing tree. The barrier messages can be delivered adaptively according to the hierarchy of the established barrier tree to void congestion and faulty nodes in the network. We then propose a new technique, called bandwidth-preempting technique, for a blocked barrier message to preempt a channel occupied by a data message so that the latency of a barrier message can be controlled without affecting much of the overall system performance. We also propose an analytical performance model and present simulation results for the performance evaluation of the proposed scheme. Performance evaluations show that the proposed scheme outperforms the existing algorithms for barrier synchronization.

Efficient barrier synchronization in wormhole-routed mesh networks supporting turn model

Barrier is an important synchronization operation. On scalable parallel computers, it is often implemented as a collective communication. A typical barrier synchronization operation consists of a reduction operation followed by a distribution operation. In this paper, we introduce a systematic way of generating efficient algorithms to perform barrier synchronization in mesh networks. The scheme works with any base routing algorithm that is derivable from the turn model C.J. Glass, L.M. Ni, in: Proc. Intl. Symp. Computer Architecture, pp. 278–297. It extends the turn grouping method proposed by K.P. Fan, C.T. King, Turn grouping for supporting efficient multicast in wormhole mesh networks, in: Proc. 6th Symp. on the Frontiers of Massively Parallel Computing (Frontiers '96), October 1996 with two new algorithms, Tail_to_Central and Central_to_Tail. These two algorithms schedule the transmissions of synchronization messages in the reduction and distribution phase respectively. Performance of the proposed method is evaluated using four typical turn-model based algorithms. The simulation results show that our approach can take advantage of the adaptivity of the base routing algorithms and outperforms methods proposed previously.

Efficient barrier synchronization techniques and their applications in large-scale shared memory multiprocessors

Shared memory multiprocessors offer a relatively simple programming model and are suitable for a wide variety of parallel applications. Unfortunately, shared memory multiprocessors are not as scalable as distributed memory multiprocessors owing to memory and switch contentions that can result in the formation of hot spots. Spinning on synchronization variables appears to be the main culprit behind the formation of hot spots, affecting system scalability adversely. The purpose of this paper is to address the issue of performing barrier synchronization efficiently in large-scale shared memory multiprocessors. We propose a very simple design for a hardware barrier synchronizer that has the characteristics of what one would call an ideal barrier synchronizer. In particular, the proposed barrier synchronizer allows fast barrier synchronization without injecting spin traffic to create hot spots and can be reused as soon as it has completed a barrier synchronization. We also show that by augmenting this barrier synchronizer with a few gates, it can be used to perform dynamic barrier synchronization, where neither the number, nor the exact identity of processors participating in the barrier is known a priori. We will also show that a low-latency barrier synchronizer can be used not only for high-speed barrier synchronization but also, very profitably, for implementing software combining (allowing distributed hot spot accessing), for data and producer-consumer type synchronization and for the implementation of a variety of other useful applications. A high-speed barrier synchronizer can also be used to implement highly concurrent data structures and will also allow a MIMD (Multiple Instruction streams, Multiple Data streams) system to be effectively operated in a SIMD (Single Instruction stream, Multiple Data streams)-style mode, giving rise to a number of potential advantages. We use simulations to confirm that our proposed synchronizers and their applications outperform the existing barrier synchronization schemes.