Barrier Synchronization: Simplified, Generalized, and Solved Without Mutual Exclusion

Abstract

Barrier synchronization is a fundamental concurrency issue encountered in a large number of concurrent and parallel applications that involve parallel processes cooperatively solving complex problems. Barrier synchronization essentially forces these parallel processes to wait until each one of them has reached a certain point in execution. Therefore, barrier synchronization has been considered as an inevitable synchronization construct for most parallel applications. Hence barrier synchronization or its variant has become an integral component of the most widely used parallel programming models such as OpenMP, MPI, MapReduce, and the bulk synchronous parallel (BSP) models. A recent work stated that the performance of BSP model depends on 4 parameters – the number of nodes, the speed of each node, the communication cost, and the synchronization cost. Among these, the synchronization cost is considered critical in improving the performance of any BSP implementation. From parallel algorithm design and implementation perspective, we consider that the synchronization component is the most important one and its cost is the most critical performance factor for almost all the parallel programming models including BSP. This paper presents a class of simple barrier synchronization algorithms for shared memory systems. It includes general, efficient, and universal algorithms that are appealing from both practical and theoretical point of view. The correctness of the algorithms are proved. The algorithms are briefly analyzed to expose their strengths and weaknesses.