Improving Efficiency in Parallel Computing Leveraging Local Synchronization

Abstract

In a parallel computing scenario, a complex task is typically split among many computing nodes, which are engaged to perform portions of the task in a parallel fashion. Except for a very limited class of application, computing nodes need to coordinate with each other in order to carry out the parallel execution in a consistent way. As a consequence, a synchronization overhead arises, which can significantly impair the overall execution performance. Typically, synchronization is achieved by adopting a centralized synchronization barrier involving all the computing nodes. In many application domains, though, such kind of global synchronization can be relaxed and a lean synchronization schema, namely local synchronization, can be exploited. By using local synchronization, each computing node needs to synchronize only with a subset of the other computing nodes. In this work, we evaluate the performance of the local synchronization mechanism when compared to the global synchronization approach. As a key performance indicator, the efficiency index is considered, which is defined as the ratio between useful computation time and total computation time, including the synchronization overhead. The efficiency trend is evaluated both analytically and through numerical simulation.