Optimal periodic remapping of dynamic bulk synchronous computations

Abstract

Dynamic remapping is critical to the performance of bulk synchronous computations that have non-deterministic behaviors because of the need of barrier synchronization between phases. There are two basic issues in remapping: when and how to remap. This paper presents a formal analysis of the issue of when to remap for dynamic computations with a priori known statistical behaviors, with an objective of finding optimal remapping frequencies for a given tolerance of load imbalance. The problem is formulated as two complement sequential stochastic optimization. Since general optimization techniques tend to reveal stationary properties of the workload process, they are not readily applicable to the analysis of the effect of periodic remapping. Instead, this paper develops new analytical approaches to precisely characterize the transient statistical behaviors of the workload process on both homogeneous and heterogeneous machines. Optimal remapping frequencies are derived for various random workload change processes with known or unknown probabilistic distributions. They are shown accurate via simulations.