Multi‐installment divisible load processing in heterogeneous distributed systems

Abstract

AbstractDivisible loads are parallel applications with fine granularity and negligible data dependencies. Such computations can be divided into parts of arbitrary sizes and processed independently in parallel. The load distribution process incurs considerable communication delays. To reduce processor waiting time during the computation initialization phase, the load is distributed in multiple small installments rather than in one big chunk. In this paper we analyze multi‐installment divisible load processing in heterogeneous distributed systems. Scheduling divisible loads in heterogeneous systems is hard because the sizes of the installments should be adjusted to the communication and computation capabilities of the system. We show that ignoring heterogeneity of the distributed system may result in arbitrarily bad solutions. Two algorithms are proposed to gear the load chunk sizes to different communication and computation speeds: an optimization branch‐and‐bound algorithm and a heuristic based on a genetic search method. The running times of both methods and the quality of the solutions are compared. Then, we use these algorithms to study the features of the multi‐installment divisible load scheduling problem. We demonstrate that it has both combinatorial and algebraic nature, and that optimum solutions are harder to find with the growing heterogeneity of the system. Copyright © 2007 John Wiley & Sons, Ltd.