Distributed processing of divisible jobs with communication startup costs

Abstract

In this work we analyze the problem of an optimal distribution of a computational task among a set of processors. We assume that the task can be arbitrarily divided and its parts can be processed in parallel on different processors. A wide range of interconnection architectures of distributed computer systems is taken into consideration: a chain, a loop, a tree, a star of processors, a set of processors using shared buses, and a hypercube of processors. It is assumed that the communication time is equal to some startup value plus some amount proportional to the volume of transferred data. Using a uniform methodology we present a method to find the distribution of the load so that the minimum completion time is achieved for the considered data distribution scheme. The results can also be used to find such parameters of the processor network as equivalent speed, speedup and utilization. Moreover, the methodology presented here can be a model of the application roll-in time, and can be applied in load balancing in a heterogeneous multiprocessor system.