Scheduling Divisible Loads with Return Messages on Multi-core Heterogeneous Clusters with Unknown System Parameters

Abstract

Abstract By taking into account of the cost of return messages, a new multi-round divisible loads scheduling algorithm is designed on the heterogeneous cluster of multi-core processors with unknown system parameters. The algorithm consists of probing system parameters stage and distributing loads stage. In the two-round probing stage, master first distributes a small part of the loads to the workers to process and utilizes the required time when it has received return messages to estimate the system parameters such as the amount of workers, the number of the processing cores for each worker, computation capability and delay for each worker, the main memory capacity and size of shared L2 cache for each worker, the communication capability and delay between master and a worker. In the multi-round distribution loads stage, master distributes the remainder of the loads to the workers and their corresponding processing cores by FIFO strategy and multi-round scheduling mode, which each round scheduling considers the constraint of capacity of main memory for each multi-core node on the heterogeneous cluster. The experimental results on the heterogeneous cluster with multi-core machines show that the two-round probing method can estimate the system parameters and the multi-round distribution load strategy can obtain shorter scheduling length and has stronger practicability.