Adaptive Dynamic Process Scheduling on Distributed Memory Parallel Computers

Wei Shu

doi:10.1155/1994/856294

Abstract

One of the challenges in programming distributed memory parallel machines is deciding how to allocate work to processors. This problem is particularly important for computations with unpredictable dynamic behaviors or irregular structures. We present a scheme for dynamic scheduling of medium-grained processes that is useful in this context. The adaptive contracting within neighborhood (ACWN) is a dynamic, distributed, load-dependent, and scalable scheme. It deals with dynamic and unpredictable creation of processes and adapts to different systems. The scheme is described and contrasted with two other schemes that have been proposed in this context, namely the randomized allocation and the gradient model. The performance of the three schemes on an Intel iPSC/2 hypercube is presented and analyzed. The experimental results show that even though the ACWN algorithm incurs somewhat larger overhead than the randomized allocation, it achieves better performance in most cases due to its adaptiveness. Its feature of quickly spreading the work helps it outperform the gradient model in performance and scalability.

Highlights

Large distributed memory parallel machines are becoming increasingly available
To e ciently use such large machines to solve an application problem, the computation must rst be divided into parallel actions
In this paper we describe a dynamic and distributed scheduling scheme called Adaptive Contracting Within Neighborhood (ACWN)

Summary

Introduction

To e ciently use such large machines to solve an application problem, the computation must rst be divided into parallel actions. These parallel actions are mapped and scheduled onto processors. One may decide that each processor will compute a 16 x 16 submatrix of the result matrix by using appropriate rows and columns from the original matrix This leads to 16 sub-computations, as desired, and either an automatic scheduler or a programmer can specify the appropriate data movement and computations. A scheduling scheme in such a context must deal with dynamic creation of work It must cope with work generation and consumption rates that vary from processor to processor and from time to time.

Background

Randomized Allocation and Gradient Model

Adaptive Contracting Within Neighborhood

Performance Studies

Discussion

Conclusion