MARS—A framework for minimizing the job execution time in a metacomputing environment

Abstract

Utilizing a collection of workstations and supercomputers in a metacomputing environment does not only offer an enormous amount of computing power, but also raises new problems. The true potential of WAN-based distributed computing can only be exploited if the application-to-architecture mapping reflects the different processor speeds, network performances and the application's communication characteristics. In this paper, we present the Metacomputer Adaptive Runtime System (MARS), a framework for minimizing the execution time of distributed applications on a WAN metacomputer. Work-load balancing and task migration is based on dynamic information on the processor load and network performance. Moreover, MARS uses accumulated statistical data on previous execution runs of the same application to derive an improved task-to-process mapping. Migration decisions are based on: (1) the current system load; (2) the network load; and (3) previously obtained application-specific characteristics. Our current implementation supports C applications with MPI message passing calls, but the general framework is also applicable to other programing environments like PVM, PARMACS and Express.