Performance and modularity benefits of message-driven execution

Abstract

Processor idling due to communication delays and load imbalances are among the major factors that affect the performance of parallel programs. Need to optimize performance often forces programmers to sacrifice modularity. This paper focuses on the performance benefits of message-driven execution, particularly for large parallel programs composed of multiple libraries and modules. We examine message-driven execution in the context of a parallel object-based language, but the analysis applies to other models such as multithreading as well. We argue that modularity and efficiency, in the form of overlapping communication latencies and processor idle times, can be achieved much more easily in message-driven execution than in message-passing SPMD style. Message-driven libraries are easier to compose into larger programs and they do not require one to sacrifice performance in order to break a program into multiple modules. One can overlap the idle times across multiple independent modules. We demonstrate performance and modularity benefits of message-driven execution with simulation studies. We show why it is not adequate to emulate message-driven execution with the message-passing SPMD style. During these studies, it became clear that the usual criteria of minimizing the completion time and reducing the critical path that are used in SPMD programs are not exactly suitable for message-driven programs.