A performance evaluation methodology for parallel simulation protocols

Abstract

Most experimental studies of the performance of parallel simulation protocols use speedup or number of events processed per unit time as the performance metric. Although helpful in evaluating the usefulness of parallel simulation for a given simulation model, these metrics tell us little about the efficiency of the simulation protocol used. In this paper, we describe an Ideal Simulation Protocol (ISP), based on the concept of critical path, which experimentally computes the best possible execution time for a simulation model on a given parallel architecture. Since ISP computes the bound by actually executing the model on the given parallel architecture, it is much more realistic than that computed by a uniprocessor critical path analysis. The paper illustrates, using parameterized synthetic benchmarks, how an ISP-based performance evaluation can lead to much better insights into the performance of parallel simulation protocols than what would be gained from speedup graphs alone.