Efficient Performance Prediction for Large-Scale, Data-Intensive Applications

Abstract

This paper presents a simulation-based performance prediction framework for large-scale, data-intensive applications on large-scale machines. The framework consists of two components: application emulators and a suite of simulators. Application emulators provide a parameterized model of data access and computation patterns of the applications and enable changing critical application components (input data partitioning, data declustering, processing structure, etc.). The suite of simulators executes quickly on a high performance workstation to allow performance prediction of large-scale parallel machine configurations. The key to efficient simulation of very large configurations is to elide the majority of low-level hardware events while preserving data dependencies and distributions. The authors evaluate their performance prediction tool using a set of three data-intensive applications.