Automatic generation of benchmarks for I/O-intensive parallel applications

Abstract

The benchmarks of I/O-intensive parallel applications are important for evaluating and optimizing HPC softwares and hardwares. However, extracting high-fidelity benchmarks which can fully reflect computation, communication, and I/O behaviors of original I/O-intensive parallel applications is very difficult. This work contributes a framework which can automatically generate benchmarks for I/O-intensive parallel applications. We demonstrate our framework on Taub and TianHe-2 supercomputers with five NAS Parallel Benchmarks (NPB) and four I/O-intensive parallel applications. The results show that our trace merging algorithm and trace compressing algorithm are better than others, and the generated benchmarks can accurately mimic the computation, communication, and I/O behaviors of original I/O-intensive parallel applications. Also, these generated benchmarks can be used to predict the performance of original applications, while reducing the prediction overhead by scaling down the execution time of benchmark proportionally.