Abstract
Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).
Highlights
The substantial growth in the size of supercomputers—over two orders of magnitude in terms of processing elements since 1993—has created machines of extreme computational power and scale
We introduce a postprocessing tool capable of generating statistical summaries and graphical representations of an application’s parallel input and output (I/O) activities; (ii) Using RIOT, we analyse the I/O behaviour of three industry-standard benchmarks: the Block-Tridiagonal (BT) solver, from the NAS Parallel Benchmark (NPB) Suite; the FLASH-IO benchmark, from the University of Chicago and the Argonne National Laboratory (ANL); and IOR, a high-performance computing (HPC) file system benchmark that is used during procurement and file system assessment [11, 12]
Our analysis employs three contrasting platforms: a mid-size commodity cluster located at the University of Warwick, a large-scale capacity resource housed at the Open Computing Facility (OCF) at the Lawrence Livermore National Laboratory (LLNL) and a proprietary IBM BlueGene/P (BG/P) system installed at the Daresbury Laboratory in the UK; (iii) Through using RIOT, we demonstrate the significant overhead associated with file locking on a small-scale installation of IBM’s General Parallel File System (GPFS) and contrast this to a larger GPFS installation, as well as to a large-scale Lustre installation
Summary
The substantial growth in the size of supercomputers—over two orders of magnitude in terms of processing elements since 1993—has created machines of extreme computational power and scale. This tool is able to trace parallel file operations performed by the ROMIO layer and relate these to their underlying POSIX file operations We note that this recording of low-level parameters permits the analysis of I/O middleware, file format libraries, application behaviour and even the underlying file systems utilized by large clusters. We optimize the application’s behaviour using MPI hint directives and achieve more than a 2× improvement in the write bandwidth; (v) through an I/O trace analysis, we provide insight into the performance gains reported by the Parallel Logstructured File System (PLFS) [13, 14]—a novel I/O middleware being developed by EMC Corporation and the Los Alamos National Laboratory (LANL) to improve file write times.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
