Realizing Effective MPI-I/O to a Remote Computer Using a Parallel Virtual File System

Abstract

This paper presents a newly implemented remote MPI-I/O mechanism using a Parallel Virtual File System (PVFS) to achieve high performance data-intensive I/O operations among computers. MPI-I/O extensions were realized in a flexible intermediate library named Stampi to support seamless MPI-I/O operations among computers. With the help of a flexible underlying communication mechanism of the library, MPI-I/O operations are available both inside a computer and among computers with the same MPI-I/O APIs without awareness of underlying communication and I/O mechanisms. To cope with recent data-intensive applications, PVFS was developed, and it provides a huge amount of parallel file system on a Linux PC cluster. Although it is available inside a PC cluster, it is not accessible from a remote computer. To exploit advantage of the PVFS file system in the remote MPI-I/O operations using Stampi, PVFS I/O functions have been implemented in the remote MPI-I/O mechanism. Through performance measurement of primitive MPI-I/O functions, sufficient performance has been achieved and effectiveness of the implementation has been confirmed.