A Connectivity Reduction Based Optimization Method for Task Level Parallel File System

Abstract

Task level parallel file system has been a good method to improve the application I/O performance due to its ability to flexibly configure the I/O software stack according to the computing tasks characteristics. However, in order to maintain the global consistency and a unified access view, the task level parallel file system usually constructs a full connected structure between all clients and all servers, which may affect the performance of the parallel file system, especially with large scales. In this paper, a connectivity reduction method, based on inter-process data correlation, is proposed. This solution can remove the unnecessary connectivity within the file system and guide the data access process according to the application needs; thus, the performance of the file system will be improved. The experimental results show that our method can improve the overall I/O efficiency while ensuring the required consistency of the applications.