Implementation and evaluation of GridFTP Automatic Parallelism Tuning mechanism for long-fat, networks

Abstract

In this paper, we present our implementation of GridFTP-APT (GridFTP with automatic parallelism tuning), an extension to GridFTP for optimizing its performance in a long-fat network. GridFTP has been used as a data transfer protocol for effectively transferring a large volume of data in Grid computing. GridFTP supports a feature called par allel data transfer that improves throughput by establishing multiple TCP connections in parallel. However, to achieve high GridFTP throughput, the number of TCP connections should be optimized according to network condition. We have proposed an automatic parallelism tuning mechanism called GridFTP-APT that adjusts the number of parallel TCP connections only using information measurable at Grid middleware. In this paper, we first explain our implementation of GridFTP-APT based on the GridFTP client included in the Globus Toolkit (Gobus-url-copy program). We then investigate the effectiveness of GridFTP-APT in realistic network environments through several experiments using our GridFTP-APT implementation and a network emulator. Consequently, we demonstrate the effectiveness of GridFTP APT in various network environments.