An efficient and bandwidth sensitive parallel download scheme in data grids

Abstract

For modern scientific applications such as astrophysics, astronomy, aerography, and biology, a large amount of storage space is required because of the large-scale datasets. Data Grid collects distributed storage resources such as hard disk space across heterogeneous networks to meet such requirements. In data grid environment, data replication service that copies the replicas to proper storage systems increases the reliability of the data access. By means of these replicas, parallel download creates multiple connections from the client side to the replica servers to improve the performance of the data transfer. To adapt the bandwidth-variation and to make the data transferring more efficient, a parallel download scheme which is called EA (efficient and adaptive) parallel download is proposed in this paper. The scheme is to re-evaluate all of the replica servers during the download progress and replace the decaying selected servers with better backup servers. According to our experiments in the Unigrid environment, the EA parallel download decreases the completion time by 1.63% to 13.45% in natural Unigrid environment and 6.28% to 30.56% in choreographed Unigrid environment when compared to the recursive co-allocation scheme. It means that the proposed scheme adapts to the dynamic environment nicely and decreases the total download time effectively.