A multiple parallel download scheme with server throughput and client bandwidth considerations for data grids

Abstract

Though computer technology advances quickly, the computing speed and storage capacity of a single computer still cannot satisfy the requirements of many applications. As a result, grids have emerged to utilize the collective power of many computers. One of them, the data grid, provides a mechanism for handling a large amount of data. One of the characteristics of a data grid is to replicate files to many different computers such that a popular file would be more available. If a grid site does not have a file, it will have to download it from other grid sites. Thus, the parallel download method, which allows a user to download different parts of a file from various computers simultaneously, is used to decrease download time. However, multiple parallel downloads will affect one another. Thus if all jobs in the grid system use parallel download, the problem of resource competition and conflict will happen. In this paper, we propose a parallel download scheme considering the server output throughput limits and client input bandwidth constraints. Experimental results show that the proposed download scheme outperforms static and dynamic parallel download schemes.