A Framework for an Integrated Co-allocator for Data Grid in Multi-Sender Environment

Abstract

Data grid consists of scattered computing and storage resources located dispersedly in the grid network. These large sized data sets are replicated in more than one site for the better availability to the other nodes in a grid. Downloading the dataset from these replicated locations have practical difficulties and we find interest in a co-allocated download framework, which enables parallel download of replicated data from multiple servers. In this paper, we proposed a dynamic co-allocation scheme for parallel data transfer in grid environment, which copes up with highly inconsistent network and server performance. The model comprises of co-allocator, monitor and control mechanisms. The scheme initially obtains the bandwidth parameter from the monitor module to fix the partition size and the data transfer tasks are allocated onto the servers in duplication. In this way, the process of data transfer can neither be interrupted nor paralyzed, even when the network link is broken or server crash. We used Globus toolkit for our framework by making use of grid information and GridFTP services. We compared our scheme with the existing schemes and the results show notable improvement in overall completion time of data transfer.