RapidXfer - data transfer framework for square kilometre array

Abstract

Data Management of Astronomy Data is often a laborious task and it is even more challenging for the extraordinary amounts of data expected from the world’s largest radio telescope, Square Kilometre Array. There are overt issues in transferring the voluminous SKA data and the traditional data transfer methods are fragile especially for the data transfer between two continents. To address this, a new data transfer framework RapidXfer is proposed and the data transfer is achieved using two steps: international and local transfers. The efficiency of different end-to-end data transfer tools used in RapidXfer is evaluated on different dataset sizes. Further, a comparative study of two IRIS grid data transfer methods is made to understand each methods’ advantages and disadvantages. This study can be used as a reference for the development of future SKA’s data transfer operations.