An efficient data transfer service for scientific applications in cloud environments

Abstract

Recently, more and more data-intensive scientific applications have been deployed in cloud environments. Therefore, how to improve the efficiency of data transfer becomes an important issued that needs to be addressed. In this paper, we present an efficient data transfer framework which provides an integrated platform for data transfer, data scheduling and performance monitoring. Unlike those existing studies that focus on the utilisation of bandwidth resources, the proposed framework is implemented by integrating data transfer service and data scheduling service through a performance prediction service. In this way, it provides a flexible mechanism to enable a cloud system to improve the efficiency of data transfer. The implementation of the proposed framework has been deployed in a real-world cloud system, and experimental results have shown that in can significantly improve the efficiency of massive-data transfer comparing with many existing approaches.