Network-aware scheduling for real-time execution support in data-intensive optical Grids

Abstract

High performance communication support is a critical aspect for modern Grid environments where the deployment of data-intensive applications often requires moving Petabytes of data between geographically distant data repositories in specific time ranges. The best-effort delivery system of the Internet, often used as the underlying transport network, imposes severe constraints on the transfer of massive amounts of data, and thus restricts the deployment of the above applications on wide-area scales. Besides the lack of bandwidth, the inability to provide dedicated links makes the current network technology not well suited for performance critical Grid computing. A solution is needed to provide dedicated end-to-end connections, dynamically allocable on-demand or by scheduled reservation to critical data-intensive applications. Wavelength routing is at the state of the art the most promising solution to realize such connections. Accordingly, we propose an evolutionary Grid approach built on inter-domain end-to-end lightpath provisioning, controlled by a meta-scheduling logic on advance reservation basis, offering the required connectivity services to the involved Grid nodes by influencing job scheduling decisions with network-related information. We discuss opportunities and challenges in the network control plane and Grid middleware design, together with the required interfaces, adapters and algorithms critical to the provision of network-assisted extensible resource scheduling services. The proposed solution will result in a flexible Grid architecture that supports cooperation between different scheduling entities, based on a scalable framework for dynamic configuration and interconnection of multiple types of resources for high performance Grid applications over globally distributed optical network systems.