Intelligent Scheduling and Replication in Datagrids: a Synergistic Approach

Abstract

In large-scale data-intensive applications data plays a pivotal role in the execution of these applications, and data transfer is the primary cause of job execution delay. In environments such as the data grids with the need to execute jobs requiring large amounts of data, a smart collaborative environment between the scheduling and data management services to achieve a synergistic effect on the performance of the grid becomes essential. This paper presents an intelligent data grid framework where job scheduling and data and replica management are coupled to provide an integrated environment for efficient access to data and job scheduling. The data management service predicts and estimates the appropriate locations of replica and proactively replicates the datasets in these locations while the intelligent Tabu Search based scheduler incorporating information about the datasets dispatches the jobs to the sites guaranteeing minimum job execution time and better overall system utilization. Evaluation of the framework shows significant improvement in the performance of the grid and job execution time.