Modeling and predicting execution time of scientific workflows in the Grid using radial basis function neural network

Abstract

With the maturity of electronic science (e-science) the scientific applications are growing to be more complex composed of a set of coordinating tasks with complex dependencies among them referred to as workflows. For optimized execution of workflows in the Grid, the high level middleware services (like task scheduler, resource broker, performance steering service etc.) need in-advance estimates of workflow execution times. However, modeling and predicting workflow execution time in the Grid is complex due to several tasks in a workflow, their distributed execution on multiple heterogeneous Grid-sites, and dynamic behaviour of the shared Grid resources. In this paper, we describe a novel method based on radial basis function neural network to model and predict workflow execution time in the Grid. We model workflows execution time in terms of attributes describing workflow structure and execution runtime information. To further refine our models, we employ principle component analysis to eliminate attributes of lesser importance. We recommend a set of only 14 attributes (as compared with total 21) to effectively model workflow execution time. Our reduced set of attributes improves the prediction accuracy by $$16\%$$ . Results of our prediction experiments for three real-world scientific workflows are presented to show that our predictions are more accurate than the two best methods from related work so far.