Specification and Performance Characteristics of Scientific Grid Workflows

Abstract

Grid computing promises to enable a scalable, reliable, and easy-to-use computational infrastructure for e-Science. To materialize this promise, Grids need to provide full automation of the entire development and execution cycle starting with application modeling and specification, continuing with experiment design and management, and ending with the collection and analysis of results. Often, this automation relies on the execution of workflow processes. Not much is known much about Grid workflow characteristics, scalability, and workload, which hampers the development of new techniques and algorithms, and slows the tuning of existing ones. This chapter describes techniques developed in the ASKALON project for modeling and analyzing the executions of scientific workflows in Grid environments. The authors first outline the architecture, services, and tools developed by ASKALON and then introduce a new systematic scalability analysis technique to help scientists understand the most severe sources of performance losses that occur when executing scientific workflows in heterogeneous Grid environments. A method for analyzing workload traces is presented, focusing on the intrinsic and environment-related characteristics of scientific workflows. The authors illustrate concrete results that validate the methods for a variety of real-world applications modeled as scientific workflows and executed in the Austrian Grid environment.