An agent based network resource planner for workflow applications

Abstract

Many scientific workflow applications are driven by simulation generated data, or data collected from sensors or instruments, and the processing of the data is commonly done at a different location from where the data is stored. Moving large quantities of data among different locations is thus a fr equently invoked process in scientific workflow applications. These data transfers often have high quality requirements on the network services, especially when the application requires steering from human interaction. Advanced networks such as hybrid networks make it feasible for high level applications to request network paths and service provisioning. However, current workflow applications tune the execution quality neglecting network resources, and by selecting only optimal software services and computing resources. Including network services in the resource scheduling adds an extra dimension for workflow applications to optimize the runtime performance. In this paper we present a system called NEtwork aware Workflow QoS Planner (NEWQoSPlanner) to complement existing workflow systems on selecting network resources in the context of workflow composition, scheduling and execution when advanced network services are available.