Workflow Skeletons: A Non-intrusive Approach for Facilitating Scientific Workflow Modeling

Abstract

Scientific workflows are built of highly parallel patterns comprising huge numbers of tasks. In general, modeling and configuring such scientific workflows is complex and error-prone. With the language Work SKEL, building blocks called, workflow skeletons" offer an abstraction to such recurring patterns. Thus, the parallel parts of scientific workflows can be defined with a few lines of Work SKEL code specifying the choreography of a large number of parallel tasks. Workflow skeletons facilitate the definition of workflows by accepting parameters that allow for scalable specifications and configurations that save time and cost by allocating cloud resources just in time. This paper shows an integrated approach for modeling scientific workflows using a higher abstraction due to workflow skeletons. The tool suite ProWorkE (Proxy-enhanced Workflow Engine) is a research prototype allowing for assembling scientific workflows with workflow skeletons, defining new skeletons in Work SKEL if need be, and mapping them to a target workflow management system in a non-intrusive way. Thus, ProWorkE supports the whole lifecycle from modeling the workflow to its execution. An image rendering workflow is used as an example to demonstrate how pre-defined workflow skeletons are combined to describe the workflow, scaled up to a large number of tasks, deployed on demand, and executed in a cloud.