Mechanisms for provenance collection in scientific workflow systems

Abstract

Scientific workflow management systems run scientific experiments. They manage sequences of complex process transformations and collect provenance information at various levels of abstraction. Collected provenance information from scientific experiments documents how experimental results are derived from input values along with experimental parameters and workflow configurations. Provenance greatly enhances usability and acceptance of workflow systems among scientists, because provenance allows workflow systems to capture process configuration and behaviour at different levels of detail. On this basis, a sufficient level of collected provenance information enables scientists to validate their hypotheses and make a workflow reproducible. Currently SWfMS’s do not use a standard or portable provenance model for either capturing, storing, querying or representing model. There are a variety of design issues in provenance models and mechanisms in workflow system, owing to the variation of design dimensions in workflow architectures. Given this variety, it seems desirable to classify provenance mechanisms in workflow systems. We aim to survey provenance collection mechanisms, that are either a part of scientific workflow system, or of a software infrastructure that supports collection mechanisms in a scientific workflow system. In this paper, firstly, we identify and define a set of design dimensions and conventions for provenance collection mechanisms in the context of working on scientific workflow systems. After this, we survey a set of scientific workflow projects based on our design dimensions with an emphasis on provenance collection mechanisms. Then, those conventions are used in order to evaluate a number of existing provenance collection mechanisms, presented at the end of this paper. This survey provides an understanding of primary design issues for provenance collection mechanisms along with a set of desirable design dimensions.