Scientific Workflow Provenance Research Articles

A Security Framework for Scientific Workflow Provenance Access Control Policies

The notion of collaborative scientific workflow is coined to address the increasing need for collaborative data analytics. In collaborative environments, access control policies are necessary for controlling the sharing of workflows, data products, and provenance information among collaborating parties. In particular, the protection of workflow provenance is critical because it often encodes the detailed protocol of a scientific experiment and carries the intellectual property of the respective stakeholders. In addition, since scientific workflows often evolve quickly, the corresponding access control policies for workflow provenance have to evolve as well. It is important to ensure that the evolution of workflow provenance access control policies maintain certain properties, in order to guarantee the correctness and performance of the corresponding policy enforcement. In this paper, we 1) propose a role-based access control model for scientific workflow provenance; 2) define three quality requirements for scientific workflow provenance access control policies - consistency, completeness, and conciseness; 3) develop a mechanism mapping from specifications of workflows to their counterparts in a provenance that preserves such quality properties, and 4) conduct a case study on a scientific workflow for autism behavioral data analysis that demonstrates the feasibility of our proposed analysis algorithms.

A provenance model for control-flow driven scientific workflows

Provenance in the context of workflows, both for their specification and for the data they derive, is essential for result reproducibility, sharing, and knowledge reuse in the scientific community. The information models to capture the provenance of scientific workflows, known as provenance models, have been of wide interest and studied in many fields, including the semantic Web. However, most existing provenance models rely on commonly perceived data-driven execution paradigm of scientific workflows and overlook control constructs that may appear in scientific workflows. The provenance models proposed by the semantic Web community for data-driven scientific workflows underspecify the structure of the workflows (i.e., workflow provenance). Such an underspecified workflow structure can result in the misinterpretation of a scientific experiment and precludes its conformance checking, thus restricting provenance gains.This paper shows that the design of a provenance model for control-flow constructs and the means to integrate it with the existing provenance models can maximise the provenance gains for the users of scientific workflows. In this regard, first, we specify the need for a control-flow driven scientific workflow provenance model and detail the minimal characteristics which such a model should address. Secondly, we present a formal model to specify the control-flows that may appear in scientific workflows and describe how the existing provenance models can be complemented by using the proposed model. Finally, we show that the proposed model can capture accurate provenance information for the scientific workflows, which makes it possible to understand, reproduce and validate workflows and their output.

The PBase Scientific Workflow Provenance Repository

Scientific workflows and their supporting systems are becoming increasingly popular for compute-intensive and data-intensive scientific experiments. The advantages scientific workflows offer include rapid and easy workflow design, software and data reuse, scalable execution, sharing and collaboration, and other advantages that altogether facilitate “reproducible science”. In this context, provenance – information about the origin, context, derivation, ownership, or history of some artifact – plays a key role, since scientists are interested in examining and auditing the results of scientific experiments. However, in order to perform such analyses on scientific results as part of extended research collaborations, an adequate environment and tools are required. Concretely, the need arises for a repository that will facilitate the sharing of scientific workflows and their associated execution traces in an interoperable manner, also enabling querying and visualization. Furthermore, such functionality should be supported while taking performance and scalability into account. With this purpose in mind, we introduce PBase: a scientific workflow provenance repository implementing the ProvONE proposed standard, which extends the emerging W3C PROV standard for provenance data with workflow specific concepts. PBase is built on the Neo4j graph database, thus offering capabilities such as declarative and efficient querying. Our experiences demonstrate the power gained by supporting various types of queries for provenance data. In addition, PBase is equipped with a user friendly interface tailored for the visualization of scientific workflow provenance data, making the specification of queries and the interpretation of their results easier and more effective.

OPQL: Querying scientific workflow provenance at the graph level

Provenance has become increasingly important in scientific workflows to understand, verify, and reproduce the result of scientific data analysis. Most existing systems store provenance data in provenance stores with proprietary provenance data models and conduct query processing over the physical provenance storages using query languages, such as SQL, SPARQL, and XQuery, which are closely coupled to the underlying storage strategies. Querying provenance at such low level leads to poor usability of the system: a user needs to know the underlying schema to formulate queries; if the schema changes, queries need to be reformulated; and queries formulated for one system will not run in another system. In this paper, we present OPQL, a provenance query language that enables the querying of provenance directly at the graph level. An OPQL query takes a provenance graph as input and produces another provenance graph as output. Therefore, OPQL queries are not tightly coupled to the underlying provenance storage strategies. Our main contributions are: (i) we design OPQL, including six types of graph patterns, a provenance graph algebra, and OPQL syntax and semantics, that supports querying provenance at the graph level; (ii) we implement OPQL using a Web service via our OPMProv system; therefore, users can invoke the Web service to execute OPQL queries in a provenance browser, called OPMProVis. The result of OPQL queries is displayed as a provenance graph in OPMProVis. An experimental study is conducted to evaluate the feasibility and performance of OPMProv on OPQL provenance querying.

Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databases

Provenance, the metadata that records the derivation history of scientific results, is essential in scientific workflows to support the reproducibility of scientific discovery, result interpretation, and problem diagnosis. To promote and facilitate interoperability among heterogeneous provenance systems, the Open Provenance Model (OPM) was first proposed in 2008 and since then has played an important role in the community. In this paper, we present OPMProv, a relational database-based scientific workflow provenance system, that is compliant with OPM (v1.1). Our main contributions are: (i) we design an entity–relationship diagram for OPM and translate it into a relational database schema for the storage of provenance; (ii) we show that provenance reasoning defined in OPM (v1.1) can be sufficiently supported by OPMProv using recursive views and SQL queries alone without any additional reasoning engine. Experiments are conducted to evaluate the performance of OPMProv in data insertion and provenance querying. A case study is performed, demonstrating that OPMProv can answer all except two queries out of the 16 queries defined in the Third Provenance Challenge.

Secure abstraction views for scientific workflow provenance querying

Provenance has become increasingly important in scientific workflows and services computing to capture the derivation history of a data product, including the original data sources, intermediate data products, and the steps that were applied to produce the data product. In many cases, both scientific results and the used protocol are sensitive and effective access control mechanisms are essential to protect their confidentiality. In this paper, we propose: 1) a formal scientific workflow provenance model as the basis for querying and access control for workflow provenance; 2) a security model for fine-grained access control for multilevel provenance and an algorithm for the derivation of a full security specification based on inheritance, overriding, and conflict resolution; 3) a formalization of the notion of security views and an algorithm for security view derivation; and 4) a formalization of the notion of secure abstraction views and an algorithm for its computation. A prototype called SecProv has been developed, and experiments show the effectiveness and efficiency of our approach.

RDFP rov: A relational RDF store for querying and managing scientific workflow provenance

Provenance metadata has become increasingly important to support scientific discovery reproducibility, result interpretation, and problem diagnosis in scientific workflow environments. The provenance management problem concerns the efficiency and effectiveness of the modeling, recording, representation, integration, storage, and querying of provenance metadata. Our approach to provenance management seamlessly integrates the interoperability, extensibility, and inference advantages of Semantic Web technologies with the storage and querying power of an RDBMS to meet the emerging requirements of scientific workflow provenance management. In this paper, we elaborate on the design of a relational RDF store, called RDFP rov, which is optimized for scientific workflow provenance querying and management. Specifically, we propose: i) two schema mapping algorithms to map an OWL provenance ontology to a relational database schema that is optimized for common provenance queries; ii) three efficient data mapping algorithms to map provenance RDF metadata to relational data according to the generated relational database schema, and iii) a schema-independent SPARQL-to-SQL translation algorithm that is optimized on-the-fly by using the type information of an instance available from the input provenance ontology and the statistics of the sizes of the tables in the database. Experimental results are presented to show that our algorithms are efficient and scalable. The comparison with two popular relational RDF stores, Jena and Sesame, and two commercial native RDF stores, AllegroGraph and BigOWLIM, showed that our optimizations result in improved performance and scalability for provenance metadata management. Finally, our case study for provenance management in a real-life biological simulation workflow showed the production quality and capability of the RDFP rov system. Although presented in the context of scientific workflow provenance management, many of our proposed techniques apply to general RDF data management as well.