Enactment Engine Research Articles

HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows

This paper presents HyperFlow: a model of computation, programming approach and enactment engine for scientific workflows. Workflow programming in HyperFlow combines a simple declarative description of the workflow structure with low-level implementation of workflow activities in a mainstream scripting language. The aim of this approach is to increase the programming productivity of workflow developers who are skilled programmers and desire a programming experience similar to the one offered by a mature programming ecosystem. Combining a declarative description with low-level programming enables elimination of shim nodes from the workflow graph, considerably simplifying workflow implementations. The workflow description is based on a formal model of computation (Process Networks) and is characterized by a simple and concise syntax, utilizing just three key abstractions—processes, signals and functions. Yet it is sufficient for expressing complex workflow patterns in a simple way. The adopted model of computation implemented in the HyperFlow workflow engine enables fully distributed and decentralized workflow enactment. The paper describes HyperFlow from the perspective of its workflow programming capabilities, the adopted model of computation, as well as the enactment engine, in particular its distributed workflow enactment capability. The provenance model and logging features are also presented. Several workflow examples derived from other workflow systems and reimplemented in HyperFlow are extensively discussed.

Utilization of workflow management system for virtual machine instance management on cloud

SummaryThe concept of virtualization is one of the most important technologies to construct a cloud service, and especially hardware virtualization is indispensable for infrastructure as a service (IaaS) where the cloud offering, infrastructure, is usually provided as a pool of virtual machine (VM) instances. For that reason, many public IaaS clouds like Amazon Web Service and private cloud toolkits such as Eucalyptus and OpenStack provide users with methods for managing VM instances via APIs, command‐line tools, web services, and so on. These are, however, not easy to use or customize for the average end users, especially for those in scientific research areas who just want to perform their work on a cloud and do not need to know the underlying technologies that much. Utilizing workflow management systems (WfMSs) in managing VMs on a cloud can alleviate these difficulties. Users only need to describe parameters needed for VMs and enact the workflow on a workflow enactment engine using user‐friendly interfaces. We propose a management scheme for VM instances on a cloud with the WfMS in this paper. We present a preliminary study on integrating cloud and WfMS focusing on management of VM instances and show an early implementation for a proof of concept with detailed explanations and possible usage scenarios. Copyright © 2015 John Wiley & Sons, Ltd.

Autonomic Workflow Activities

Workflows have been successfully applied to express the decomposition of complex scientific applications. This has motivated many initiatives that have been developing scientific workflow tools. However the existing tools still lack adequate support to important aspects namely, decoupling the enactment engine from workflow tasks specification, decentralizing the control of workflow activities, and allowing their tasks to run autonomous in distributed infrastructures, for instance on Clouds. Furthermore many workflow tools only support the execution of Direct Acyclic Graphs (DAG) without the concept of iterations, where activities are executed millions of iterations during long periods of time and supporting dynamic workflow reconfigurations after certain iteration. We present the AWARD (Autonomic Workflow Activities Reconfigurable and Dynamic) model of computation, based on the Process Networks model, where the workflow activities (AWA) are autonomic processes with independent control that can run in parallel on distributed infrastructures, e. g. on Clouds. Each AWA executes a Task developed as a Java class that implements a generic interface allowing end-users to code their applications without concerns for low-level details. The data-driven coordination of AWA interactions is based on a shared tuple space that also enables support to dynamic workflow reconfiguration and monitoring of the execution of workflows. We describe how AWARD supports dynamic reconfiguration and discuss typical workflow reconfiguration scenarios. For evaluation we describe experimental results of AWARD workflow executions in several application scenarios, mapped to a small dedicated cluster and the Amazon (Elastic Computing EC2) Cloud.

Report from the second workshop on scalable workflow enactment engines and technology (SWEET'13)

This report summarizes the Second International Workshop on ScalableWorkflow Enactment Engines and Technologies (SWEET'13). This workshop was held in conjunction with the 2013 SIGMOD conference in New York, NY, USA on June 23th, 2013. The goal of the workshop was to bring together researchers and practitioners to explore the state of the art in workflow-based programming for data-intensive applications, and the potential of cloud-based computing in this area. The program featured 4 paper presentations and two very well attended invited talks by Prof. Paul Watson, Newcastle University, UK and Dr Jelena Pjesivac-Grbovic from Google, Inc.

Report from the first workshop on scalable workflow enactment engines and technology (SWEET'12)

This report summarizes the presentations and discussions of SWEET 2012, the First InternationalWorkshop on ScalableWorkflow Enactment Engines and Technologies. SWEET was held in conjunction with the 2012 SIGMOD conference in Scottsdale, Arizona, USA on May 20th, 2012. The goal of the workshop was to bring together researchers and practitioners to explore the state of the art in workflow-based programming for data-intensive applications, and the potential of cloud-based computing in this area. The program featured two very well attended invited talks by Pawel Garbacki from Google and Jimmy Lin from the University of Maryland, on leave at Twitter at the time, as well as a tutorial on Oozie, Yahoo's workflow engine based on Hadoop , by Mohammad Islam from Yahoo/Cloudera.

Using time stream Petri nets for workflow modelling analysis and enactment

Timing requirements are important aspects in workflow modelling, analysis and enactment. In the last few years, though, many workflow languages and tools have been proposed but only few of them address timing issues during enactment. This paper shows that time stream Petri nets (TSPNs), originally designed for multimedia/hypermedia modelling and analysis, are a well-suited formalism also for supporting the whole lifecycle of workflow processes with timing constraints. A novel approach to modelling, analysis and distributed enactment of workflow processes specified by TSPNs is proposed. Functional and temporal properties of a TSPN model can be checked using exhaustive verification or a DEVS-based simulation tool. Enactment rests on PN-Engine, a decentralized enactment engine based on the service-oriented computing paradigm, which enables execution of workflow processes where the coordinated activities may involve cross-boundary organizations. The approach is illustrated by means of a modelling example concerned with a wine-production process.

Validation and mismatch repair of workflows through typed data streams

The type system of a language guarantees that all of the operations on a set of data comply with the rules and conditions set by the language. While language typing is a fundamental requirement for any programming language, the typing of data that flow between processing elements within a workflow is currently being treated as optional. In this paper, we introduce a three-level type system for typing workflow data streams. These types are parts of the Data Intensive System Process Engineering Language programming language, which empowers users with the ability to validate the connections inside a workflow composition, and apply appropriate data type conversions when necessary. Furthermore, this system enables the enactment engine in carrying out type-directed workflow optimizations.

Towards Next Generation Provenance Systems for e-Science

e-Science helps scientists to automate scientific discovery processes and experiments, and promote collaboration across organizational boundaries and disciplines. These experiments involve data discovery, knowledge discovery, integration, linking, and analysis through different software tools and activities. Scientific workflow is one technique through which such activities and processes can be interlinked, automated, and ultimately shared amongst the collaborating scientists. Workflows are realized by the workflow enactment engine, which interprets the process definition and interacts with the workflow participants. Since workflows are typically executed on a shared and distributed infrastructure, the information on the workflow activities, data processed, and results generated (also known as provenance), needs to be recorded in order to be reproduced and reused. A range of solutions and techniques have been suggested for the provenance of data collection and analysis; however, these are predominantly workflow enactment engine and domain dependent. This paper includes taxonomy of existing provenance techniques and a novel solution named VePS (The Vienna e-Science Provenance System) for e-Science provenance collection.

A service-based architecture for dynamically reconfigurable workflows

In the last few years, business process management systems have been employed for handling information systems of ever increasing complexity. As a consequence, the adoption of modelling languages enabling smooth and seamless transitions among the various phases of the process lifecycle, the ability of exploiting coordination schema over distributed execution contexts and the support for dynamic evolution and reconfiguration have become software engineering issues of great importance. This paper proposes the use of PN-Engine, a decentralized Petri nets execution engine, as a business process enactment engine. PN-Engine, which is based on the Jini service architecture, supports the decentralized execution of process models specified as Petri nets (PNs) enhanced with modular constructs and offers suitable mechanisms for dealing with the aforementioned design issues. PN-Engine allows to deploy and enact a new version of an existing process model without requiring the stopping/removal of older instances that are still running. The paper presents a novel approach enabling a decentralized migration procedure where concurrent portions of older instances migrate asynchronously to the new process model. Advantages of the proposed approach are demonstrated by means of an example concerning a workflow for a wine-production process.

Working in the clouds: creating an e-science collaborative environment for neurophysiology

Event Abstract Back to Event Working in the clouds: creating an e-science collaborative environment for neurophysiology Colin Ingram1* 1 University of Newcastle, Institute of Neuroscience, United Kingdom Understanding the way that neural networks function can only be achieved through detailed analysis of the spatio-temporal relationships between activity patterns of neurones within the network. The increasing sophistication of multielectrode and optical recording techniques has provided the means for us to explore the control and organisation of neural networks in ever greater detail. However, whilst developments in neuroinformatics offer new ways in which the large volumes of data generated from such techniques may be manipulated and analysed, there is a growing need to develop an infrastructure to support sharing and collaborative exploitation of these data. Cloud computing provides exciting opportunities for this exploitation with rapidly increasing availability of data storage and compute resource accessed through the web. However, any practical solution to supporting analysis and sharing of time-series neurophysiological data needs to address a number of major challenges, including interoperability of file formats, data visualisation, and effective metadata descriptions. These challenges are being addressed through a number of international research projects, including CARMEN (Code Analysis, Repository and Modelling for e-Neuroscience; www.carmen.org.uk), an open access web platform developed for handling time-series data using distributed computing technology. CARMEN has created a secure collaborative environment in which data can be shared, either between private groups or publicly, and new analysis algorithms can be deployed without restrictions of issues of software and file compatibility. The on-going implementation of an enactment engine is enabling service applications to be linked into more complex and user-defined workflows. The cloud architecture allows the co-location of data and computation and enabling users to conduct their science through a web browser. Furthermore the growing data repository enables maximum exploitation of data that are often difficult and expensive to produce. CARMEN is one example of the shift from small scale science to large scale data integration which will underpin the next major steps in understanding the brain. Infrastructure built on cloud computing will make this integration feasible and will offer new opportunities to build a global community for neuroscience. Conference: Neuroinformatics 2010 , Kobe, Japan, 30 Aug - 1 Sep, 2010. Presentation Type: Oral Presentation Topic: Keynote speakers Citation: Ingram C (2010). Working in the clouds: creating an e-science collaborative environment for neurophysiology. Front. Neurosci. Conference Abstract: Neuroinformatics 2010 . doi: 10.3389/conf.fnins.2010.13.00019 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 09 Jun 2010; Published Online: 09 Jun 2010. * Correspondence: Colin Ingram, University of Newcastle, Institute of Neuroscience, Newcastle, United Kingdom, c.d.ingram@ncl.ac.uk Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Colin Ingram Google Colin Ingram Google Scholar Colin Ingram PubMed Colin Ingram Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

An Executable Language/Enactment Engine Approach for Designing and Architecting Process-Aware Web Applications

Driven by an emerging class of applications, the Web is evolving into a Business System. Web-based business applications allow the participation of several actors to complex enterprise-wide (or even multi-enterprise) business processes and pose new challenges to the software designers and software architects. The design models have to address both navigational and process-based interactions; the software architecture has to provide the components to enact the process and has to define how these components interoperate with the other components of the Web applications. In this article we show how, promoting an effective separation of concerns, a process modeling language and its enactment engine can be used in the modeling and implementation of process-aware Web applications.

WS-RF Workflow in Triana

This paper describes the integration of WS-RF functionality into the graphical workflow enactment engine Triana. The motivation for our work is the belief that standardized communication models as represented by the numerous Web services related specifications will enrich the semantic content of service intercommunication and are likely to achieve widespread adoption. These two features will help to facilitate the kind of dynamic binding of services advocated by Service Oriented Architectures. Furthermore we maintain that exposing the capabilities offered by these specifications to the end user in a simple and intuitive manner is essential if they are to be used intelligently and flexibly, particularly when the specifications do not address the functional semantics of a service as is the case with WS-RF. We examine the possibilities that WS-RF offers to workflow creation and execution, particularly how it enables a number of optimizations which are useful in scientific workflows to be performed in a standardized manner, and describe the visual interpretation of WS-RF, the underlying technologies that enable this interpretation as well as issues arising from the integration of WS-RF into Triana.

Special Issue: First International Workshop on Workflow Systems in Grid Environments (WSGE2006)

This special issue of Concurrency and Computation: Practice and Experience contains selected high-quality papers from the First International Workshop on Workflow Systems in Grid Environments (WSGE2006) which was sponsored by Hunan University of Science and Technology in China and held in the university on 23 October 2006 1. The WSGE workshop series aims to provide an international forum for the presentation and discussion of research and development trends to support workflow in Grid environments. WSGE2006 attracted a number of international attendants. A keynote speech entitled ‘Time Management in Workflow Systems’ was delivered by Professor Yunshun Fan from Tsinghua University in China. A panel discussion was also part of this event, comparing ‘Business Workflow’ vs ‘Grid Workflow’. One of the conclusions reached by the panel members was that a Grid workflow is not significant in any way compared to other such approaches, with the key additional aspects being the computation and data resources used (over a Grid) to enact the workflow. Grid workflow has been under investigation for several years 2-6. In particular, GGF (Global Grid Forum) 10 was focused on it in 2004. GGF 10 was then concluded as a special issue in Concurrency and Computation: Practice and Experience. The title of the special issue is ‘Workflow in Grid Systems’ 7, 8 and it was edited by Professor Geoffrey C. Fox and Professor Dennis Gannon from Indiana University in U.S.A. This WSGE2006 special issue is a follow-up of the GGF 10 special issue. Many existing research and development efforts have focused on the functional issues of Grid workflow, such as support for modelling languages or Grid workflow enactment engines 2, 4-6. Gradually, more and more attention is being paid to non-functional issues such as QoS (Quality of Service), time management, or verification and validation 9-12. Accordingly, this special issue has a focus on the non-functional issues. This special issue contains six papers based on those that were presented at WSGE2006. They are listed as 13-18. Particular attention was paid to the non-functional issues. Besides, research problems must be analysed systematically. For specific approaches or models, evaluation must be presented. Based on these, the six papers were selected and also reviewed thoroughly. They are briefed as follows. Zheng et al. 13 propose an enhanced workflow simulation model supporting the simulation of event-based interaction between composed (Grid) services. Using the proposed approach, the behaviour between composed services can be simulated and in particular evaluation of QoS for service composition can be carried out. An example is presented on the implemented prototype to demonstrate the feasibility of the proposed method. Brandic et al. 14 argue that there is a lack of a holistic environment for Grid workflows that supports QoS in all phases of the workflow lifecycle, i.e. from specification to execution. Existing Grid workflow systems either support the whole workflow lifecycle but lack QoS support, or provide only partial QoS support for certain stages of the workflow lifecycle. Hence, Brandic et al. 14 present ‘Amadeus’—a novel service-oriented environment for QoS-aware Grid workflows. A distinguishing feature of Amadeus is the holistic approach to QoS support during all stages of the workflow lifecycle from specification to execution. The architecture of Amadeus is described in detail. For each stage of the workflow lifecycle, experiments are conducted to evaluate the corresponding Amadeus components. Chen and Yang 15 propose a taxonomy of Grid workflow verification and validation. It classifies different issues in Grid workflow verification and validation into a particular set of categories. The relationship of some existing projects to the proposed taxonomy is also presented. Lee and Min 16 propose a PLC (Peak Load Control)-based orchestration system that can stably execute hybrid Grid services. The PLC mechanism uses the delay time algorithm for controlling a workflow engine's heavy peak load caused by request congestion for a short period of time. With PLC, the performance of workflow engines can be improved. Comparison and simulation analysis further demonstrate this point. Liu et al. 17 propose a time model for large-scale workflow execution spanning different working time zones. The model is called WdCM (Workday Calendar Model). WdCM can allow for the difference in world-wide working time zones. A case study is presented to demonstrate the feasibility of WdCM. Rygg et al. 18 present an intuitive workflow environment called GPFlow to support scientists with their research. The workflow in GPFlow wraps legacy tools and presents a high-level interactive Web-based graphical interface to scientists, so that they can access and execute their scientific workflows. Two bioinformatics experiments are performed to demonstrate its flexibility and simplicity.

Specification and Runtime Workflow Support in the ASKALON Grid Environment

We describe techniques to support the runtime execution of scientific workflows in the ASKALON Grid environment. We present a formal model and three middleware services that support in combination the effective execution in heterogeneous and dynamic Grid environments: performance prediction, scheduling, and enactment engine. We validate our techniques with concrete experimental results for two real-world applications executed in the Austrian Grid environment.

Dynamic Computational Workflows: Discovery, Optimisation and Scheduling

The Grid computing community is converging on a service-oriented architecture in which applications are composed from geographically-distributed, interacting web services, and are expressed in a workflow description language, usually based on XML. Such workflows are viewed as offering a useful representation of service-based applications or applications composed of standalone components that are to be run in a distributed environment. A workflow can be conveniently displayed as a directed graph in which nodes represent services and edges represent the flow of data and/or control between them. In some sense workflows represent “how the Grid is programmed”, and consequently much recent research has focussed on software and tools that support the use of workflows for grid computing. The semantic description of services, components and workflows, service and resource discovery and selection, and the scheduling of workflows present significant research challenges in making workflows of practical benefit to grid users. The solution of these challenges, in turn, depends on languages for workflow representation, enactment engines for workflow execution, and ontologies. This special issue arose mainly out of two international workshops:

Modelling clinical goals: a corpus of examples and a tentative ontology.

Knowledge of clinical goals and the means to achieve them are either not represented in most current guideline representation systems or are encoded procedurally (e.g. as clinical algorithms, condition-action rules). There would be a number of major benefits if guideline enactment systems could reason explicitly about clinical objectives (e.g. whether a goal has been successfully achieved or not, whether it is consistent with prevailing conditions, or how the system should adapt to circumstances where a recommended action has failed to achieve the intended result). Our own guideline specification language, PROforma, includes a simple goal construct to address this need, but the interpretation is unsatisfactory in current enactment engines, and goals have yet to be included in the language semantics. This paper discusses some of the challenges involved in developing an explicit, declarative formalism for goals. As part of this, we report on a study we have undertaken which has identified over 200 goals in the routine management of breast cancer, and outline a tentative formal structure for this corpus.

PRO forma: a general technology for clinical decision support systems

The need for flexible and well understood knowledge representations which are capable of capturing clinical guidelines and protocols for decision support systems is widely recognised. The PRO forma method for specifying clinical guidelines and protocols comprises a graphical notation for their design, and a formal knowledge representation language to enable them to be executed by a computer to support the management of medical procedures and clinical decision making. PRO forma technology consists of a graphical knowledge editor for the creation of guidelines, and an enactment engine for testing and executing them. This paper provides an overview of the motivation and structure of PRO forma, and illustrates its use in the development of clinical applications.