Distributed Workflow Execution Research Articles

An autonomous blockchain-based workflow execution broker for e-science

AbstractScientific workflows are essential for many applications, enabling the configuration and execution of complex tasks across distributed resources. In this paper, we contribute an Ethereum blockchain-based scientific workflow execution manager, which distributes workflows to run on cluster computing providers that utilize the Slurm workload manager to execute them. We extended our blockchain-based autonomous resource broker called eBlocBroker, which is a DAO-based decentralized coordinator, by providing distributed workflow execution via blockchain. Through various tests, we demonstrate how our eBlockBroker autonomous organization, which is programmed as a smart contract, can manage scientific workflow submission, scheduling, and execution on cluster computing providers. The utilization of blockchain for distributed workflow execution is a new concept. We are motivated because our system has been developed with e-Science in mind where scientific workflows are widely utilized.

Optimized P2P Data Management for ReliableWorkflow Execution in Mobile Environments

Workflows allow to build complex applications out of existing services. Hence, workflows are inherently distributed. At the same time, workflows are mostly executed by centralized workflow engines that invoke the services of a workflow in a request/reply style. These engines may become a performance bottleneck and limit the scalability of the entire system. As a consequence, approaches to distributed workflow execution have been proposed. While these approaches have better scalability characteristics, they require additional efforts to deal with (partial) failures of the system. To cope with such failures, workflow instance data must be stored redundantly at different sites. This is even more important in mobile environments or mixed mobile/stationary environments where the nodes hosting services for workflows may be mobile and are thus more likely to fail permanently or become temporarily unavailable. This is the case, for instance, for sensor net applications where mobile devices capture data, or when smartphone apps share and jointly process data. In this paper, we present the combination of OSIRIS-SR, an extension to the distributed and decentralized workflow engine OSIRIS that focuses on reliable workflow execution by means of instance data replication, and Compass, an extension to Chord that is particularly tailored for efficient P2P data management on mobile devices.

Research and Design on P2P Business Process Execution Framework for Workflow Management Systems

Efficient business workflow management in large-scale areas is in great demand. However, current business workflow management systems are short of distributed workflow execution support. In our paper, we design and implement a distributed framework called PeerODE for Apache ODE (Orchestration Director Engine) [1], an open-sourced business workflow engine. PeerODE presents a scalable approach to P2P business process execution. The scheduling experiment on PeerODE shows that the framework handles the distributed business process execution effectively.

Distributed Workflow Execution Optimization of SBA Based on Service-Oriented Architecture

According to the characteristics of the information resources in modern acquisition, such as heterogeneous, distributed, loosely coupled, the problems and deficiencies of workflow technology and products were analyzed. Dynamic service grouping strategies based on SOA was proposed and was proved in theory. In addition, the workflow architecture based on SOA was built and the workflow engine of simulation-based acquisition (SBA) was designed. Therefore, the legacy code can be wrapped and the service can be grouped dynamically from the dynamic generic service factory. Grouping strategy was proved according to the SPOT satellite image application which was part of the SBA toolbox on grid. Experimental results show that: services grouping of a workflow can reduce the total overhead induced by the submission, scheduling, queuing and data transfer time. On a production grid infrastructure, the optimization proposed introduces a significant speed-up when compared to a traditional execution. The range is from 1.21 to 2.97.

Special Issue: 3rd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2008)

Special Issue: 3rd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2008)

The Grid Resource Broker workflow engine

AbstractIncreasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and/or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread computing and storage resources are made available to scientists belonging to virtual organizations sharing resources across multiple administrative domains through established service‐level agreements. Grids provide an unprecedented opportunity for distributed workflow execution; indeed, many applications are well beyond the capabilities of a single computer, and partitioning the overall computation on different components whose execution may benefit from runs on different architectures could provide better performances. In this paper we describe the design and implementation of the Grid Resource Broker (GRB) workflow engine. Copyright © 2008 John Wiley & Sons, Ltd.

Execution coordination in mobile agent-based distributed job workflow execution

Mobile agent-based distributed job workflow execution requires the use of execution coordination techniques to ensure that an agent executing a subjob can locate its predecessors' execution results. This paper describes the classification, implementation, and evaluation of execution coordination techniques in the mobile agent-based distributed job workflow execution system. First, a classification of the existing execution coordination techniques is developed for mobile agent systems. Second, to put the discussion into perspective, our framework for mobile agent-based distributed job workflow execution over the Grid (that is, MCCF: Mobile Code Collaboration Framework) is described. How the existing coordination techniques can be applied in the MCCF is also discussed. Finally, a performance study has been conducted to evaluate three coordination techniques using real and simulated job workflows. The results are presented and discussed in the paper.

A Service-Oriented Architecture enabling dynamic service grouping for optimizing distributed workflow execution

In this paper, we describe a Service-Oriented Architecture allowing the optimization of the execution of service workflows. We discuss the advantages of the service-oriented approach with regard to the enactment of scientific applications on a grid infrastructure. Based on the development of a generic Web-Services wrapper, we show how the flexibility of our architecture enables dynamic service grouping for optimizing the application execution time. We demonstrate performance results on a real medical imaging application. On a production grid infrastructure, the optimization proposed introduces a significant speed-up (from 1.2 to 2.9) when compared to a traditional execution.