Distributed Process Execution Research Articles

Architectural Concept and Evaluation of a Framework for the Efficient Automation of Computational Scientific Workflows: An Energy Systems Analysis Example

Scientists and engineers involved in the design of complex system solutions use computational workflows for their evaluations. Along with growing system complexity, the complexity of these workflows also increases. Without integration tools, scientists and engineers are often highly concerned with how to integrate software tools and model sets, which hinders their original research or engineering aims. Therefore, a new framework for streamlining the creation and usage of automated computational workflows is introduced in the present article. It uses state-of-the-art technologies for automation (e.g., container-automation) and coordination (e.g., distributed message oriented middleware), and a microservice-based architecture for novel distributed process execution and coordination. It also supports co-simulations as part of larger workflows including additional auxiliary computational tasks, e.g., forecasting or data transformation. Using Apache NiFi, an easy-to-use web interface is provided to create, run and control workflows without the need to be concerned with the underlying computing infrastructure. Initial framework testing via the implementation of a real-world workflow underpins promising performance in the realms of parallelizability, low overheads and reliable coordination.

Process Evolution in a Distributed Process Execution Environment

To allow the distribution of control and visibility of cross-organizational process models and increase availability and performance of the processes, a process model can be fragmented into logically different parts and distributed in the enterprise architecture. Fragmentation algorithms and execution environments which connect the fragmented process model parts together, recreating the original process execution semantics, have been proposed in earlier work. However, a critical challenge that is left open is the ability of the distributed process execution environment to respond effectively to process changes. In this paper, the authors describe the difficulties, advantages and issues of process model change support in a fragmented and distributed environment. Moreover, the authors propose a system which tackles the identified issues and allows the propagation and coordination of process changes at runtime in the distributed process execution architecture.

Decentralized Communication for Data Dependency Analysis Among Process Execution Agents

This paper presents the authors results with the investigation of decentralized data dependency analysis among concurrently executing processes in a service-oriented environment. Distributed Process Execution Agents (PEXAs) are responsible for controlling the execution of processes that are composed of web services. PEXAs are also associated with specific distributed sites for the purpose of capturing data changes that occur at those sites in the context of service executions using Delta-Enabled Grid Services. PEXAs then exchange this information with other PEXAs to dynamically discover data dependencies that can be used to enhance recovery activities for concurrent processes that execute with relaxed isolation properties. This paper outlines the functionality of PEXAs, describing the data structures, algorithms, and communication mechanisms that are used to support decentralized construction of distributed process dependency graphs, demonstrating a more dynamic and intelligent approach to identifying how the failure of one process can potentially affect other concurrently executing processes.

Distributed process execution in collaborative networks

Processes are a central entity in enterprise collaboration. Collaborative processes need to be executed and coordinated in a distributed computational platform where computers are connected through heterogeneous networks and systems. Life cycle management of such collaborative processes requires a framework able to handle their diversity based on different computational and communication requirements. This paper proposes a rational for such framework, points out key requirements and proposes a strategy for a supporting technological infrastructure. Beyond the portability of collaborative process definitions among different technological bindings, a framework to handle different life cycle phases of those definitions is presented and discussed.

Extended memory management (XMM): lessons learned

This paper describes the lessons learned from the development and use of the eXtended Memory Management (XMM) subsystem in the Mach microkernel. XMM provides distributed memory management for single system image Unix on systems ranging from a few nodes to over 1000 nodes. XMM enables complete cross-node transparency at the Mach virtual memory interface in support of distributed file systems and distributed process execution, including making distributed shared memory available to applications. Our experience with XMM has revealed severe problems in scalability, performance, and complexity, leading us to conclude that significant portions of XMM's architecture and design should be viewed as failed experiments. Among the lessons we have learned from our experience are that copy on reference should be used instead of copy on write for memory management in distributed systems, and that virtual copy optimizations are inappropriate for distributed coherent virtual memory. © 1998 John Wiley & Sons, Ltd.

Extended memory management (XMM): lessons learned

This paper describes the lessons learned from the development and use of the eXtended Memory Management (XMM) subsystem in the Mach microkernel. XMM provides distributed memory management for single system image Unix on systems ranging from a few nodes to over 1000 nodes. XMM enables complete cross-node transparency at the Mach virtual memory interface in support of distributed file systems and distributed process execution, including making distributed shared memory available to applications. Our experience with XMM has revealed severe problems in scalability, performance, and complexity, leading us to conclude that significant portions of XMM's architecture and design should be viewed as failed experiments. Among the lessons we have learned from our experience are that copy on reference should be used instead of copy on write for memory management in distributed systems, and that virtual copy optimizations are inappropriate for distributed coherent virtual memory. © 1998 John Wiley & Sons, Ltd.

The LOCUS distributed operating system

LOCUS is a distributed operating system which supports transparent access to data through a network wide filesystem, permits automatic replication of storage, supports transparent distributed process execution, supplies a number of high reliability functions such as nested transactions, and is upward compatible with Unix. Partitioned operation of subnet's and their dynamic merge is also supported. The system has been operational for about two years at UCLA and extensive experience in its use has been obtained. The complete system architecture is outlined in this paper, and that experience is summarized.