Distributed Workflow System Research Articles

A science data gateway for environmental management

SummaryScience data gateways are effective in providing complex science data collections to the world‐wide user communities. In this paper we describe a gateway for the Advanced Simulation Capability for Environmental Management (ASCEM) framework. Built on top of established web service technologies, the ASCEM data gateway is specifically designed for environmental modeling applications. Its key distinguishing features include (1) handling of complex spatiotemporal data, (2) offering a variety of selective data access mechanisms, (3) providing state‐of‐the‐art plotting and visualization of spatiotemporal data records, and (4) integrating seamlessly with a distributed workflow system using a RESTful interface. ASCEM project scientists have been using this data gateway since 2011. Copyright © 2015 John Wiley & Sons, Ltd.

Research of the Combination of Distributed Business Processes Based on Dynamic Planning

In order to achieve the collaborative completion of entire business process by composition between different workflow management systems, a process composition method based on dynamic programming has been put forward. By analyzing differences and relations between process composition and Web services composition, many aspects have been analyzed with regard to service interface of the workflow system, the interaction control between the workflow systems, parameters and message’s delivery and the planning indexes according to the characteristics of process interaction among the distributed workflow systems. When the process is running, according to the indicators and expectations of users, the optimal workflow services and sub-processes would be selected through dynamic planning method, to provide support to the dynamic combination of workflow processes in a distributed environment.

Research of the Interconnection of Workflow System Based on Web Service

In order to achieve the interconnection between different workflow management systems, it was proposed that all the distributed workflow systems would be encapsulated as web services to perform the entire business process collaboratively by the way of processes’ composition in this paper. By analyzing the comparison between the composition of processes and ordinary Web service, we studied interactive control, the parameters required to be passed through the distributed workflow systems, the workflow system service’s interfaces and its packaging. Furthermore we put forward a general method of the workflow systems interactive interfaces’ extension and the way of the workflow service’s encapsulating and invoking. By this approach, it can easily combine the processes or process fragments which deployed on different workflow systems without other agents and components. It also provides support for the interconnection of the workflow systems in distributed environment, and ultimately achieves a coordinated operation between different workflow engines.

Alert Messaging in the CMS Distributed Workflow System

WMAgent is the core component of the CMS workload management system. One of the features of this job managing platform is a configurable messaging system aimed at generating, distributing and processing alerts: short messages describing a given alert-worthy information or pathological condition. Apart from the framework's sub-components running within the WMAgent instances, there is a stand-alone application collecting alerts from all WMAgent instances running across the CMS distributed computing environment. The alert framework has a versatile design that allows for receiving alert messages also from other CMS production applications, such as PhEDEx data transfer manager. We present implementation details of the system, including its Python implementation using ZeroMQ, CouchDB message storage and future visions as well as operational experiences. Inter-operation with monitoring platforms such as Dashboard or Lemon is described.

Integration of cloud-based services into distributed workflow systems: challenges and solutions

The paper introduces the challenges in modern workflow management in distributed environments spanning multiple cluster, grid and cloud systems. Recent developments in cloud computing infrastructures are presented and are referring how clouds can be incorporated into distributed workflow management, aside from local and grid systems considered so far. Several challenges concerning workflow definition, optimisation and execution are considered. These range from configuration, integration of business and scientific services, data management, dynamic monitoring and tracking, reusable workflow patterns, semantic search and distributed execution of distributed services. Finally, the author recommends a solution to these challenges based on the BeesyCluster middleware for distributed management of services with static and dynamic rescheduling within a market of services.

A model-driven workflow fragmentation framework for collaborative workflow architectures and systems

This paper focuses on a workflow distribution methodology for rationally deploying workflow models onto a distributed workflow system running on cloud computing environments, and we particularly lay a stress upon that those workflow systems operable on cloud computing environments are dubbed collaborative workflow systems, which are not only built upon the collaborative workflow architectures proposed in the paper, but pursuing the so-called collaborative computing paradigm characterized by focusing collaboration over cloud computing environments. The essential idea of the workflow distribution methodology is about how to fragment a workflow model and how to allocate its fragments to each of the architectural components configuring the underlying collaborative workflow architecture and system. As a reasonable solution to realize the essential idea, the paper proposes a model-driven workflow fragmentation framework, which provides a series of fragmentation algorithms that semantically fragmentate a workflow model by considering the semantic factors – performer, role, control-flow, data-flow, etc. – of the ICN-based workflow model as fragmentation criteria. The algorithms are classified into the vertical fragmentation approach, the horizontal fragmentation approach, and the hybrid approach of both. Conclusively, this paper conceives a possible set of collaborative workflow architectures embedding the collaborative computing paradigm, and describes the detailed formalism of the framework and about how the framework works on those collaborative workflow architectures and systems.

Holistic Production Optimization Achieved One Workflow at a Time

Technology Update Sustained production optimization, a longstanding objective within the exploration and production sector, has witnessed only incremental and sporadic advances. Obstacles to adoption of a holistic approach seem intransigent—an apparently daunting challenge to individual stakeholder groups. Yet, within the industry, there is a renewed sense of drive toward consistency, and growing consensus, on key components. Effective production-data management, model management, and real-time production-optimization systems are necessary but not sufficient. Integration of these components by means of orchestrating critical workflows—to provide a human/system interface for efficient organizational, operational, and technological collaboration—is required. To overcome inertia requires a reductionist approach to the problem. As the adage goes, we eat the elephant one bite at a time. Workflow orchestration within an overall scheme for integrated production operations is a pragmatic implementation approach, one now being applied successfully to a diverse range of projects. Production as a System of Interrelated Workflows Operational-excellence initiatives have improved individual workflows to reflect best practices. However, because of the degree of fragmentation inherent in many assets, these initiatives often remain inefficient. Cultural and operational influences significantly determine the degree to which an organization is either experience-driven or process-driven. The orchestration of technical workflows can play an important role in creating a system capable of meeting the necessary demands by means of capturing implicit expertise and automating many processes to a significant degree. Consider the components of a workflow in which data are analyzed and interpreted against some context (e.g., a model) antecedent to decision making. The degree of discontinuity experienced in a workflow cycle and its connectivity to the context of the entire production system determine the effectiveness and efficiency of the workflow. Thus, integrating these components and orchestrating the workflow cycle across an organization are necessary for improved optimization. The accurate definition of workflows and their interaction can be achieved by means of decomposing the main value-chain activities. Improving decline analysis or monthly economics forecasting may be of value; however, greater value is realized by combining these tasks into a prediction of the economic impact of a new decline curve. A holistic effort must be guided within the context of the system in a synchronized way that reflects the interrelatedness of the workflows. The next step change involves orchestration of work processes and their technology interfaces. Workflows reflect the needs of a particular basin, well, facility, operation, and organization, and workflows accurately reflect the entire value chain from reservoir to sales meter. Removal of workflow bottlenecks across the value chain improves the asset's overall effectiveness, thus adding value. Spanning discipline silos, systematizing knowledge capture, providing timely data to stakeholders, and streamlining the numerous human/system interfaces involved in a workflow improve cycle times and the quality of decisions, as well as minimizing the need for change management. Perhaps most significantly, orchestrated workflows leverage the expert human resources available to the asset. This distributed workflow system addresses the specific needs and challenges of operational and technical workflows.

The sqlLoader Data-Loading Pipeline

Using a database management system (DBMS) is essential to ensure the data integrity and reliability of large, multidimensional data sets. However, loading multiterabyte data into a DBMS is a time-consuming and error-prone task that the authors have tried to automate by developing the sqlLoader pipeline-a distributed workflow system for data loading.

분산 트랜잭션 워크플로우 모니터링 시스템의 설계 및 구현

This paper describes the design and implementation details of a distributed transactional workflow monitoring system. There have been prevalent research and development trends in the workflow literature - workflow systems tend to be completely distributed architectures to support very large-scale workflow applications on object-oriented and internet-based infrastructures. That is, the active (object), distributed (architecture), system-oriented (transaction), and large-scale (application) workflow systems are the key targets in terms of the research and development aspects. While the passive, centralized, human-oriented, and small/medium scale workflow systems are the typical instances of the traditional workflow systems. Unlike in the traditional (the client-server architecture) workflow systems, the workflow monitoring features should not be easily supported in the recent (the fully distributed architecture) workflow systems. At the same time, they need a set of additional monitoring features, such as gathering and displaying statistical (or overload status) information of the workflow architectural components dispersed on the internet. We, in this paper, introduce the additional workflow monitoring features that are necessarily required for the recent workflow systems, and show how to embed those features into a web-based distributed workflow system.

Extending RDF in distributed knowledge-intensive applications

The resource description framework (RDF) has become a formal language tool to specify the semantics of distributed systems, such as web services nowadays. In fact, it can also be extended to describe entities and relationships within specific application environments to support knowledge sharing and ontology construction. This paper presents two case studies on a network management knowledge model and a distributed workflow system process ontology. With practical experiences, the authors suggest how RDF can be applied innovatively and effectively to reengineer data integration solutions in different novel knowledge-intensive areas, which, in the past, were built upon traditional modelling languages, such as XML.

GM-WTA: An efficient workflow task allocation method in a distributed execution environment

A workflow is a collection of units of work called workflow tasks which cooperatively realize a business objective by utilizing system resources such as databases. The roles of workflow tasks are performed in the order driven by a computer representation of the workflow logic. During completing each task’s role, the remote control transfers and the remote resource accesses may often occur in a distributed workflow system. Hence, the efficient distribution of workflow components, especially workflow tasks, is so effective as to improve the performance of workflow processing. If we can place adjacent workflow tasks as close as possible and locate workflow tasks near to the required resources, we can significantly reduce the overhead of workflow processing. In this paper, we propose an efficient workflow task allocation method in a distributed workflow system, which is based on the locality principle. The method that utilizes the concept of graph partitioning can improve the performance of workflow processing by minimizing the remote processing costs incurred during workflow execution. In addition, we perform several experiments to evaluate our proposed method.

IntelliGEN: A Distributed Workflow System for Discovering Protein-Protein Interactions

A large genomics project involves a significant number of researchers and technicians performing dozens of tasks, either manual (e.g. performing laboratory experiments), computer assisted (e.g. looking for genes in the GENBANK database), or sometimes performed entirely automatically by the computer (e.g. sequence assembly). It has become apparent that managing such projects poses overwhelming problems and may lead to results of lower or even unacceptable quality, or possibly drastically increased project costs. In this paper, we present a design and an initial implementation of a distributed workflow system created to schedule and support activities in a genomics laboratory. The focus of the activities in the laboratory is the discovery of protein-protein interactions of fungi, specifically Neurospora crassa. We present our approach of developing, adapting and applying workflow technology in the genomics lab and illustrate it using one distinct part of a larger workflow to discover protein-protein interactions. Novel features of our system include the ability to monitor the quality and timeliness of the results and if necessary, suggesting and incorporating changes to the selected tasks and their scheduling.

Data management issues for large scale, distributed workflow systems on the Internet

The Internet is providing worldwide connectivity for business organizations and has created a much greater opportunity for people to collaborate remotely in an automated workflow setting. However, the speed of delivery of workflow data on the Internet can be unpredictable due to variability in traffic and limited bandwidth. Further, the issue of data management becomes more complex when the system consists of various clients with different system and networking settings. In this paper, we investigate several data management problems in distributed workflow management on the Internet and propose solutions for improving workflow efficiency. More specifically, we explore research issues with respect to system architectures and various data delivery techniques. Our study indicates that it is possible to improve performance considerably by selecting the right combinations of data delivery techniques and system characteristics.

Object‐Oriented Protocol Hierarchies for Distributed Workflow Systems

Modern information systems will increasingly have to support the cooperation between more or less independent partners that are distributed across local or global networks. Since the partners can naturally be modeled by objects, means must be provided that permit to model the cooperation among objects. In this paper we propose an extended object model which allows the explicit definition of dynamic relationships between objects. The new construct, called alliances, can be viewed as a protocol that materializes and enforces the cooperation of an arbitrary number of objects by relying on a set of communication rules. The rule‐based approach allows to specify cooperation protocols which circumscribe a space of possible cooperations. In other words, cooperation protocols restrict cooperation among objects to the minimum extent needed to leave the objects enough leeway for spontaneous reactions while still guaranteeing that the constraints inherent in a cooperation on a common task are being met. Workflows are specifications of cooperative applications. We demonstrate how alliances can be used as an implementation model for distributed workflow systems, which is superior to a pure object‐based implementation. This suggests that alliances are a lower‐level, more implementation‐oriented support structure for workflows. In particular, it should be possible to derive the cooperation constraints of an alliance from higher‐level workflow specifications which model business processes. Alliances can in turn be implemented on top of common distributed object technology which for its part relies on standard database and communication services.