Workflow Management Framework Research Articles

Scanflow: A multi-graph framework for Machine Learning workflow management, supervision, and debugging

Machine Learning (ML) is more than just training models, the whole workflow must be considered. Once deployed, a ML model needs to be watched and constantly supervised and debugged to guarantee its validity and robustness in unexpected situations. Debugging in ML aims to identify (and address) the model weaknesses in not trivial contexts. Several techniques have been proposed to identify different types of model weaknesses, such as bias in classification, model decay, adversarial attacks, etc., yet there is not a generic framework that allows them to work in a collaborative, modular, portable, iterative way and, more importantly, flexible enough to allow both human- and machine-driven techniques. In this paper, we propose a novel containerized directed graph framework to support and accelerate end-to-end ML workflow management, supervision, and debugging. The framework allows defining and deploying ML workflows in containers, tracking their metadata, checking their behavior in production, and improving the models by using both learned and human-provided knowledge. We demonstrate these capabilities by integrating in the framework two hybrid systems to detect data drift distribution which identify the samples that are far from the latent space of the original distribution, ask for human intervention, and whether retrain the model or wrap it with a filter to remove the noise of corrupted data at inference time. We test these systems on MNIST-C, CIFAR-10-C, and FashionMNIST-C datasets, obtaining promising accuracy results with the help of human involvement.

Dynamically Partitioning Workflow over Federated Clouds for Optimising the Monetary Cost and Handling Run-Time Failures

Several real-world problems in domain of healthcare, large scale scientific simulations, and manufacturing are organised as workflow applications. Efficiently managing workflow applications on the Cloud computing data-centres is challenging due to the following problems: (i) they need to perform computation over sensitive data (e.g., Healthcare workflows) hence leading to additional security and legal risks especially considering public cloud environments and (ii) the dynamism of the cloud environment can lead to several run-time problems such as data loss and abnormal termination of workflow task due to failures of computing, storage, and network services. To tackle above challenges, this paper proposes a novel workflow management framework call Deploy on Federated Cloud Framework (DoFCF) that can dynamically partition scientific workflows across federated cloud (public/private) data-centres for minimising the financial cost, adhering to security requirements, while gracefully handling run-time failures. The framework is validated in cloud simulation tool (CloudSim) as well as in a realistic workflow-based cloud platform (e-Science Central). The results showed that our approach is practical and is successful in meeting users security requirements and reduces overall cost, and dynamically adapts to the run-time failures.

GNOMO: a multi-omics pipeline for integrated host and microbiome analysis of non-model organisms.

The study of bacterial symbioses has grown exponentially in the recent past. However, existing bioinformatic workflows of microbiome data analysis do commonly not integrate multiple meta-omics levels and are mainly geared toward human microbiomes. Microbiota are better understood when analyzed in their biological context; that is together with their host or environment. Nevertheless, this is a limitation when studying non-model organisms mainly due to the lack of well-annotated sequence references. Here, we present gNOMO, a bioinformatic pipeline that is specifically designed to process and analyze non-model organism samples of up to three meta-omics levels: metagenomics, metatranscriptomics and metaproteomics in an integrative manner. The pipeline has been developed using the workflow management framework Snakemake in order to obtain an automated and reproducible pipeline. Using experimental datasets of the German cockroach Blattella germanica, a non-model organism with very complex gut microbiome, we show the capabilities of gNOMO with regard to meta-omics data integration, expression ratio comparison, taxonomic and functional analysis as well as intuitive output visualization. In conclusion, gNOMO is a bioinformatic pipeline that can easily be configured, for integrating and analyzing multiple meta-omics data types and for producing output visualizations, specifically designed for integrating paired-end sequencing data with mass spectrometry from non-model organisms.

Dynamic Multilevel Workflow Management Concept for Industrial IoT Systems

Workflow management is implemented in manufacturing at many levels. The nature of processes varies at each level, hindering the use of a standard modeling or implementation solution. The creation of a flexible workflow management framework that overarches the heterogeneous business process levels is challenging. Still, one of the promises of the Industry 4.0 initiative is precisely this: to provide easy-to-use models and solutions that enable efficient execution of enterprise targets. By addressing this challenge, this article proposes a workflow execution model that integrates information and control flows of these levels while keeping their hierarchy. The overall model builds on the Business Process Model and Notation (BPMN) for modeling at the enterprise level and recipe modeling based on colored Petri net (CPN) at the production level. Models produced with both alternatives are implemented and executed in a framework supported by an Enterprise Service Bus (ESB). Loosely coupled, late-bound system elements are connected through the Arrowhead framework, which is built upon the Service-Oriented Architecture (SOA) concept. To prove its feasibility, this article presents the practical application of the model via an automotive production scenario. Note to Practitioners —The methodology detailed in this article can serve as a basis for experts who are dealing with industrial workflows. Reacting to the requirements of Industry 4.0, i.e., the virtualization, decentralization, modularity, real-time capability, and service orientation, this article provides a concept that can answer all the defined criteria. First, it adopts a new two-level approach to workflow management, which makes the understanding and control of workflows easier, enhancing transparency. Furthermore, it demonstrates how—even completely different—applications and modeling languages can be integrated into a Service-Oriented Architecture (SOA). The presented composition and the used tools are all tried and tested. Behind the solution described in this article, there is a genuine, working code wherewith the presented end-to-end workflow management can be achieved. Following the methodology detailed in this article, the readers can construct their workflow management composition. In order to report on the performance of the created solution, this article presents different measurement compositions that allow the investigation of the essential components separately, demonstrating the scalability and temporal parameters.

An ontology-driven framework to support the dynamic formation of an interdisciplinary healthcare team

Background and purposeTeamwork has become a modus operandi in healthcare and delivery of patient care by an interdisciplinary healthcare team (IHT) is now a prevailing modality of care. We argue that a formal and automated support framework is needed for an IHT to properly leverage information technology resources. Such a framework should allow for patient preferences and expand a representation of a clinical workflow with a formal model of dynamic formation of a team, especially with regards to team leader- and membership, and the assignment of tasks to team members. Our goal was to develop such a support framework, present its prototype software implementation and verify the implementation using a proof-of-concept use case. Specifically, we focused on clinical workflows for in-patient tertiary care and on patient preferences with regards to selecting team members and team leaders. Materials and methodsDrawing on the research on clinical teamwork we defined the conceptual foundations for the proposed framework. Then, we designed its architecture and used ontology-driven design and first-order logic with associated reasoning methods to create and operationalize architectural elements. Finally, we incorporated existing solutions for business workflow modeling and execution as a backend for implementing the proposed framework. ResultsWe developed a Team and Workflow Management Framework (TWMF) with semantic components that allow for formalizing and operationalizing team formation in in-patient tertiary care setting and support provider-related patient preferences. We also created a prototype software implementation of TWMF using the IBM Business Process Manager platform. This implementation was evaluated in several simulated patient scenarios. ConclusionsTWMF integrates existing workflow technologies and extends them with the capabilities to support dynamic formation of an IHT. Results of this research can be used to support real-time execution of clinical workflows, or to simulate their execution in order to assess the impact of various conditions (e.g., patterns of work shifts, staffing) on IHT operations.

Activity prediction in process mining using the WoMan framework

Process Management techniques are useful in domains where the availability of a (formal) process model can be leveraged to monitor, supervise, and control a production process. While their classical application is in the business and industrial fields, other domains may profitably exploit Process Management techniques. Some of these domains (e.g., people’s behavior, General Game Playing) are much more flexible and variable than classical ones, and, thus, raise the problem of predicting which activities will be carried out next, a problem that is not so compelling in classical fields. When the process model is learned automatically from examples of process executions, which is the task of Process Mining, the prediction performance may also provide indirect indications on the correctness and reliability of the learned model. This paper proposes and compares two strategies for activity prediction using the WoMan framework for workflow management. The former proved to be able to handle complex processes, the latter is based on the classic and consolidated Naive Bayes approach. An experimental validation allows us to draw considerations on the pros and cons of each, used both in isolation and in combination.

Adaptive business rules framework for workflow management

PurposeChanging scattered and dynamic business rules in business workflow systems has become a growing problem that hinders the use and configuration of workflow-based applications. There is a gap in the existing research studies which currently focus on solutions that are application specific, without accounting for the universal logical dependencies between the business rules and, as a result, do not support adaptation of the business rules in real time. The paper aims to discuss this issue.Design/methodology/approachTo tackle the above problems, this paper adopts a bottom-up approach, which puts forward a component model of the business process workflows and then adds business rules which have clear logical semantics. This allows incremental development of the workflows and semantic indexing of the rules which govern them during the initial acquisition.FindingsThe paper introduces an event-driven model for development of business workflows which is purely logic-based and can be easily implemented using an object-oriented technology, together with a model of the business rules dependencies which supports incremental semantic indexing. It also proposes a two-level inference mechanism as a vehicle for controlling the business process execution and the process of adaptation of the business rules at real time based on propagating the dependencies.Research limitations/implicationsThe framework is strictly logical and completely domain-independent. It allows to account both synchronous and asynchronous triggering events as well as both qualitative and quantitative description of the conditions of the rules. Although our primary interest is to apply the framework to the business processes typical in the construction industry we believe our approach has much wider potential due to its strictly logical formalization and domain independence. In fact it can be used to control any business processes where the execution is governed by rules.Practical implicationsThe framework could be applied to both large business process modelling tasks and small but very dynamic business processes like the typical digital business processes found in online banking or e-Commerce. For example, it can be used for adjusting security policies by adding the capability to adapt automatically the access rights to account for additional resources and new channels of operation which can be very interesting ion both B2C and B2B applications.Social implicationsThe potential scope of the impact of the research reported here is linked to the wide applicability of rule-based systems in business. Our approach makes it possible not only to control the execution of the processes, but also to identify problems in the control policies themselves from the point of view of their logical properties – consistency, redundancies and potential gaps in the logics. In addition to this, our approach not only increases the efficiency, but also provides flexibility for adaptation of the policies in real time and increases the security of the overall control which improves the overall quality of the automation.Originality/valueThe major achievement reported in this paper is the construction of a universal, strictly logic-based event-driven framework for business process modelling and control, which allows purely logical analysis and adaptation of the business rules governing the business workflows through accounting their dependencies. An added value is the support for object-oriented implementation and the incremental indexing which has been possible thanks to the bottom-up approach adopted in the construction of the framework.

Report on the first workshop on negative and null results in eScience

Report on the first workshop on negative and null results in eScience

SCW+: A Service Oriented Framework for Cloud Workflow Systems

Cloud workflow is a special type of cloud computing systems which mainly concentrates on workflow management. One of the major issues with cloud workflow systems is automatic multi-cloud workflow management. This paper proposes a service oriented framework for cloud workflow management which integrates heterogeneous multi-cloud platforms to provide integrated applications for users by minimizing the human intervention as far as possible. The proposed framework involves a language to define some basic entities for environments and uses these definitions to integrate applications and services in a cloud workflow. This framework has been already implemented. In addition, its main operations have been evaluated by a case study and the results show that the framework works properly as a cloud integrator, and the framework main activities are done automatically with a reasonable performance.

Towards a framework for standardized semantic workflow modeling and management in the surgical domain

Abstract An essential aspect for workflow management support in operating room environments is the description and visualization of the underlying processes and activities in a machine readable format as Surgical Process Models (SPM). However, the process models often vary in terms of granularity, naming and representation of process elements and their modeling structure. The aim of this paper is to present a new methodology for standardized semantic workflow modeling and a framework for semantic work-flow execution and management in the surgical domain.

Enabling scalable scientific workflow management in the Cloud

Cloud computing is gaining tremendous momentum in both academia and industry. In this context, we define the term “Cloud Workflow” as the specification, execution and provenance tracking of large-scale scientific workflows, as well as the management of data and computing resources to support the execution of large-scale scientific workflows in the Cloud. In this paper, we first analyze the gap between these two complementary technologies, and what it means to bring Clouds and workflows together. Then, we present the key challenges in supporting Cloud workflows, and present our reference framework for scientific workflow management in the Cloud. Last we present our experience in integrating a scientific workflow management system—Swift into the Cloud. We discuss the performance of cluster provisioning within the OpenNebula Cloud platform, the Eucalyptus Cloud platform and Amazon EC2, and we demonstrate the capability and efficiency of the integration using a NASA MODIS image processing workflow and the Montage image mosaic workflow.Note to practitioners. Scientific workflow management plays a very important role for scientific computing and application coordination, while Cloud computing offers scalability and resource on-demand. We devise autonomous methods to integrate scientific workflow management systems with Cloud platforms and also provision resources for large scale workflows, which can facilitate scientists to easily manage their workflows in the Cloud, and take advantage of large scale Cloud resources. There are a few integration options and many challenges in the process, and the experience we gain will help researchers in migrating their workflow management systems and workflow applications into the Cloud.

WoMan: Logic-Based Workflow Learning and Management

Workflow management is fundamental to efficiently, effectively, and economically carry out complex working and domestic activities. Manual engineering of workflow models is a complex, costly, and error-prone task. The WoMan framework for workflow management is based on first-order logic. Its core is an automatic procedure that learns and refines workflow models from observed cases of process execution. Its innovative peculiarities include incrementality (allowing quick learning even in the presence of noise and changed behavior), strict adherence to the observed practices, ability to learn complex conditions for the workflow components, and improved expressive power compared to the state of the art. This paper presents the entire algorithmic apparatus of WoMan, including translation and learning from a standard log format for case representation, import/export of workflow models from/into standard formalisms (Petri nets), and exploitation of the learned models for process simulation and monitoring. Qualitative and quantitative experimental evaluation shows the power and efficiency of WoMan, both in controlled and in real-world domains.

Revised Framework for ETL Workflow Management for Efficient Business Decision-Making

Business decision-making is not a simple task. There are many reasons for that but the main reason is data comes from heterogeneous operational sources of an organization. Therefore, it is difficult to organize and maintain especially if a huge volume of data is involved. A data warehouse is helpful in this regard as it can assists in business decision-making. Data collection and loading it into a data warehouse is difficult job because data sources are not in consistent form. This job usually consists of three main processes that involve extraction, transformation and loading. To extract the data from different sources, then transform it into a unified format and consequently load it into the warehouse, ETL (Extract, transform and load) tools are required. Nowadays, the majority of ETL tools organize workflow. An ETL workflow can be considered as a group of ETL jobs with dependencies between them. In this research paper a revised ETL workflow management framework which is based upon different considerations is proposed. These considerations along with the addition of the components in the workflow scheduling layer would help in making more effective and quality business decisions.

A Workflow Management Framework Based on Peer-to-Peer Grid Environment

A Workflow Management Framework Based on Peer-to-Peer Grid Environment

Fuzzy Multiple Criteria Workflow Robustness and Resiliency Modeling with Petri Nets

The increasing complexity and tight coupling between people and computer systems in military operations has led to improved efficiency, as well as greater vulnerability due to system failure. Careful management of workflow systems can minimize operational vulnerability in command and control. Tavana et al. (2011) developed a workflow management framework capable of both modeling structure and providing a wide range of quantitative analysis with high-level Petri nets (PNs). The framework is based on a sustainability index that captures the concepts of self-protecting and self-healing systems. This index uses crisp numerical values to measure the robustness and resiliency of the system. However, the observed values of data in real-world military operations are often imprecise or vague. These inexact data can be represented by fuzzy numbers to reflect the decision makers’ intuition and subjective judgments. In this paper, the authors extend this model to a fuzzy framework by proposing a new fuzzy workflow modeling system with PNs. The new model plots the fuzzy robustness and resiliency measures in a Cartesian coordinate system and derives an overall fuzzy sustainability index for the system based on the theory of displaced ideals. The proposed model also considers multiple criteria to produce this fuzzy index.

Semantics-based composition of EMBOSS services

BackgroundMore than in other domains the heterogeneous services world in bioinformatics demands for a methodology to classify and relate resources in a both human and machine accessible manner. The Semantic Web, which is meant to address exactly this challenge, is currently one of the most ambitious projects in computer science. Collective efforts within the community have already led to a basis of standards for semantic service descriptions and meta-information. In combination with process synthesis and planning methods, such knowledge about types and services can facilitate the automatic composition of workflows for particular research questions.ResultsIn this study we apply the synthesis methodology that is available in the Bio-jETI workflow management framework for the semantics-based composition of EMBOSS services. EMBOSS (European Molecular Biology Open Software Suite) is a collection of 350 tools (March 2010) for various sequence analysis tasks, and thus a rich source of services and types that imply comprehensive domain models for planning and synthesis approaches. We use and compare two different setups of our EMBOSS synthesis domain: 1) a manually defined domain setup where an intuitive, high-level, semantically meaningful nomenclature is applied to describe the input/output behavior of the single EMBOSS tools and their classifications, and 2) a domain setup where this information has been automatically derived from the EMBOSS Ajax Command Definition (ACD) files and the EMBRACE Data and Methods ontology (EDAM). Our experiments demonstrate that these domain models in combination with our synthesis methodology greatly simplify working with the large, heterogeneous, and hence manually intractable EMBOSS collection. However, they also show that with the information that can be derived from the (current) ACD files and EDAM ontology alone, some essential connections between services can not be recognized.ConclusionsOur results show that adequate domain modeling requires to incorporate as much domain knowledge as possible, far beyond the mere technical aspects of the different types and services. Finding or defining semantically appropriate service and type descriptions is a difficult task, but the bioinformatics community appears to be on the right track towards a Life Science Semantic Web, which will eventually allow automatic service composition methods to unfold their full potential.

Collaborative Workflow Management for Interagency Crime Analysis

To strengthen homeland security, there is a critical need for new tools that can facilitate real time collaboration among various law enforcement agencies. Through a field study, we find that law enforcement work is knowledge intensive and involves complex collaborative processes interrelating a large number of disparate units in a loosely defined virtual organization. To support knowledge intensive collaboration, we propose a new workflow centric framework to seamlessly integrate previously separate techniques from the fields of information retrieval and workflow management. In this paper, we develop a collaborative workflow management framework for interagency crime analysis. The key contribution of our research is that by integrating various state-of-the-art techniques innovatively, the proposed system can support real time collaboration processes in a virtual organization that evolves dynamically.

Enterprise frameworks for workflow management systems

AbstractThis paper proposes an enterprise framework for workflow management (WFM) systems based on the Workflow Management Coalition generic architecture. In addition, it presents one smaller object‐oriented framework for task scheduling. This framework is obtained by refining one of the packages of the overall framework. The task scheduling framework was designed following the concept of model framework as in Catalysis. This sort of framework can be used as a template to generate code components within an application domain. According to this approach we can develop WFM components that can interoperate within a well‐defined enterprise framework. This paper also presents the process of developing interoperable code components from model frameworks. Copyright © 2002 John Wiley & Sons, Ltd.

Temporal workflow management in a claim handling system

Temporal workflow management is important for processes that are time-driven. Claim handling, which requires the documentation, diagnosis, and resolution of customer claims due to faulty products or unsatisfactory services, is an example of such a process because fast turnaround is critical for customer satisfaction. However, little research has been reported in this area, especially at the policy level. In this paper, we develop a framework for temporal workflow management, which includes issues such as turnaround time predication, time allocation, and task prioritization. We propose also the use of reward functions to guide workers' behavior with the goal of increasing efficiency while allowing flexibility.

FlowBack

The Distributed Systems Technology Centre (DSTC) framework for workflow specification, verification and management captures workflows transaction-like behavior for long lasting processes. FlowBack is an advanced prototype functionally enhancing an existing workflow management system by providing process backward recovery. It is based on extensive theoretical research ([3],[4],[5],[6],[8],[9]), and its architecture and construction assumptions are product independent. FlowBack clearly demonstrates the extent to which generic backward recovery can be automated and system supported. The provision of a solution for handling exceptional business process behavior requiring backward recovery makes workflow solutions more suitable for a large class of applications, therefore opening up new dimensions within the market. For the demonstration purpose, FlowBack operates with IBM FlowMark, one of the leading workflow products.