Workflow Structures Research Articles

Towards a taxonomy of team workflow structures

Team workflow represents interactions between individuals and specific actions or tasks. Individuals’ interactions have important effects on fellow teammates’ actions by expanding or constraining actions available to them. For example, teammates may avoid performing the same action to avoid duplication of effort or they may perform their actions sequentially if one task’s completion is a prerequisite for another task. Complex dependencies embedded in these interactions suggest the need to understand team workflows from a relational perspective. As workflow structures are shaped by elements of organizational design, cognitive factors, and features of the task environment, no single workflow structure is optimal for all teams, and team workflows may manifest in countless distinct configurations. Through a systematic, network-based representation of team workflows, this paper uses a sample of 139,500 teams on GitHub to identify common patterns of team workflows. Each team is represented as a two-mode network where individuals form ties to up to fifteen distinct actions capturing productivity, discussion, and team management. Several node-level and graph-level centrality indices highlight patterns of differentiation across team workflows, and a k-means clustering algorithm detects three distinct clusters of team workflow structures: small teams of highly active generalists, small teams with a moderately active mix of focused and generalist members, and large, segmented teams of focused individuals collectively engaging in a few extremely popular actions. These results demonstrate how a structural representation of team workflows provides unique insight into team behavior and highlights distinctions that may otherwise be lost when examining team activity in aggregate.

Dynamic variable analysis guided adaptive evolutionary multi-objective scheduling for large-scale workflows in cloud computing

Energy consumption and makespan of workflow execution are two core performance indicators in operating cloud platforms. But, simultaneously optimizing these two indicators encounters various challenges, such as elastic resources, large-scale decision variables, and sophisticated workflow structures. To handle these challenges, we design an adaptive evolutionary scheduling algorithm, namely AESA, with three innovative strategies. First, a heuristic population initialization strategy is devised to gather workflow tasks onto limited potential resources, thereby alleviating the negative impact of redundant cloud resources on evolutionary search efficiency. Then, a variable analysis strategy is designed to dynamically measure the contribution of each decision variable in pushing the population towards Pareto-optimal fronts. Moreover, AESA embraces an adaptive strategy to reward more evolutionary opportunities for decision variables with higher contributions to handle large-scale decision variables in a targeted manner, further improving the efficiency of evolutionary search. Finally, extensive experiments are performed based on real-world cloud platforms and workflow traces to verify the effectiveness of the proposed AESA. The comparison results validate its superior performance by significantly outperforming five representative baselines in optimizing makespan and energy consumption. Also, the results of ablation experiments demonstrate that all three components contribute to AESA’s overall performance, with the adaptive reward mechanism being the most significant.

Efficient Workflow Scheduling in Edge Cloud-Enabled Space-Air-Ground-Integrated Information Systems

To address the challenges posed by the dynamism, high latency, and resource scarcity in integrated air-space-ground hybrid edge cloud environments on task completion times and node load, we designed a task scheduling system for scenarios involving the transmission and processing of interdependent tasks. This system integrates a graph neural network with attention mechanism and deep reinforcement learning. Specifically, we employ a graph encoder to extract features from DAG tasks and resources. Task scheduling solutions for dynamic environments are then generated using attention mechanism-equipped graph decoder, which are subsequently optimized based on performance metrics through the use of an Advantage Actor-Critic algorithm. Experimental results indicate that this algorithm performs well in terms of completion time and node load balance across tasks with different workflow structures, demonstrating its adaptability to highly dynamic edge cloud environments.

Grammar-based evolutionary approach for automated workflow composition with domain-specific operators and ensemble diversity

The process of extracting valuable and novel insights from raw data involves a series of complex steps. In the realm of Automated Machine Learning (AutoML), a significant research focus is on automating aspects of this process, specifically tasks like selecting algorithms and optimising their hyper-parameters. A particularly challenging task in AutoML is automatic workflow composition (AWC). AWC aims to identify the most effective sequence of data preprocessing and machine learning algorithms, coupled with their best hyper-parameters, for a specific dataset. However, existing AWC methods are limited in how many and in what ways they can combine algorithms within a workflow.Addressing this gap, this paper introduces EvoFlow, a grammar-based evolutionary approach for AWC. EvoFlow enhances the flexibility in designing workflow structures, empowering practitioners to select algorithms that best fit their specific requirements. EvoFlow stands out by integrating two innovative features. First, it employs a suite of genetic operators, designed specifically for AWC, to optimise both the structure of workflows and their hyper-parameters. Second, it implements a novel updating mechanism that enriches the variety of predictions made by different workflows. Promoting this diversity helps prevent the algorithm from overfitting. With this aim, EvoFlow builds an ensemble whose workflows differ in their misclassified instances.To evaluate EvoFlow’s effectiveness, we carried out empirical validation using a set of classification benchmarks. We begin with an ablation study to demonstrate the enhanced performance attributable to EvoFlow’s unique components. Then, we compare EvoFlow with other AWC approaches, encompassing both evolutionary and non-evolutionary techniques. Our findings show that EvoFlow’s specialised genetic operators and updating mechanism substantially outperform current leading methods in predictive performance. Additionally, EvoFlow is capable of discovering workflow structures that other approaches in the literature have not considered.

Towards a Time Data Library to Enable Process Optimization in Engineer-to-Order Industries: The Case of Control and Switchgear Manufacturing

This study addresses the crucial role of Time Data Management (TDM) in switchgear and control cabinet manufacturing, sectors integral to industrial automation and characterized by custom-made or small-batch production. The research highlights the challenges in accurately forecasting production times in these domains, where products often have unique, non-repetitive characteristics. This study systematically categorizes production to ensure precise TD collection and analysis. A possible approach to logically divide control cabinet production into distinct manufacturing areas and workflow structures is presented, facilitating a multiple case study method and a comparative TD analysis across different companies. The contribution of this study is threefold: providing detailed workflow structures for an under-researched industry, offering empirical data for academic and practical assessment, and enabling efficient TDM in ETO industries, traditionally hampered by process measurement difficulties and reliance on expert time estimates. This research promises significant advancements in TDM, ensuring more accurate production planning, and cost calculation, and enhancing market competitiveness in switchgear and control cabinet manufacturing.

Predictors of Clinical Nurse Leader implementation success across a national sample of settings: A Bayesian multilevel modeling analysis.

The Clinical Nurse Leader (CNL) care model is a different way of organizing frontline nursing care delivery in contrast to the traditional "staff nurse" model and is increasingly being adopted by health systems across the United States and abroad. However, variability in implementation and outcomes has been noted across health settings. A recently validated CNL Practice Model provides an explanatory pathway for CNL model integration into practice. The purpose of this study was to identify and compare patterns of empirical correspondence to the CNL Practice Model and predict their influence on implementation success. We conducted a secondary analysis of a 2015 national-level study with clinicians and administrators involved with CNL initiatives in their health system. A psychometrically validated CNL Practice Survey was used to collect data measuring the presence (0%-100%) of the five domains of the CNL Practice Model (organizational readiness, CNL structuring, CNL practices, outcomes, and value) and one measure of CNL implementation success. We modeled the complex hierarchical structure of the data using a Bayesian multilevel regression mixed modeling approach. A zero-one-inflated beta distribution, a mixture of Bernoulli distributions for the minimum and maximum responses and a beta distribution for the responses between the minimum and maximum, was used to fit success ratings in the model. A total of 920 participants responded, 540 (59%) provided success scores. The model captured ratings skewed toward upper bound, while also adequately modeling data between the minimum and maximum values. The Bayesian model converged and gave estimates for all hierarchical parameters, which would likely have failed to converge in a pure maximum likelihood framework. The variability around success score across CNL Practice Model element ratings was greatest at the component level, 0.29 (0.18-0.48), compared to either the domain level, 0.16 (0.01-0.54), or the item level, 0.09 (0.01-0.17). The components most predictive of implementation success were (a) consensus CNL model can close gaps, (b) organization level implementation strategy, and (c) alignment of empirical CNL microsystem level structuring to the model's conceptualization. Findings provide further empirical evidence to support the explanatory pathway proposed by the CNL Practice Model and identified specific organizational readiness and CNL workflow structures that are critical antecedents predictive of CNL practice manifestation and production of expected outcomes. Findings indicate actionable implementation evidence that can be successfully adopted across real-world healthcare settings to achieve safer and higher quality patient care. CNL integrated care delivery is a frontline nursing care model that is being increasingly adopted by health systems across the United States and abroad. However, variability in CNL implementation and outcomes has been noted across health settings, limiting its evidence base. Findings of this study contribute a better understanding about the variability of CNL practice and outcomes found in the literature and contribute empirical and conceptual clarity about the relationships between modes of CNL implementation and successful adoption in healthcare settings.

Energy-aware intelligent scheduling for deadline-constrained workflows in sustainable cloud computing

It is challenging to handle the non-linear power consumption model, complex workflow structures, and diverse user-defined deadlines for energy-efficient workflow scheduling in sustainable cloud computing. Although metaheuristics are very attractive to solve this problem, most of the existing work regards the problem as a black-box and ignores the use of domain knowledge. To make up for their shortcomings, this paper tailors an energy-aware intelligent scheduling algorithm (EIS) with three new mechanisms. First, we derive the optimal execution time that minimizes energy consumption for each task on a given resource. Second, based on the optimal execution time of each workflow task, the EIS distributes the workflow slack time (difference between its completion time and deadline) to reduce the voltages and frequencies of task executions for energy saving. Third, the EIS mines the idle time gaps caused by task precedence constraints to further reduce dynamic energy consumption whilst satisfying workflows’ deadline constraints. To measure the performance of the EIS, we conduct extensive comparison experiments based on actual workflow applications. The results demonstrate that the energy consumption of the EIS is much lower than that of the competitors under different deadlines, and has a faster descend rate with the evolution process.

Knowledge-Based Evolutionary Optimizing Makespan and Cost for Cloud Workflows

Workflow scheduling is essential to simultaneously optimize the makespan and economic cost for cloud services and has attracted intensive interest. Most of the existing multi-objective cloud workflow scheduling algorithms regard the focused problems as black-boxes and design evolutionary operators to perform random searches, which are inefficient in dealing with the elasticity and heterogeneity of cloud resources as well as complex workflow structures. This study explores the characteristics of cloud resources and workflow structures to design a knowledge-based evolutionary optimization operator, named KEOO, with two novel features. First, we develop a task consolidation mechanism to reduce the number of cloud resources used, reducing the economic cost of workflow execution without delaying its finish time. Then, we develop a critical task adjustment mechanism to selectively move the critical predecessors of some tasks to the same resources to eliminate the data transmission overhead between them, striving to improve the economic cost and finish time simultaneously. At last, we embed the proposed KEOO into four classical multi-objective algorithms, i.e., NSGA-II, HypE, MOEA/D, and RVEA, forming four variants: KEOO-NSGA-II, KEOO-HypE, KEOO-MOEA/D, and KEOO-RVEA, for comparative experiments. The comparison results demonstrate the effectiveness of the KEOO in improving these four algorithms in solving cloud workflow scheduling problems.

Flow past two tandem square cylinders placed in parallel walls

In this work, pulsating and non-pulsating flow structures past two tandem square cylinders placed in parallel walls was investigated. The variation of the distance between the cylinders and the frequency of flow pulsation was also examined. The Particle Image Velocimetry (PIV) method was applied to reveal hydrodynamic characteristics such as streamlines patterns, <Ψ>, the velocity vector distribution, the vorticity field, <ω>, the normalized streamwise velocity contours, <U/Uavg>, and turbulent statistics such as streamwise velocity fluctuations, urms, cross-streamwise velocity fluctuations, vrms, and turbulent Reynolds stress, u′v′‾/U02 distributions for non-pulsating flow cases. In addition, instantaneous and phase averaged flow data over the flow field is presented in order to evaluate the influence of flow pulsation on the quantitative analyses. The experiments were performed at Re=500. According to the results, the confinement profoundly affects the flow regime past two tandem cylinders. The alteration from the reattach flow to the co-shedding vortex flow regime take place at a lower gap ratio. Moreover, flow pulsation considerably contributes to mainstream penetration into the gap between cylinders. The mechanism of the fluid interactions near the wall and mainstream is highly dependent on the ratio of L/D and pulsation frequency.

Resource scheduling and provisioning for processing of dynamic stream workflows under latency constraints

There has been an increasing demand for large-scale processing of stream workflows. Many streaming data sources, such as video and audio, differ in terms of the type and density of information they contain over time, which require dynamic workflow structures where paths change in response to data variations. Additionally, information is often only inferred from examining discrete chunks and hence, cannot be arbitrarily parallelised like other streaming data sources. Stream processes using these data sources often operate as dynamic workflows, where different paths are selected based on prior processing information, relating to differences in the nature of the data itself. To better enable low latency analysis in such scenarios, this work investigates strategies to schedule and provision streaming dynamic workflow scenarios of various complexities and degrees of unpredictability. It is found that, as scenarios become increasingly uncertain and unpredictable, it is better to use simpler strategies which do not make assumptions about workflow structures or completion times.

Automatic construction of accurate bioacoustics workflows under time constraints using a surrogate model

Automated bioacoustics analysis is being increasingly used to describe environmental phenomena such as species abundance and biodiversity. Within this research area, many algorithms have been proposed. These achieve different sub-objectives within bioacoustics processes and can be combined to form workflows. However, these algorithms are typically evaluated in a limited number of scenarios and are rarely evaluated with different combinations of other tasks. This can result in workflows that are not well optimised to serve a given scenario, particularly under resource and time constraints, which ultimately leads to inaccurate bioacoustics analyses. This work examines the problem of bioacoustics workflow construction by searching and ordering combinations of tasks to determine which produce the most accurate output while remaining under user-defined time constraints. Workflow construction is investigated within a scenario where species need to be classified within synthetically generated soundscapes with different numbers of species, noise levels, and densities of species. A search algorithm is created that applies Particle Swarm Optimisation (PSO) to a neural network-based surrogate model. This algorithm is used to efficiently search for candidate workflow structures. This is compared to a random search, a genetic algorithm, and a PSO algorithm without the surrogate model, as well as existing workflows based on previous research. It is found that for all scenarios, the surrogate model-based search method can quickly find effective workflows in a low number of searches. Furthermore, it is found that workflow effectiveness varies depending on the scenarios and recordings used.

The Workflow Trace Archive: Open-Access Data From Public and Private Computing Infrastructures

Realistic, relevant, and reproducible experiments often need input traces collected from real-world environments. In this work, we focus on traces of workflows-common in datacenters, clouds, and HPC infrastructures. We show that the state-of-the-art in using workflow-traces raises important issues: (1) the use of realistic traces is infrequent and (2) the use of realistic, open-access traces even more so. Alleviating these issues, we introduce the Workflow Trace Archive (WTA), an open-access archive of workflow traces from diverse computing infrastructures and tooling to parse, validate, and analyze traces. The WTA includes > 48 million workflows captured from > 10 computing infrastructures, representing a broad diversity of trace domains and characteristics. To emphasize the importance of trace diversity, we characterize the WTA contents and analyze in simulation the impact of trace diversity on experiment results. Our results indicate significant differences in characteristics, properties, and workflow structures between workload sources, domains, and fields.

Strategic approaches for structuring investment securities operations

The investment securities operations world changed significantly post the 2008 crisis with the increased requirements of market infrastructure and regulations and the changing investment strategies to navigate market conditions for earning returns. Investment middle offices had to both redesign and implement operating workflow structures that could meet and achieve the regulatory requirements and support the front-office investment management strategies. This paper will examine the approaches undertaken by an investment operations division to balance and achieve the various objectives that middle-office operations needs to navigate and undertake. The paper will include key elements that have been implemented within the operating structure of a securities operations group in order to be a strong base for the investment asset management business. The key strategic operating tenets undertaken and implemented will include discussions on automated and integrated workflow models to fulfil all the functions of a global, multi-asset business. Automation will include both internal to the firm’s structure and operations and external to the marketplace. It will also include discussions on a data-centric approach imbedded in the operating model through the implementation of a comprehensive data strategy for the investment business.

Digitalization of building LCA and international activities – in the context of German assessment system for sustainable building

In this paper it is shown how digitalization and the establishment of an online infrastructure for life cycle assessment (LCA) in the context of the assessment system for sustainable building (BNB) by the German government forms starting point for an open international data network, as brought forward by the activities of InData (“International Open Data Network for Sustainable Building”). The establishment of the ILCD+EPD data format for EPD, the provision of interfaces, the development of workflow structures, guidelines and rules, which are openly published, internationally allow to access data from several databases. All these aspects are subject of the InData activities, and are decisive for a harmonization of LCA for sustainable building. With this concept of digitalization the propagation of the use of environmental product declarations (EPD) in building LCA was enhanced, and also new applications for using EPD data are offered, e.g. in BIM or other context.

Standing on the shoulders of giants: Seeding search-based multi-objective optimization with prior knowledge for software service composition

Search-Based Software Engineering, in particular multi-objective evolutionary algorithm , is a promising approach to engineering software service composition while simultaneously optimizing multiple conflicting Quality-of-Service (QoS) objectives. Yet, existing applications of evolutionary algorithms have failed to consider domain knowledge about the problem into the optimization, which is a perhaps obvious but challenging task. This paper aims to investigate different strategies of exploring and injecting knowledge about the problem into the Multi-Objective Evolutionary Algorithm (MOEA) by seeding . Further, we investigate various factors that contribute to the effectiveness of seeding, including the number of seeds, the importance of crossover operation and the similarity of historical problems. We conduced empirical evaluations with NSGA-II, MOEA/D and IBEA based on a wide spectrum of problem instances, including 10 different workflow structures, from 5 to 100 abstract services and 510 to 5.529 × 10 203 candidate concrete services with diverse QoS on latency, throughput and cost, which was chosen from the real-world WS-DREAM dataset that contains 4500 QoS values. We found that, (i) all seeding strategies generally outperform their non-seeded counterparts under the same search budget with large statistical significance. Yet, they may involve relatively smaller compromise on one or two of the quality aspects among convergence, uniformity and spread. (ii) The implication of the number of seeds on the service composition problems is minimal in general (except for IBEA). (iii) In contrast to the non-seeded counterparts, the seeding strategies suffer much less implications by the crossover operation. (iv) The differences of historical problems, which are important for two proposed seeding strategies, can indeed affect the results in a non-linear manner; however, the results are still greatly better than the non-seeded counterparts even with up to 90% difference of the problem settings. The paper concludes that (i) When applying the seeding strategies, the number of seeds to be placed in is less important in general, except for the pre-optimization based strategies under IBEA. (ii) Eliminating or having less crossover is harmful for multi-objective service composition optimization, but the seeding strategies are much less sensitive to this operator than their non-seeded counterparts. (iii) For the history based seeding strategies, the seeds do not have to come from the most similar historical composition problem to achieve the best HV value, but a largely different historical problem should usually be avoided, unless they are the only available seeds.

Fuzzy-based Security-Driven Optimistic Scheduling of Scientific Workflows in Cloud Computing

ABSTRACTCloud computing is a new computing paradigm which is gaining wide acceptance among scientific fraternity in the recent years. The services of cloud could be effectively used for running large-scale data and computation-intensive scientific workflow applications. Finding the optimal schedule for such workflows has been a major concern among the cloud users. In the present work, a novel approach of combining both optimization of the schedule along with the allocation of the virtual machines (VMs) based on security requirements is envisaged. This paper focuses on generating an optimized schedule for the complex workflow structures. The main objective of the schedule is to minimize the makespan of the schedule. In this paper, we design the scheduling heuristic based on the cost prediction matrix (CPM) for optimized cost calculation. The CPM will estimate the execution cost by considering the child’s child task also. This leads to a prophetic estimation on the available VMs. In addition to this, we have used a fuzzy-based decision model for deciding the selection of the VMs based on security constraints in the cloud. This fuzzy model is combined with the optimized cost calculation from CPM for each and every task of the workflow. The proposed secured cost prediction-based scheduling (SCPS) algorithm then schedules the task in the best possible VM, so that the makespan is minimized. Our results show that the newly developed SCPS algorithm yields efficient schedule compared to other existing scheduling models in spite of the inclusion of security constraints besides scheduling. Nevertheless, this secured scheduling is done without much increase in the time complexity.

Performance characterization of scientific workflows for the optimal use of Burst Buffers

Scientific discoveries are increasingly dependent upon the analysis of large volumes of data from observations and simulations of complex phenomena. Scientists compose the complex analyses as workflows and execute them on large-scale HPC systems. The workflow structures are in contrast with monolithic single simulations that have often been the primary use case on HPC systems. Simultaneously, new storage paradigms such as Burst Buffers are becoming available on HPC platforms. In this paper, we analyze the performance characteristics of a Burst Buffer and two representative scientific workflows with the aim of optimizing the usage of a Burst Buffer, extending our previous analyses (Daley et al., 2016). Our key contributions are (a) developing a performance analysis methodology pertinent to Burst Buffers, (b) improving the use of a Burst Buffer in workflows with bandwidth-sensitive and metadata-sensitive I/O workloads, (c) highlighting the key data management challenges when incorporating a Burst Buffer in the studied scientific workflows.

Scidac-Data: Enabling Data Driven Modeling of Exascale Computing

The SciDAC-Data project is a DOE-funded initiative to analyze and exploit two decades of information and analytics that have been collected by the Fermilab data center on the organization, movement, and consumption of high energy physics (HEP) data. The project analyzes the analysis patterns and data organization that have been used by NOvA, MicroBooNE, MINERvA, CDF, D0, and other experiments to develop realistic models of HEP analysis workflows and data processing. The SciDAC-Data project aims to provide both realistic input vectors and corresponding output data that can be used to optimize and validate simulations of HEP analysis. These simulations are designed to address questions of data handling, cache optimization, and workflow structures that are the prerequisites for modern HEP analysis chains to be mapped and optimized to run on the next generation of leadership-class exascale computing facilities.We present the use of a subset of the SciDAC-Data distributions, acquired from analysis of approximately 71,000 HEP workflows run on the Fermilab data center and corresponding to over 9 million individual analysis jobs, as the input to detailed queuing simulations that model the expected data consumption and caching behaviors of the work running in high performance computing (HPC) and high throughput computing (HTC) environments. In particular we describe how the Sequential Access via Metadata (SAM) data-handling system in combination with the dCache/Enstore-based data archive facilities has been used to develop radically different models for analyzing the HEP data. We also show how the simulations may be used to assess the impact of design choices in archive facilities.

Probabilistic Model Checking-Based Service Selection Method for Business Process Modeling

Business process modeling is a way to the organizational change management, which provides abstract workflows to describe business logics for customer demands analysis and IT infrastructures improvement. In order to reuse business process for covering different application scopes, it requires the capacity of configuring each task of business process by selecting appropriate services from the candidate services set, and then assembling them together to properly work with each other as domain-specific software. Considering choosing an executable business process with high quality service (QoS), the service selection is regarded as the pivotal step of determining process instances since service displays probabilistic behaviors under the uncertainty of Internet making business process shown different reliability. In this paper, it proposes an approach to the service selection for business process modeling. In the first phase, the function similarity method is used to pick out services from service repository in order to build a set of candidate services, which checks the functions description to find matching services, especially service may publish one or more functions through multiple interfaces. In the second phase, the probabilistic model checking-based method is employed to the quantitative verification of process instances, which involves services composition and stochastic behaviors computing according to workflow structures. Then, corresponding algorithms are discussed for service selection purposes. Furthermore, it introduces the probabilistic model checking-based framework for prototype design and implementation. Finally, experiments are conducted to demonstrate the effectiveness and efficiency of the proposed method comparing with traditional methods.

A Behavioral Similarity Metric for Semantic Workflows Based on Semantic Task Adjacency Relations With Importance

For modern enterprises and organizations, new business workflow can be constructed by reusing already available similar workflows in the repository. Workflow reuse is an important method for implementing business workflow management. Semantic workflows contain control-flow, data-flow, and semantic information relevant to a domain, which facilitates workflow reuse and adaptation. A similarity metric for semantic workflows is important for achieving workflow reuse. However, the existing similarity metrics for semantic workflows focus on workflow structures while ignoring their behaviours, which affect the quality of retrieved similar semantic workflows. Therefore, this paper proposes a behavioral similarity metric for semantic workflows based on semantic task adjacency relations with importance (ISTARs) that incorporate domain knowledge. First, ISTARs that involve semantic tasks and importance of semantic task adjacency relations are defined, and the ISTARs set is used to express the behaviour of the semantic workflow. The ISTAR similarity proposal is based on the similarity between two ISTARs sets and represents the similarity between semantic workflows. The ISTAR distance deduced from ISTAR similarity satisfies the properties of distance metrics. An experimental evaluation revealed that the proposed ISTAR similarity resulted in more effective retrieval of similar semantic workflows than did existing popular behavioral similarity measures.