Concurrent Workflow Research Articles

Autonomous Battery Optimization by Deploying Distributed Experiments and Simulations

AbstractNon‐trivial relationships link individual materials properties to device‐level performance. Device optimization therefore calls for new automation approaches beyond the laboratory bench with tight integration of different research methods. This study demonstrates a Materials Acceleration Platform (MAP) in the field of battery research based on the problem‐agnostic Fast INtention‐Agnostic LEarning Server (FINALES) framework, which integrates simulations and physical experiments while leaving the active control of the hardware and software resources executing experiments or simulations with the partners running the respective units. This decentralization of control is a distinctive feature of MAPs using the FINALES framework. The connected capabilities entail the formulation and characterization of electrolytes, cell assembly and testing, early lifetime prediction, and ontology‐mapped data storage provided by institutions distributed across Europe. The infrastructure is used to optimize the ionic conductivity of electrolytes and the End Of Life (EOL) of lithium‐ion coin cells by varying the electrolyte formulation. Trends in ionic conductivity are rediscovered and the effect of the electrolyte formulation on the EOL is investigated. Further, the capability of this MAP to bridge diverse research modalities, scales, and institutions enabling system‐level investigations under asynchronous conditions while handling concurrent workflows on the material‐ and system‐level is shown, demonstrating true intention‐agnosticism.

Time-Sensitive and Resource-Aware Concurrent Workflow Scheduling for Edge Computing Platforms Based on Deep Reinforcement Learning

The workflow scheduling on edge computing platforms in industrial scenarios aims to efficiently utilize the computing resources of edge platforms to meet user service requirements. Compared to ordinary task scheduling, tasks in workflow scheduling come with predecessor and successor constraints. The solutions to scheduling problems typically include traditional heuristic methods and modern deep reinforcement learning approaches. For heuristic methods, an increase in constraints complicates the design of scheduling rules, making it challenging to devise suitable algorithms. Additionally, whenever the environment undergoes updates, it necessitates the redesign of the scheduling algorithms. For existing deep reinforcement learning-based scheduling methods, there are often challenges related to training difficulty and computation time. The addition of constraints makes it challenging for neural networks to make decisions while satisfying those constraints. Furthermore, previous methods mainly relied on RNN and its variants to construct neural network models, lacking a computation time advantage. In response to these issues, this paper introduces a novel workflow scheduling method based on reinforcement learning, which utilizes neural networks for direct decision-making. On the one hand, this approach leverages deep reinforcement learning, eliminating the need for researchers to define complex scheduling rules. On the other hand, it separates the parsing of the workflow and constraint handling from the scheduling decisions, allowing the neural network model to focus on learning how to schedule without the necessity of learning how to handle workflow definitions and constraints among sub-tasks. The method optimizes resource utilization and response time, as its objectives and the network are trained using the PPO algorithm combined with Self-Critic, and the parameter transfer strategy is utilized to find the balance point for multi-objective optimization. Leveraging the advantages of reinforcement learning, the network can be trained and tested using randomly generated datasets. The experimental results indicate that the proposed method can generate different scheduling outcomes to meet various scenario requirements without modifying the neural network. Furthermore, when compared to other deep reinforcement learning methods, the proposed approach demonstrates certain advantages in scheduling performance and computation time.

And synopses for all: A synopses data engine for extreme scale analytics-as-a-service

In this work, we detail the design and structure of a Synopses Data Engine (SDE) which combines the virtues of parallel processing and stream summarization towards delivering interactive analytics at extreme scale. Our SDE is built on top of Apache Flink and implements a novel synopses-as-a-service paradigm. In that, it achieves (i) concurrently maintaining thousands of synopses of various types for thousands of streams, on demand, (ii) reusing synopses that are common across various concurrent workflows, (iii) providing data summarization facilities even for cross-(Big Data) platform workflows, (iv) pluggability of new synopses on-the-fly, (v) increased potential for workflow execution optimization. The proposed SDE-as-a-service provides interactive analytics at scale by enabling 3 types of scalability: (i) enhanced horizontal scalability, i.e., not only scaling out the computation to a number of processing units available in a computer cluster, but also harnessing the processing load assigned to each by operating on carefully-crafted data summaries, (ii) vertical scalability, i.e., scaling the computation to very high numbers of processed streams and (iii) federated scalability i.e., scaling across geo-distributed clusters and clouds by controlling the communication required to answer global queries.

Data Aggregation Model of Enterprise Financial Supervision Based On BP Neural Network

Aiming at the problems of high damping coefficient and long aggregation response time of traditional enterprise financial supervision data aggregation model, an enterprise financial supervision data aggregation model based on BP neural network is designed. In order to effectively gather enterprise financial supervision data, an enterprise model of financial supervision data is proposed based on concurrent workflow. Using the basic financial data, construct the tree workflow, use the financial data summary server tree, construct the multi-level data aggregation mechanism, compress the enterprise financial supervision data based on BP neural network, and realize the model through the tree workflow and multi-level data aggregation mechanism. The experimental results show that the aggregation damping coefficient and response time of the design model are significantly lower than those of the control group, which can solve the problems of high damping coefficient and long aggregation response time of the traditional enterprise financial supervision data aggregation model.

Consulting Business Workflow and Design Performance Metrics for BIM Based Construction Design in Nigeria

Extant literature has confirmed the unavoidability of changes to the design of construction projects. These changes have also been identified as major reasons for project cost, quality, and time failure. Despite BIM’s documented capabilities to enhance project success, its implementation remains unsatisfactory in Nigeria. This paper, therefore, examines the impacts of BIM-based design workflow in consulting business on construction project design performance. Using questionnaire as the research instrument, quantitative survey research was adopted to collect data from consulting organizations. Analysis revealed sequential and concurrent workflows as the most used and preferred, respectively. Design coordination, information, and communication are enhanced with sequential workflow within a firm, while concurrent metrics are enhanced when collaborating with other firms. Both workflows are jointly declared suitable for the timely delivery of designs. The study, therefore, recommended retraining of professionals on sequential and concurrent workflows because of their appropriateness for efficient design within and among firms.

Towards Automated Choreographing of Web Services Using Planning

For Web service composition, choreography has recently received great attention and demonstrated a few key advantages over orchestration such as distributed control, fairness, data efficiency, and scalability. Automated design of choreography plans, especially distributed plans for multiple roles, is more complex and has not been studied before. Existing work requires manual generation assisted by model checking. In this paper, we propose a novel planning-based approach that can automatically convert a given composition task to a distributed choreography specification. Although planning has been used for orchestration, it is difficult to use planning for choreography, as it involves decentralized control, concurrent workflows, and contingency. We propose a few novel techniques, including compilation of contingencies, dependency graph analysis, and communication control, to handle these characteristics using planning. We theoretically show the correctness of this approach and empirically evaluate its practicability.

TOPSIS inspired Budget and Deadline Aware Multi-Workflow Scheduling for Cloud computing

Abstract Scheduling is a decision-making mechanism that allows resource sharing among several activities by determining their order of execution on the available resources. In the heterogeneous distributed systems, it is a great challenge to schedule concurrent workflows submitted by different users at different times. Scheduling with deadline and budget constraints are becoming an even more challenging issue for cloud systems due to the cloud dynamics such as on-demand provisioning, elasticity, abundant resource types, and various pricing schemes. A well-managed budget and deadline constraint scheduling is required to optimize the system performance and end-user satisfaction. Hence, improving system performance and optimizing multiple scheduling criteria at the same time is a big challenge. To address these issues, a novel multi-workflow scheduling algorithm based on the Multi-Criteria Decision Making (MCDM) approach, TOPSIS (Technique of Order Preference by Similarity to Ideal Solution) is presented. A weighted sum of run time, cost and data transfer time are used to determine the optimal resource among the available resources in accordance with the task requirements. The performance of the proposed algorithm is compared with the state-of-the-art algorithms such as Budget-Heterogeneous Earliest Finish Time (BHEFT), Budget and Deadline Constraint Heterogeneous Earliest Finish Time (BDHEFT) and Cloud-based Workflow Scheduling Algorithm (CWSA) algorithms based on budget constraint,deadline constraint, and resource utilization. The experimental results demonstrate that the proposed T-BDMWS outperforms current state-of-the-art heuristics with the criteria of achieving the user-specified budget or deadline constraints and resource efficiency.

Energy- and Resource-Aware Computation Offloading for Complex Tasks in Edge Environment

Mobile users typically have a series of complex tasks consisting of time-constrained workflows and concurrent workflows that need to be processed. However, these tasks cannot be performed directly locally due to resource limitations of the mobile terminal, especially for battery life. Fortunately, mobile edge computing (MEC) has been recognized as a promising technology which brings abundant resource at the edge of mobile network enabling mobile devices to overcome the resource and capacity constraints. However, edge servers, such as cloudlets, are heterogeneous and have limited resources. Thus, it is important to make an appropriate offloading strategy to maximize the utility of each cloudlet. In view of this, the time consumption and energy consumption of mobile devices and resource utilization of cloudlets have been taken into consideration in this study. Firstly, a multiconstraint workflow mode has been established, and then a multiobjective optimization mode is formulated. Technically, an improved optimization algorithm is proposed to address this mode based on Nondominated Sorting Genetic Algorithm II. Both extensive experimental evaluations and detailed theoretical analysis are conducted to show that the proposed method is effective and efficiency.

TOPSIS inspired cost-efficient concurrent workflow scheduling algorithm in cloud

Scheduling is a decision-making mechanism that enables the sharing of resources among several activities by determining their execution order on the set of available resources. In distributed systems, it is a great challenge to schedule multiple workflows submitted at different times. In particular, concurrent workflow scheduling with time constraints makes the problem more complex in the cloud due to the dynamics of the cloud such as elasticity, non-homogeneous resource types, various pricing schemes, and virtualization. A well-managed deadline workflow scheduling is required to improve end-user satisfaction and system performance. In the meantime, the intrinsic uncertainty in the cloud increases the difficulties of scheduling problems. Therefore, it is a great challenge to improve system performance and optimize several scheduling criteria simultaneously. To address the above issues, a novel concurrent workflow scheduling method for heterogeneous distributed environments based on the new Multi-Criteria Decision Making (MCDM) method i.e., TOPSIS (Technique of Order Preference by Similarity to Ideal Solution) is presented. A weighted sum of execution time, cost and communication time are used to find out the optimal resource among the existing resources as per the workflow task requirements. The proposed method minimizes the makespan and execution cost of the workflow and improves the resource efficiency under uncertain environment. The performance of the proposed work is compared with the state-of-the-art algorithms such as Cloud-based Workflow Scheduling Algorithm (CWSA), Earliest Finish Time-Maximum Effective Reduction (EFT-MER) and Heterogeneous Earliest-Finish-Time (HEFT) algorithms based on deadline constraint and resource utilization. Our experimental results demonstrate that the proposed T-CCWSA outperforms current state-of-the-art heuristics with the criteria of achieving the deadline constraint, minimizing the cost of execution and resource efficiency.

Incremental Analysis of Temporal Constraints for Concurrent Workflow Processes With Dynamic Changes

Nowadays, workflow process changes frequently in a fast-changing business environment. When updating a workflow process via some structural changes, one of the most important tasks is to maintain its consistency under temporal constraints. Several approaches have been developed to cope with this issue. However, they are either inaccurate in locating changed parts or inefficient in analyzing temporal constraints owing to their excessive or erroneous estimation of affected portions for the updated workflow processes. Based on a sprouting graph (a graph that records the structure and time information of all paths in a workflow process in advance), this paper proposes a novel approach to analyzing temporal constraints for workflow processes with dynamic changes. First, changed parts are located via affected collaboration execution paths in the new model, i.e., the workflow process after changes. Second, instead of updating all the elements, only necessary (changed) nodes corresponding to the changed parts are updated in the sprouting graph of the original model, i.e., the workflow process before changes. Finally, based on the updated sprouting graph, only affected temporal constraints (the temporal constraints whose partial or all paths are contained in the affected collaboration execution paths) are checked. Compared with the existing works, our approach is applicable and efficient to check temporal constraints for large-scale and complex workflow processes thanks to its much lower time and space complexity.

Concurrent workflow budget- and deadline-constrained scheduling in heterogeneous distributed environments

In heterogeneous distributed environment, it is a great challenge to schedule multiple workflows submitted at different times. Particularly, scheduling of concurrent workflows with deadline and budget constraints makes the problem become more complex. Recent studies have proposed dynamic scheduling strategies for concurrent workflows which have limitations in inconsistent environments. Therefore, this paper presents a new dynamic scheduling algorithm for concurrent workflows. This algorithm proposes a uniform ranking that considers the time and costs for both workflows and workgroups to assign priorities for tasks. In the resource selection phase, it controls the resource selection range for each task based on an optimistic budget for the current task and selects resources for the current task according to a defined bi-factor. The experimental results show that our algorithm outperforms the existing algorithms in both consistent and inconsistent environments.

Load Balancing Scheduling Algorithm for Concurrent Workflow

Concurrent workflow scheduling algorithm works in three phases, namely rank computation, tasks selection, and resource selection. In this paper, we introduce a new ranking algorithm that computes the rank of a task, based on its successor rank and its predecessors average communication time, instead of its successors rank. The advantage of this ranking algorithm is that two dependent tasks are assigned to the same machine and as a result the scheduled length is reduced. The task selection phase selects a ready task from each workflow and creates a task pool. The resource selection phase initially assigns tasks using min-min heuristic, after initial assignment, tasks are moved from the highly loaded machines to the lightly loaded machines. Our resource selection algorithm increases the load balance among the resources due to tasks assignment heuristic and reassignment of tasks from the highly loaded machines. The simulation results show that our proposed scheduling algorithm performs better over existing approaches in terms of load balance, makespan and turnaround time.

DSM: a dynamic scheduling method for concurrent workflows in cloud environment

Cloud computing, emerged as a commercial service model, has been widely concerned in both industry and academia. Massive workflow applications could be performed simultaneously on the cloud platforms, which significantly benefits from the elasticity and convenience of cloud computing. However, it is still a challenge to schedule virtualized resources for the concurrent workflows in cloud environment, with limited high-performance resources in a timesaving and efficient manner. In view of this challenge, a dynamic scheduling method for concurrent workflows, named as DSM, in cloud environment is proposed to satisfy the various resource requirements of the workflows. Technically, a time overhead model for the workflows and a resource utilization model for cloud datacenter are presented. Then a relevant dynamic scheduling method is designed based on critical path lookup, which aims at minimizing the makespan of workflows, and maximizing the resource utilization of the datacenter during the execution of the workflows. Extensive experimental evaluations demonstrate the efficiency and effectiveness of our proposed method.

In brief

The execution of a workflow application can result in an imbalanced workload among allocated processors, ultimately resulting in a waste of resources and a higher cost to the user. Here, we consider a dynamic resource management system in which processors are reserved not for a job but only to run a task, thus allowing a higher resource usage rate. This paper presents a scheduling algorithm that manages concurrent workflows in a dynamic environment in which jobs are submitted by users at any moment in time, on shared heterogeneous resources, and constrained to a specified budget and deadline for each job. Recent research attempted to propose dynamic strategies for concurrent workflows but only addressed fairness in resource sharing among applications while minimizing the execution time. The Multi-QoS Profit-Aware scheduling algorithm (MQ-PAS) proposed here is able to increase the profit achieved by the provider by considering the budget available for each job to define tasks priorities. We study the scalability of the algorithm with different types of workflows and infrastructures. The experimental results show that our strategy improves provider revenue significantly and obtains comparable successful rates of completed jobs.

Expansion slot backfill scheduling for concurrent workflows with deadline on heterogeneous resources

The cost of using a computational resource is measured from startup to shutdown, including the cost of time slots between tasks. Concurrent scheduling of multiple scientific workflows on heterogeneous resources can improve resources utilization and reduce the cost. To make full use of waste time slots between the tasks and improve the completion rate of workflows, Expansion Slot Backfill (ESB) algorithm is proposed in this paper for scheduling multiple deadline-constrained workflows on a fixed set of resources. All workflows are mapped to the resources with one by one strategy. Each new task tries to backfill the earliest time slot in turn. When the time slot is not enough to backfill, it can be expanded elastically by the slide of the earlier tasks. If these tasks slide cause some workflow to exceed the deadline constraint, such workflow with fewer time slots is discarded and the slide is withdrawn. Experiments with multiple parameter variations show that the algorithm get better performance in resource utilization, workflows throughput and time complexity.

Analyzing Degree of Parallelism for Concurrent Timed Workflow Processes With Shared Resources

Degree of parallelism is an important factor in workflow process management, because it is useful to accurately estimate the server costs and schedule severs in workflow processes. However, existing methods that are developed to compute degree of parallelism neglect to consider activities with uncertain execution time. In addition, these methods are limited in dealing with the situation where activities in multiple concurrent workflow processes use shared resources. To address the limitations, we propose a new approach to analyzing degree of parallelism for concurrent workflow processes with shared resources. Superior over the existing methods, our approach can compute degree of parallelism for multiple concurrent workflow processes that have activities with uncertain execution time and shared resources. Expectation degree of parallelism is useful to estimate the server costs of the workflow processes, and maximum degree of parallelism can guide managers to allocate severs or virtual machines based on the business requirement. We demonstrate the application of the approach and evaluate the effectiveness in a real-world business scenario.

ATSDS: adaptive two-stage deadline-constrained workflow scheduling considering run-time circumstances in cloud computing environments

A significant aspect of cloud computing is scheduling of a large number of real-time concurrent workflow instances. Most of the existing scheduling algorithms are designed for a single complex workflow instance. This study examined instance-intensive workflows bounded by SLA constraints, including user-defined deadlines.The scheduling method for these workflows with dynamic workloads should be able to handle changing conditions and maximize the utilization rate of the cloud resources. The study proposes an adaptive two-stage deadline-constrained scheduling (ATSDS) strategy that considers run-time circumstances of workflows in the cloud environment. The stages are workflow fragmentation and resource allocation.In the first stage, the workflows according to cloud run-time circumstances (number of Virtual Machines (VMs) and average available bandwidth) are dynamically fragmented. In the second stage, using the workflow deadline and the capacity of the VMs, the workflow fragments created are allocated to the VMs to be executed. The simulation results show improvements in terms of workflow completion time, number of messages exchanged, percentage of workflows that meet the deadline and VM usage cost compared to other approaches.

Reprint of “Multi-QoS constrained and Profit-aware scheduling approach for concurrent workflows on heterogeneous systems”

The execution of a workflow application can result in an imbalanced workload among allocated processors, ultimately resulting in a waste of resources and a higher cost to the user. Here, we consider a dynamic resource management system in which processors are reserved not for a job but only to run a task, thus allowing a higher resource usage rate. This paper presents a scheduling algorithm that manages concurrent workflows in a dynamic environment in which jobs are submitted by users at any moment in time, on shared heterogeneous resources, and constrained to a specified budget and deadline for each job. Recent research attempted to propose dynamic strategies for concurrent workflows but only addressed fairness in resource sharing among applications while minimizing the execution time. The Multi-QoS Profit-Aware scheduling algorithm (MQ-PAS) proposed here is able to increase the profit achieved by the provider by considering the budget available for each job to define tasks priorities. We study the scalability of the algorithm with different types of workflows and infrastructures. The experimental results show that our strategy improves provider revenue significantly and obtains comparable successful rates of completed jobs.

Maximizing the completion rate of concurrent scientific applications under time and budget constraints

In many domains of science, scientific applications are represented by workflows. In this paper, we introduce a resource management strategy to maximize the success rate of concurrent workflow applications constrained by individual deadline and budget values. The Multi-Workflow Deadline-Budget Scheduling (MW-DBS) algorithm can schedule multiple workflows that can arrive in the system at any time, with the aim of satisfying individual job requirements. MW-DBS produces schedules without performing optimizations but guarantees that the deadline and budget defined for each job are not exceeded. Experimental results show that our strategy increases the scheduling success rate of finding valid solutions.

Multi-QoS constrained and Profit-aware scheduling approach for concurrent workflows on heterogeneous systems

The execution of a workflow application can result in an imbalanced workload among allocated processors, ultimately resulting in a waste of resources and a higher cost to the user. Here, we consider a dynamic resource management system in which processors are reserved not for a job but only to run a task, thus allowing a higher resource usage rate. This paper presents a scheduling algorithm that manages concurrent workflows in a dynamic environment in which jobs are submitted by users at any moment in time, on shared heterogeneous resources, and constrained to a specified budget and deadline for each job. Recent research attempted to propose dynamic strategies for concurrent workflows but only addressed fairness in resource sharing among applications while minimizing the execution time. The Multi-QoS Profit-Aware scheduling algorithm (MQ-PAS) proposed here is able to increase the profit achieved by the provider by considering the budget available for each job to define tasks priorities. We study the scalability of the algorithm with different types of workflows and infrastructures. The experimental results show that our strategy improves provider revenue significantly and obtains comparable successful rates of completed jobs.