Deadline-constrained Workflow Scheduling Research Articles

PCP–ACO: a hybrid deadline-constrained workflow scheduling algorithm for cloud environment

PCP–ACO: a hybrid deadline-constrained workflow scheduling algorithm for cloud environment

A Fully Hybrid Algorithm for Deadline Constrained Workflow Scheduling in Clouds

With the migration of more and more workflows to clouds, the workflow scheduling in clouds (WSC) becomes a critical problem. Although many algorithms have been presented for WSC, there is still room and need for improvement. This paper formulates WSC as a constrained optimization problem that optimizes workflow execution cost within a workflow deadline constraint and proposes a fully hybrid workflow scheduling algorithm, called HPCP-PSO to solve it. Unlike previous works, HPCP-PSO is based on the repeated and alternated execution of two different methods, namely, the heuristic IaaS Cloud Partial Critical Paths (IC-PCP) and meta-heuristic Particle Swarm Optimization (PSO). Moreover, HPCP-PSO incorporates with two novel designs: 1) a new solution encoding strategy not only to sufficiently embody the elasticity of cloud resources, but also to reflect the scheduling relationship between assigned and unassigned tasks; 2) a solution repair strategy on each infeasible lease process to utilize a user-defined deadline more effectively and enhance the solution efficiency of the algorithm. Extensive experiments are conducted on four real-world scientific workflows and the results show that compared with IC-PCP, PSO, and HGSA, the proposed algorithm outperforms them on average by 35.83%, 70.53%, and 87.71% in terms of workflow execution cost.

Cost-effective approaches for deadline-constrained workflow scheduling in clouds

Cost-effective approaches for deadline-constrained workflow scheduling in clouds

Cost-Efficient Workflow Scheduling Algorithm for Applications With Deadline Constraint on Heterogeneous Clouds

In recent years, more and more large-scale data processing and computing workflow applications run on heterogeneous clouds. Such cloud applications with precedence-constrained tasks are usually deadline-constrained and their scheduling is an essential problem faced by cloud providers. Moreover, minimizing the workflow execution cost based on cloud billing periods is also a complex and challenging problem for clouds. In realizing this, we first model the workflow applications as I/O Data-aware Directed Acyclic Graph (DDAG), according to clouds with global storage systems. Then, we mathematically state this deadline-constrained workflow scheduling problem with the goal of minimum execution financial cost. We also prove that the time complexity of this problem is NP-hard by deducing from a multidimensional multiple-choice knapsack problem. Third, we propose a heuristic cost-efficient task scheduling strategy called CETSS, which includes workflow DDAG model building, task subdeadline initialization, greedy workflow scheduling algorithm, and task adjusting method. The greedy workflow scheduling algorithm mainly consists of dynamical task renting billing period sharing method and unscheduled task subdeadline relax technique. We perform rigorous simulations on some synthetic randomly generated applications and real-world applications, such as Epigenomics, CyberShake, and LIGO. The experimental results clearly demonstrate that our proposed heuristic CETSS outperforms the existing algorithms and can effective save the total workflow execution cost. In particular, CETSS is very suitable for large workflow applications.

ET2FA: A Hybrid Heuristic Algorithm for Deadline-constrained Workflow Scheduling in Cloud

Cloud computing is an emerging computational infrastructure for cost-efficient workflow execution that provides flexible and dynamically scalable computing resources at pay-as-you-go pricing. Workflow scheduling, as a typical NP-Complete problem, is one of the major issues in cloud computing. However, in the cloud scenario with unlimited resources, how to generate an efficient and economical workflow scheduling scheme under the deadline constraint is still an extraordinary challenge. In this paper, we propose a hybrid heuristic algorithm called enhanced task type first algorithm (ET2FA) to solve deadline-constrained workflow scheduling in cloud with new features such as hibernation and per-second billing. The objectives to be minimized include the total cost and total idle rate. ET2FA involves three phases: 1) Task type first algorithm, which schedules tasks based on topological level and task types, and utilizes a compact-scheduling-condition based VM selection method to assign each task. 2) Delay operation based on block structure, which further optimizes total cost and total idle rate based on block structure properties. 3) Instance hibernate scheduling heuristic, which sets an instance to hibernate if idle for a duration. Extensive simulation experiments based on seven well-known real-world workflow applications show that ET2FA delivers better performance in comparison to the state-of-the-art algorithms.

Deadline‐constrained cost‐energy aware workflow scheduling in cloud

AbstractNowadays, scientists are dealing with large‐scale scientific workflows that need a high processing capacity platform to facilitate on‐time completion. Cloud computing is the ideal platform to overcome this problem as it has several resources that scientists may choose from depending on the size of their applications. However, using cloud computing requires some monetary charges. Recently cloud computing providers started a new pricing schema that offers to their users a set of resources with specific combinations of CPU frequency configurations settings and price. The selected configurations settings reflect energy consumption. Besides, the configuration selection to meet users' satisfaction (minimum cost) and providers' satisfaction (energy saving) is crucial. Therefore, a multiobjective (cost and energy) efficient mechanism is essential. In this article, we address an important novel problem concerning multiobjective deadline constrained workflow scheduling in the cloud. We first study the relationship between cost minimization and minimization of the energy consumption in a cloud environment, and then discuss, develop, and propose an algorithm with two variants to help the system satisfy both sides (users and providers) at the same time during the selection of the configuration. The proposed heuristic is evaluated using specified real‐world applications. The observed results indicate that our heuristic can reduce significantly the energy consumption and the cost at the same time.

User defined weight based budget and deadline constrained workflow scheduling in cloud

AbstractCloud computing is a technology that is being used by scientists for the execution of huge scale workflows as it has various features such as availability, faster, cheaper, elasticity, and high accessibility. In addition, the users are charged on a pay per use basis which makes it more efficient to be used by various clients. Scheduling workflows in the cloud is a difficult task as it takes into account the dependencies of workflows along with quality of service (QoS) demands. The problem becomes more challenging when there are multiple QoS objectives which are often disagreeing to each other. A lot of research has been done to minimize makespan and execution cost under deadline and budget constraints. This article presents a workflow scheduling algorithm based on Jaya to minimize them both and returns solutions according to the weights a user gives. The algorithm has been compared with Min‐Min, Max‐Min, Round Robin, heterogeneous earliest finish time first, first come first serve, minimum completion time, dynamic heterogeneous earliest finish time first in WorkflowSim. The results are compared based on both cost and time. In the end, a trade‐off between the objectives has been shown. Results show that Jaya performs much better than other algorithms when averaged over several iterations.

An adaptive and deadline-constrained workflow scheduling algorithm in infrastructure as a service clouds

An adaptive and deadline-constrained workflow scheduling algorithm in infrastructure as a service clouds

HBDCWS: heuristic-based budget and deadline constrained workflow scheduling approach for heterogeneous clouds

The predilection of scientific applications toward a high-performance computing system is attained through the emergence of the cloud. Large-scale scientific applications can be modeled as workflows and are scheduled on the cloud. However, such scheduling becomes even more onerous due to the dynamic and heterogeneous nature of cloud and therefore considered as a problem of NP-Complete. The scheduling of workflows is always constrained to QoS parameters. Most of the applications are bound to time and cost, which is observed to be the most crucial parameter. Therefore, in this paper, a heuristic-based budget and deadline constrained workflow scheduling algorithm (HBDCWS) has been proposed to utilize those applications that have the budget and deadline constraints. The novelty of the proposed work is to provide a simple budget and deadline distribution strategy where budget and deadline of workflow are converted to level budget and level deadline. Additionally, the level budget is again transferred to each task. This strategy not only satisfies the given constraints but also proves to be efficient for minimizing the makespan and reducing the cost of execution. Experimental results on several workflows demonstrate that the proposed HBDCWS algorithm finds a feasible solution that accomplishes the given constraints with a higher success rate in most cases.

Deadline-constrained workflow scheduling in IaaS clouds with multi-resource packing

Abstract Workflow is a common model to represent large computations composed of dependent tasks. Most existing workflow scheduling algorithms use computing resources in a non-multiprogrammed way, by which only one task can run on a service (machine) at a time. In this paper, we study a new workflow scheduling model on heterogeneous Infrastructure-as-a-Service (IaaS) platforms, which allows multiple tasks to run concurrently on a virtual machine (VM) according to their multi-resource demands. First, we propose a list-scheduling framework for the new multiprogrammed cloud resource model. In the order of a priority list, this framework gradually appoints tasks the best placements found on both existing and new VMs on the platform. Different task prioritization and placement comparison methods can be employed for different scheduling objectives. To fully exploit the heterogeneity of IaaS platforms, the VMs can be scaled up during the scheduling process. Then, we propose a deadline-constrained workflow scheduling algorithm (called DyDL) based on this framework to optimize the cost of workflow execution. This algorithm prioritizes tasks by their latest start times and appoints tasks the placements which can meet their latest start times and incur the minimal cost increases. Experimental results show that DyDL can achieve significantly better schedules in most test cases compared to several existing deadline-constrained workflow scheduling algorithms.

A Predictive and Evolutionary Approach for Cost-Effective and Deadline-Constrained Workflow Scheduling Over Distributed IaaS Clouds

Clouds provide highly elastic resource provisioning styles through which scientific workflows are allowed to acquire desired resources ahead of the execution and build required software environment on virtual machines (VMs). However, various challenges for cloud workflow, especially its optimal scheduling, are yet to be addressed. Traditional approaches mainly consider VMs to be with non-fluctuating, time-invariant, stochastic, or bounded performance. This work describes workflows to be deployed and executed over distributed infrastructure-as-a-service clouds with time-varying performance of VMs and is aimed at reducing the execution cost of workflow while meeting deadline constraints. For this purpose, the authors employ time-series-based prediction approaches to capture dynamic performance fluctuations, feed an evolutionary algorithm with predicted performance information, and generate schedules at real-time. A case study based on multiple randomly-generated workflow templates and third-party commercial clouds shows that their proposed approach outperforms traditional ones.

Cost optimization heuristics for deadline constrained workflow scheduling on clouds and their comparative evaluation

SummaryNowadays, cloud service providers usually offer users virtual machines with various combinations of configurations and prices. As this new service scheme emerges, the problem of choosing the cost‐minimized combination under a deadline constraint is becoming more complex for users. The complexity of determining the cost‐minimized combination may be resulted from different causes: the characteristics of user applications and providers' setting on the configurations and pricing of virtual machine. In this paper, we proposed an algorithm with two variants to help the users to schedule their workflow applications on clouds so that the cost can be minimized and the deadline constraints can be satisfied. The proposed algorithm is evaluated by extensive simulation experiments with two realistic workflows.

A benchmark approach and its toolkit for online scheduling of multiple deadline-constrained workflows in big-data processing systems

As distributed systems such as clouds get increasingly popular in the use for big-data processing, there is a need of shifting research attention from minimizing the workflow completion time to satisfying the deadline constraints specified by the users and boosting the benefit of the resource providers. This paper focuses on the online deadline-constrained workflow scheduling problem of how to schedule a set of sequentially submitted workflows with deadline constraints to maximize the resource utilization as well as the success rate of meeting the deadlines. A discrete-event based simulator with a novel benchmark approach is proposed to ease the analysis of the problem. Extensive evaluation has been done to exhibit the effectiveness and significance of the proposed benchmark approach and the developed simulator.

Scheduling deadline-constrained scientific workflow using chemical reaction optimisation algorithm in clouds

The advent of cloud computing as a new model of service provisioning in distributed systems encourages researchers to investigate its benefits and drawbacks on executing scientific applications such as workflows. One of the most challenging problems in clouds is workflow scheduling, i.e., the problem of satisfying the QoS requirements of the users as well as minimising the cost of workflow execution. In this paper, a novel meta-heuristic method, called chemical reaction optimisation (CRO), is developed to solve deadline-constrained workflow scheduling, which tries to minimise the cost of workflow execution while meeting a user-defined deadline. A set of appropriate parameters can be obtained based on orthogonal experimental design (OED) and factor analysis. Experiments are done in two real workflow applications, and the results demonstrate the effectiveness of the proposed algorithm.

A novel cost-efficient approach for deadline-constrained workflow scheduling by dynamic provisioning of resources

Workflow scheduling is a crucial aspect of cloud computing that should be performed in an efficient manner for optimal utilization of resources. The development of a cost-efficient algorithm has always been an important topic of research in this regard. In this paper, we propose a novel workflow scheduling algorithm, which is cost-efficient and deadline-constrained. The proposed algorithm is consolidated by dynamic provisioning of the resources, using k-means clustering technique and a variant of the Subset-Sum problem. In the algorithm, we consider level based scheduling using the concept of Bag of Tasks (bots) and develop a new technique for associating deadlines with each bot. Through extensive simulation runs, we show that the proposed algorithm outperforms the existing algorithms like Dynamic Provisioning Dynamic Scheduling (DPDS) and Infrastructure as a Service (IaaS) Cloud-Partial Critical Path (IC-PCP). The effectiveness of our algorithm over these two algorithms is also illustrated through the popular statistical test ANOVA and its subsequent post-hoc analysis.

Deadline Constrained Workflow Scheduling Optimization by Initial Seeding with ANT Colony Optimization

Deadline Constrained Workflow Scheduling Optimization by Initial Seeding with ANT Colony Optimization

ATSDS: adaptive two-stage deadline-constrained workflow scheduling considering run-time circumstances in cloud computing environments

A significant aspect of cloud computing is scheduling of a large number of real-time concurrent workflow instances. Most of the existing scheduling algorithms are designed for a single complex workflow instance. This study examined instance-intensive workflows bounded by SLA constraints, including user-defined deadlines.The scheduling method for these workflows with dynamic workloads should be able to handle changing conditions and maximize the utilization rate of the cloud resources. The study proposes an adaptive two-stage deadline-constrained scheduling (ATSDS) strategy that considers run-time circumstances of workflows in the cloud environment. The stages are workflow fragmentation and resource allocation.In the first stage, the workflows according to cloud run-time circumstances (number of Virtual Machines (VMs) and average available bandwidth) are dynamically fragmented. In the second stage, using the workflow deadline and the capacity of the VMs, the workflow fragments created are allocated to the VMs to be executed. The simulation results show improvements in terms of workflow completion time, number of messages exchanged, percentage of workflows that meet the deadline and VM usage cost compared to other approaches.

CCA: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud

Workflows are adopted as a powerful modeling technique to represent diverse applications in different scientific fields as a number of loosely coupled tasks. Given the unique features of cloud technology, the issue of cloud workflow scheduling is a critical research topic. Users can utilize services on the cloud in a pay-as-you-go manner and meet their quality of service (QoS) requirements. In the context of the commercial cloud, execution time and especially execution expenses are considered as two of the most important QoS requirements. On the other hand, the remarkable growth of multicore processor technology has led to the use of these processors by Infrastructure as a Service cloud service providers. Therefore, considering the multicore processing resources on the cloud, in addition to time and cost constraints, makes cloud workflow scheduling even more challenging. In this research, a heuristic workflow scheduling algorithm is proposed that attempts to minimize the execution cost considering a user-defined deadline constraint. The proposed algorithm divides the workflow into a number of clusters and then an extendable and flexible scoring approach chooses the best cluster combinations to achieve the algorithm's goals. Experimental results demonstrate a great reduction in resource leasing costs while the workflow deadline is met.

Deadline Constrained based Workflow Scheduling for Cloud Computing

Cloud computing is the emerging trend in distributing computing that facilitates software applications, platform, and hardware infrastructures as a service. Cloud service providers offer these services based on Service Level Agreements (SLAs) which defines user's required Quality of Service (QoS) parameters. Workflow scheduling is one of the major problems in cloud systems. A good scheduling algorithm must minimize the completion time and cost of workflow application along with QoS requirements of the user. Deadline constrained workflow scheduling model is proposed in this paper to execute workflows with manageable cost and priority while meeting the deadline constraint. Deadline is defined as the latest time by which all the tasks should be completed. The proposed model uses Earliest Finish (EF) to find the deadline and job schedule assignment algorithm to define the paths which is used to schedule the tasks. The proposed model reduces the execution cost and completes the application workflows within the available deadline. This paper extends our earlier work on Artificial Bee Colony algorithm to schedule workflow jobs within the deadline while optimizing the cost and the completion time. The proposed model has been simulated using CloudSim toolkit, and the experimental results show that the proposed algorithm meets the deadline constraint while minimising exhibits less completion time, and also within manageable cost.

Low-time complexity budget–deadline constrained workflow scheduling on heterogeneous resources

The execution of scientific applications, under the utility computing model, is constrained to Quality of Service (QoS) parameters. Commonly, applications have time and cost constraints such that all tasks of an application need to be finished within a user-specified Deadline and Budget. Several algorithms have been proposed for multiple QoS workflow scheduling, but most of them use search-based strategies that generally have a high time complexity, making them less useful in realistic scenarios. In this paper, we present a heuristic scheduling algorithm with quadratic time complexity that considers two important constraints for QoS-based workflow scheduling, time and cost, named Deadline–Budget Constrained Scheduling (DBCS). From the deadline and budget defined by the user, the DBCS algorithm finds a feasible solution that accomplishes both constraints with a success rate similar to other state-of-the-art search-based algorithms in terms of the successful rate of feasible solutions, consuming in the worst case only approximately 4% of the time. The DBCS algorithm has a low-time complexity of O(n2.p) for n tasks and p processors.