Scheduling Workflow Applications Research Articles

Energy-efficient time and cost constraint scheduling algorithm using improved multi-objective differential evolution in fog computing

The recent surge in Internet of Things (IoT) applications and smart devices has led to a substantial rise in the data generation. One of the major issues involved is to meet strict quality of service (QoS) requirements for computing these applications in terms of execution time, cost and in an energy-efficient manner. To extract useful information, fast processing and analysis of data is needed. Consequently, moving all the data to centralized cloud data centers would lead to high processing times, increased cost and energy consumption and more bandwidth usage; thus, processing of applications with strict latency requirements becomes challenging. The addition of fog layer between cloud and IoT devices has provided promising solutions to such issues. However, efficient employment of computing resources in the hybrid infrastructure of fog and cloud nodes is of great significance and demands an optimal scheduling strategy. Toward this direction, a novel Pareto-based algorithm in fog computing, namely energy-efficient time and cost (ETC) constraint scheduling algorithm, is introduced in this paper for scheduling workflow applications. ETC attempts to optimize monetary cost along with time and energy objectives. Improved multi-objective differential evolution (I-MODE) meta-heuristic is introduced and incorporated with deadline-aware stepwise frequency scaling approach that is based on our previously proposed energy makespan multi-objective optimization (EM-MOO) algorithm. Synthetic and real-world application workflows are used to conduct evaluation of the proposed work with existing well-known algorithms from the literature. The experimental results for synthetic workflows reveal that the proposed algorithm lessens energy utilization by 14–21%, execution time by almost 25% and cost consumption by 22–27%, while for real-world application workflows, energy consumption is reduced by 12–24%, execution time by 14–16% and cost consumption by 23–29%.

Decentralized and scalable hybrid scheduling-clustering method for real-time applications in volatile and dynamic Fog-Cloud Environments

A major challenge in Cloud-Fog settings is the scheduling of workflow applications with time constraints as the environment is highly volatile and dynamic. Furthermore, adding the complexities of handling IoT nodes, as the major owners of the workflow requests, renders the problem space even harder to address. This paper presents a hybrid scheduling-clustering method for addressing this challenge. The proposed lightweight, decentralized, and dynamic clustering algorithm is based on fuzzy inference with intrinsic support for mobility to form stable and well-sized clusters of IoT nodes while avoiding global clustering and recurrent re-clustering. The proposed distributed method uses Cloud resources along with clusters of mobile and inert Fog nodes to schedule time-constrained workflow applications with considering a proper balance between contradicting criteria and promoting scalability and adaptability. The Velociraptor simulator (version 0.6.7) has been used to throughtly examine and compare the proposed method in real workloads with two contemporary and noteworthy methods. The evaluation results show the superiority of the proposed method as the resource utilization is about 20% better and the schedule success rate is almost 21% better compared with the two other methods. Also, other parameters such as throughput and energy consumption have been studied and reported.

Enhanced hybrid multi-objective workflow scheduling approach based artificial bee colony in cloud computing

This paper presents a hybrid approach based Binary Artificial Bee Colony (BABC) and Pareto Dominance strategy for scheduling workflow applications considering different Quality of Services (QoS) requirements in cloud computing. The main purpose is to schedule a given application onto the available machines in the cloud environment with minimum makespan (i.e. schedule length) and processing cost while maximizing resource utilization without violating Service Level Agreement (SLA) among users and cloud providers. The proposed approach is called Enhanced Binary Artificial Bee Colony based Pareto Front (EBABC-PF). Our proposed approach starts by listing the tasks according to priority defined by Heterogeneous Earliest Finish Time (HEFT) algorithm, then gets an initial solution by applying Greedy Randomized Adaptive Search Procedure (GRASP) and finally schedules tasks onto machines by applying Enhanced Binary Artificial Bee Colony (BABC). Further, several modifications are considered with BABC to improve the local searching process by applying circular shift operator then mutation operator on the food sources of the population considering the improvement rate. The proposed approach is simulated and implemented in the WorkflowSim which extends the existing CloudSim tool. The performance of the proposed approach is compared with Heterogeneous Earliest Finish Time (HEFT) algorithm, Deadline Heterogeneous Earliest Finish Time (DHEFT), Non-dominated Sort Genetic Algorithm (NSGA-II) and standard Binary Artificial Bee Colony (BABC) algorithm using different sizes of tasks and various benchmark workflows. The results clearly demonstrate the efficiency of the proposed approach in terms of makespan, processing cost and resources utilization.

Energy and cost aware workflow scheduling in clouds with deadline constraint

AbstractCloud computing is a promising platform for executing scientific workflow applications. Commonly, the scientific workflow applications are complex in size and computation intensive. Task scheduling in clouds is a familiar NP‐complete problem. Efficient workflow application scheduling is critical for meeting multi‐objectives in cloud environments. By reason of its pivotal role, this problem has been widely studied, and many approaches have been developed. Most of the algorithms focus on schedule makespan and execution cost. The cloud data centers consume a large amount of energy while running workflow applications due to a lack of efficient scheduling algorithms to perform the task to a virtual machine (VM) mapping. Hence, there is a significant need to address energy utilization in the cloud data center (CDC) for two reasons such as data center operational cost optimization and to improve the environment. This article presents an energy and cost‐aware scheduling (ECWS) approach with the objectives of decreasing energy utilization, execution cost, and maximizing utilization of the resources. The ECWS algorithm is a heterogeneous earliest finish time (HEFT) based energy‐efficient heuristic for cloud scheduler. The ECWS algorithm includes three sub‐algorithms like RE calculation, RE Threshold selection, and slack algorithm. The effectiveness of the proposed algorithm is evaluated on the WorkflowSim tool using distinct workloads from various scientific areas. The experimental outcome exhibit that the proposed algorithm achieved significant energy conservation, maximized resource utilization, and cost‐saving when compared with related well‐known algorithms.

MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems

Workflow scheduling is a long-studied problem in parallel and distributed computing (PDC), aiming to efficiently utilize compute resources to meet user's service requirements. Recently proposed scheduling methods leverage the low response times of edge computing platforms to optimize application Quality of Service (QoS). However, scheduling workflow applications in mobile edge-cloud systems is challenging due to computational heterogeneity, changing latencies of mobile devices and the volatile nature of workload resource requirements. To overcome these difficulties, it is essential, but at the same time challenging, to develop a long-sighted optimization scheme that efficiently models the QoS objectives. In this work, we propose MCDS: Monte Carlo Learning using Deep Surrogate Models to efficiently schedule workflow applications in mobile edge-cloud computing systems. MCDS is an Artificial Intelligence (AI) based scheduling approach that uses a tree-based search strategy and a deep neural network-based surrogate model to estimate the long-term QoS impact of immediate actions for robust optimization of scheduling decisions. Experiments on physical and simulated edge-cloud testbeds show that MCDS can improve over the state-of-the-art methods in terms of energy consumption, response time, SLA violations and cost by at least 6.13, 4.56, 45.09 and 30.71 percent respectively.

Comparison of Algorithms for W orkflow Applications in C loud C omputing

Cloud computing model has evolved to deliver resources on pay per use model to businesses, service providers and end-users. Workflow scheduling has become one of the research trends in cloud computing as many applications in scientific, business, and big data processing can be expressed in the form of a workflow. The scheduling aims to execute scientific or synthetic workloads on the cloud by utilizing the resources by meeting QoS requirements, makespan, energy and cost. There has been extensive research in this area to schedule workflow applications in a distributed environment, to execute background tasks in IoT applications, event-driven and web applications. This paper focuses on the comprehensive survey and classification of workflow scheduling algorithms designed for the cloud.

Distributed scheduling method for multiple workflows with parallelism prediction and DAG prioritizing for time constrained cloud applications

Fog computing is an emerging popular paradigm that extends the availability of resources to the network's edge in order to improve the quality metrics of existing Cloud-based applications. However, scheduling workflow applications with time-constraints are complex regarding the count of resources, physical topology of clusters, and the structure of the task graph of the workflows. Adding Fog resources to the intricate problem space of Cloud-based scheduling needs even more time-consuming and complicated algorithms. In this paper, a multi-criteria Mamdani fuzzy algorithm is proposed to analyze the workflow graphs with the assistance of a Long-Short Term Memory neural network parallelism prediction module. The group-based priority assignment schema performed by the fuzzy inference system assigns a priority value to workflows to indicate the relative precedence of requests. Distributed schedulers then send the workflows to target sites according to their current workloads. The whole process is performed in a decentralized manner to prevent any bottlenecks. We have used an extensive software simulation study to compare the proposed algorithm in real workloads with two recent and notable algorithms. The simulation results confirm the proposed algorithm's superiority in fulfilling time-constraints, resource utilization, and overall application scheduling success rate.

Uncertainty-Aware Online Scheduling for Real-Time Workflows in Cloud Service Environment

Scheduling workflows in cloud service environment has attracted great enthusiasm, and various approaches have been reported up to now. However, these approaches often ignored the uncertainties in the scheduling environment, such as the uncertain task start/execution/finish time, the uncertain data transfer time among tasks, the sudden arrival of new workflows. Ignoring these uncertain factors often leads to the violation of workflow deadlines and increases service renting costs of executing workflows. This study devotes to improving the performance for cloud service platforms by minimizing uncertainty propagation in scheduling workflow applications that have both uncertain task execution time and data transfer time. To be specific, a novel scheduling architecture is designed to control the count of workflow tasks directly waiting on each service instance (e.g., virtual machine and container). Once a task is completed, its start/execution/finish time are available, which means its uncertainties disappearing, and will not affect the subsequent waiting tasks on the same service instance. Thus, controlling the count of waiting tasks on service instances can prohibit the propagation of uncertainties. Based on this architecture, we develop an unceRtainty-aware Online Scheduling Algorithm (ROSA) to schedule dynamic and multiple workflows with deadlines. The proposed ROSA skillfully integrates both the proactive and reactive strategies. During the execution of the generated baseline schedules, the reactive strategy in ROSA will be dynamically called to produce new proactive baseline schedules for dealing with uncertainties. Then, on the basis of real-world workflow traces, five groups of simulation experiments are carried out to compare ROSA with five typical algorithms. The comparison results reveal that ROSA performs better than the five compared algorithms with respect to costs (up to 56 percent), deviation (up to 70 percent), resource utilization (up to 37 percent), and fairness (up to 37 percent).

Reinforcement Learning for Security-Aware Workflow Application Scheduling in Mobile Edge Computing

Mobile edge computing as a novel computing paradigm brings remote cloud resource to the edge servers nearby mobile users. Within one-hop communication range of mobile users, a number of edge servers equipped with enormous computation and storage resources are deployed. Mobile users can offload their partial or all computation tasks of a workflow application to the edge servers, thereby significantly reducing the completion time of the workflow application. However, due to the open nature of mobile edge computing environment, these tasks, offloaded to the edge servers, are susceptible to be intentionally overheard or tampered by malicious attackers. In addition, the edge computing environment is dynamical and time-variant, which results in the fact that the existing quasistatic workflow application scheduling scheme cannot be applied to the workflow scheduling problem in dynamical mobile edge computing with malicious attacks. To address these two problems, this paper formulates the workflow scheduling problem with risk probability constraint in the dynamic edge computing environment with malicious attacks to be a Markov Decision Process (MDP). To solve this problem, this paper designs a reinforcement learning-based security-aware workflow scheduling (SAWS) scheme. To demonstrate the effectiveness of our proposed SAWS scheme, this paper compares SAWS with MSAWS, AWM, Greedy, and HEFT baseline algorithms in terms of different performance parameters including risk probability, security service, and risk coefficient. The extensive experiments results show that, compared with the four baseline algorithms in workflows of different scales, the SAWS strategy can achieve better execution efficiency while satisfying the risk probability constraints.

Distributed Grey Wolf Optimizer for scheduling of workflow applications in cloud environments

Optimal scheduling of workflows in cloud computing environments is an essential element to maximize the utilization of Virtual Machines (VMs). In practice, scheduling of dependent tasks in a workflow requires distributing the tasks to the available VMs on the cloud. This paper introduces a discrete variation of the Distributed Grey Wolf Optimizer (DGWO) for scheduling dependent tasks to VMs. The scheduling process in DGWO is modeled as a minimization problem for two objectives: computation and data transmission costs. DGWO uses the largest order value (LOV) method to convert the continuous candidate solutions produced by DGWO to discrete candidate solutions. DGWO was experimentally tested and compared to well-known optimization-based scheduling algorithms (Particle Swarm Optimization (PSO), Grey Wolf Optimizer). The experimental results suggest that DGWO distributes tasks to VMs faster than the other tested algorithms. Besides, DGWO was compared to PSO and Binary PSO (BPSO) using WorkflowSim and scientific workflows of different sizes. The obtained simulation results suggest that DGWO provides the best makespan compared to the other algorithms.

MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems

Workflow scheduling is a long-studied problem in parallel and distributed computing (PDC), aiming at efficiently utilizing compute resources to meet user's service requirements. Recently proposed scheduling methods leverage the low response times of edge computing platforms to optimize application Quality of Service (QoS). However, scheduling workflow applications in mobile edge-cloud systems is challenging due to computational heterogeneity, changing latencies of mobile devices and the volatile nature of workload resource requirements. To overcome these difficulties it is important, but at the same time challenging, to develop a long-sighted optimization scheme with efficient modelling of the QoS objectives. In this work, we propose MCDS: Monte Carlo Learning using Deep Surrogate Models to efficiently schedule workflow applications in mobile edge-cloud computing systems. MCDS is an Artificial Intelligence (AI) based scheduling approach that uses a tree-based search strategy and a deep neural network based surrogate model to estimate the long-term QoS impact of immediate actions for robust optimization of scheduling decisions. Experiments on physical and simulated edge-cloud testbeds show that MCDS can improve over the state-of-the-art methods in terms of energy consumption, response time, SLA violations and cost by at least 6.13, 4.56, 45.09 and 30.71 percent respectively.

Discrete Island-Based Cuckoo Search with Highly Disruptive Polynomial Mutation and Opposition-Based Learning Strategy for Scheduling of Workflow Applications in Cloud Environments

The optimization-based scheduling algorithms used for scheduling workflows in cloud computing environments may easily get trapped in local optima, especially in the beginning of their simulation processes because of some limitations in their exploration methods. Moreover, the performance of some optimization-based scheduling algorithms may severely degrade when dealing with medium- or large-size scheduling problems. The Island-based Cuckoo Search with highly disruptive polynomial mutation (iCSPM) algorithm is a parallel version of the Cuckoo Search (CS) algorithm. The iCSPM algorithm incorporates the island model into CS and uses an exploration function based on the highly disruptive polynomial mutation. It has been empirically proven that iCSPM performs better than popular optimization algorithms (e.g., CS and island-based Genetic algorithm). This paper presents a variation of iCSPM called Discrete iCSPM with opposition-based learning strategy (DiCSPM) for scheduling workflows in cloud computing environments based on two objectives: computation and data transmission costs. DiCSPM includes two new features compared to iCSPM. First, it uses the opposition-based learning approach (OBL) in the initialization step at the level of islands, where each island in the island model contains the opposite population of another island. Second, the smallest position value method is used in the DiCSPM algorithm to determine the correct values of the decision variables in the candidate solutions. The proposed algorithm was experimentally evaluated and compared to well-known scheduling algorithms [Best Resource Selection, Particle Swarm Optimization (PSO) and Grey Wolf Optimizer] using two types of workflows: balanced and imbalanced workflows. The overall experimental and statistical results indicate that DiCSPM provides solutions for the scheduling problem of workflows in cloud computing environment faster than the other compared algorithms. Moreover, DiCSPM was evaluated and compared to state-of-the-art algorithms, namely PSO, binary PSO and discrete binary cat swarm optimization using scientific workflows of different sizes using WorkflowSim. The obtained results suggest that DiCSPM provides the best makespan compared to the other algorithms.

Dependable Scheduling for Real-Time Workflows on Cyber–Physical Cloud Systems

Cyber-physical cloud systems (CPCS) are integrations of cyber-physical systems (CPS) and cloud computing infrastructures. Integrating CPS into cloud computing infrastructures could improve the performance in many aspects. However, new reliability and security challenges are also introduced. This fact highlights the need to develop novel methodologies to tackle these challenges in CPCS. To this end, this article is oriented toward enhancing the soft-error reliability of real-time workflows on CPCS while satisfying the lifetime reliability, security, and real-time constraints. In this article, we propose a dependable algorithm for scheduling workflow applications on CPCS. The proposed algorithm uses slack to recover failed tasks and allows all tasks to share the available slack in the system. To improve soft-error reliability, the algorithm first determines the priority of tasks, then assigns the maximum frequency to each task, and finally assigns the recoveries to tasks dynamically. Slack also can be used to utilize security services for satisfying system security requirements. The lifetime reliability constraint is met by dynamically scaling down the operating frequency of low-priority tasks. Extensive experiments on real-world workflow benchmarks demonstrate that the proposed scheme reduces the probability of failure by up to $52.1\%$ and improves the scheduling feasibility by up to $83.5\%$ compared to a number of representative approaches.

An Optimal Time-Based Resource Allocation for Biomedical Workflow Applications in Cloud

ABSTRACT Biomedical workflow applications’ necessities have increased with the progress of the biotechnology industry and it involves large volumes of data which require large-scale computation. Due to the upcoming data deluge of biomedical data, processing and scheduling of the same in computing resources have presented many challenges. A biomedical workflow application requires heterogeneous resources for execution. Thus, it utilizes the advantage of cloud computing, as a cloud provides scalable high-performance computing resources. Scheduling of these workflows in the cloud is a crucial problem for which the proposed solution contributes greatly towards execution improvement. The proposed algorithm provides optimal and effective utilization of computational resources by novel scheduling of algorithms. Least Execution Time Cloud Workflow Scheduling (LETCWS) and Revised Heterogeneous Earliest Finish Time (RHEFT) algorithms are proposed and implemented in two distinct levels to schedule the biomedical workflow applications in such a way that it reduces the execution time while reducing the cost. The proposed algorithms are evaluated using the Workflowsim tool with real-world biomedical workflow applications. Experimental results illustrate the performance of scheduling workflow applications over the other existing approaches. The result shows that the proposed algorithm achieves better performance in terms of execution time and cost.

Aggregation Measure Factor-Based Workflow Application Scheduling in Heterogeneous Environments

Aggregation Measure Factor-Based Workflow Application Scheduling in Heterogeneous Environments

I- Rosa, Improved Uncertainty Aware Workflow Scheduling Algorithm

Many applications running in cloud computing environment are workflow applications which contain large number of precedence specific tasks and so require proper schedule in order to complete successfully. Efficient scheduling of workflow applications is a challenging task. In workflows, the uncertainties like ‘uncertain data transfer time’ among dependent tasks and the uncertain task execution time, if ignored may lead to deadline violation. The proposed workflow scheduling algorithms so far unconcerned these uncertainties. This paper presents an improved unceRtainty aware workflow scheduling algorithm abbreviated as i-ROSA, that considers the uncertainties of scheduling workflows such as the uncertain running time of tasks in distributed environment when focused upon gave the superior outcome in way of cost and resource utilization, for DAG when compared with the original algorithm. The compared task scheduling algorithms are implemented in Workflowsim.

Scheduling Workflow Applications with Makespan and Reliability Constraints

In the last few years, workflows are becoming richer and more complex. Workflow scheduling management system to be robust, flexible with multicriteria scheduling algorithms. It needs to satisfy the Quality of Service (QoS) parameters. However, QoS parameters and workflow system objectives are often contradictory. In our analysis, we derived an efficient strategy to minimize the overall processing time for scheduling workflows modelled by using Directed Acyclic Graph (DAG). We studied the problem of workflow scheduling that lead to optimizing makespan and reliability. The proposed algorithm handles unsuccessful job execution or resource failure by dynamically scheduling workflows to available resources. Based on the experiments results, our proposed Failure-Aware Workflow Scheduling (FAWS) Algorithm can significantly optimize the makespan and minimize the reliability by rescheduling the failed task to the unused resources. The effectiveness of the FAWS algorithm was validated based on a simulation-driven analysis based on the workflow application.

A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing

Workflow Scheduling in cloud computing has drawn enormous attention due to its wide application in both scientific and business areas. This is particularly an NP-complete problem. Therefore, many researchers have proposed a number of heuristics as well as meta-heuristic techniques by considering several issues, such as energy conservation, cost and makespan. However, it is still an open area of research as most of the heuristics or meta-heuristics may not fulfill certain optimum criterion and produce near optimal solution. In this paper, we propose a meta-heuristic based algorithm for workflow scheduling that considers minimization of makespan and cost. The proposed algorithm is a hybridization of the popular meta-heuristic, Gravitational Search Algorithm (GSA) and equally popular heuristic, Heterogeneous Earliest Finish Time (HEFT) to schedule workflow applications. We introduce a new factor called cost time equivalence to make the bi-objective optimization more realistic. We consider monetary cost ratio (MCR) and schedule length ratio (SLR) as the performance metrics to compare the performance of the proposed algorithm with existing algorithms. With rigorous experiments over different scientific workflows, we show the effectiveness of the proposed algorithm over standard GSA, Hybrid Genetic Algorithm (HGA) and the HEFT. We validate the results by well-known statistical test, Analysis of Variance (ANOVA). In all the cases, simulation results show that the proposed approach outperforms these algorithms.

Time-discretization for speeding-up scheduling of deadline-constrained workflows in clouds

In this paper we deal with the problem of scheduling workflow applications in multiple infrastructure-as-a-service (IaaS) providers, where the scheduler must determine on which computational resource each component of a workflow should be scheduled in an attempt to minimize the monetary cost of the workflow execution. In our previous work, we describe the λ-granularity approach that increases the degree of discrete time intervals in an integer linear programming (ILP)to speed up the scheduling of workflow applications with deadline constraints. To assist the selection of a specific discrete time interval (λ value) given a workflow and its deadline, we propose in this paper an aggregate objective function which is derived from historical evaluations of scheduling costs and scheduler runtimes. Results of simulations evince that the proposed AOF-based procedure was able to calculate a specific λ value that allowed the reduction of the ILP-based scheduler running time, although yet achieving cost-effective solutions.

An Improved Multi-Objective Optimization for Workflow Scheduling in Cloud Platform

Cloud computing is a promising research domain used to enhance the performance of information systems and applications used at our daily life. Workflow scheduling in cloud platform is a complex optimization problem because of its heterogeneous nature, complex billing models and infinite resources requirements which has lead the users to look for an optimal or a suboptimal solution that satisfy multiple competing goals such as minimum makespan and energy efficient solutions. The focus of this paper is to schedule workflow applications, optimizing makespan and energy consumption. A high-performance method for scheduling workflow on cloud computing platform will be presented in this paper, called non-dominated sorting particle swarm optimization (NSPSO). The proposed method is evaluated on synthetic representation of real world scientific workflow applications. The simulation results shows that the proposed method can provide better workflow scheduling solution for cloud when compared with other state-of-the-art methods.