Scientific Workflow Scheduling Research Articles

Evaluating Scheduling Efficiency for Complex Scientific Workflows in Cloud Infrastructures

Scientific workflow scheduling in Cloud computing is critical for efficiently managing data-exhaustive and compute-rigorous applications. With the rising demand of distributed computing, the interest in feasible assignment planning calculations has developed primarily. Revised text: This paper presents a comprehensive survey of advanced scheduling techniques focusing on minimizing energy consumption and improving resource utilization. Various scheduling algorithms, including heuristic-based, meta-heuristic-based, and reinforcement learning-based methods, are analyzed and compared. Additionally, the paper addresses the challenges of scheduling workflows with complex dependencies, offering a novel multi-objective workflow scheduling algorithm using reinforcement learning. The algorithm outperforms current methods in terms of makespan and energy consumption. Finally, the paper highlights open research issues and future directions in scientific workflow scheduling for distributed computing. Through our experiments, we have achieved significant improvements in scheduling efficiency, demonstrated by various performance metrics illustrated in our graphs.

Scientific workflow scheduling algorithms in cloud environments: a comprehensive taxonomy, survey, and future directions

Scientific workflow scheduling algorithms in cloud environments: a comprehensive taxonomy, survey, and future directions

Enhancing workflow efficiency with a modified Firefly Algorithm for hybrid cloud edge environments

Efficient scheduling of scientific workflows in hybrid cloud-edge environments is crucial for optimizing resource utilization and minimizing completion time. In this study, we evaluate various scheduling algorithms, emphasizing the Modified Firefly Optimization Algorithm (ModFOA) and comparing it with established methods such as Ant Colony Optimization (ACO), Genetic Algorithm (GA), and Particle Swarm Optimization (PSO). We investigate key performance metrics, including makespan, resource utilization, and energy consumption, across both cloud and edge configurations. Scientific workflows often involve complex tasks with dependencies, which can challenge traditional scheduling algorithms. While existing methods show promise, they may not fully address the unique demands of hybrid cloud-edge environments, potentially leading to suboptimal outcomes. Our proposed ModFOA integrates cloud and edge computing resources, offering an effective solution for scheduling workflows in these hybrid environments. Through comparative analysis, ModFOA demonstrates improved performance in reducing makespan and completion times, while maintaining competitive resource utilization and energy efficiency. This study highlights the importance of incorporating cloud-edge integration in scheduling algorithms and showcases ModFOA’s potential to enhance workflow efficiency and resource management across hybrid environments. Future research should focus on refining ModFOA’s parameters and validating its effectiveness in practical hybrid cloud-edge scenarios.

Retraction Note: A modified shuffled frog leaping algorithm for scientific workflow scheduling using clustering techniques

Retraction Note: A modified shuffled frog leaping algorithm for scientific workflow scheduling using clustering techniques

Optimizing scientific workflow scheduling in cloud computing: a multi-level approach using whale optimization algorithm

Cloud computing has evolved into an indispensable tool for facilitating scientific research due to its ability to efficiently distribute and process workloads in a virtual environment. Scientific tasks that involve complicated task dependencies and user-defined constraints related to quality of service (QoS) and time constraints require the efficient use of cloud resources. Planning these scientific workflow tasks represents an NP-complete problem, prompting researchers to explore various solutions, including conventional planners and evolutionary optimization algorithms. In this study, we present a novel, multistage algorithm specifically designed to schedule scientific workflows in cloud computing contexts. This approach addresses the challenges of efficiently mapping complex workflows onto distributed cloud resources while considering factors like resource heterogeneity, dynamic workloads, and stringent performance requirements. The algorithm uses the whale optimization algorithm (WOA) with a two-phase approach to shorten execution time, minimize financial costs, and effectively maintain load balancing.

Optimizing execution time and cost while scheduling scientific workflow in edge data center with fault tolerance awareness

Abstract Scheduling scientific workflows is essential for edge data centers operations. Fault tolerance is a crucial focus in workflow scheduling (WS) research. This study proposed fault-tolerant WS in edge data centers using Task Prioritization Adaptive Particle Swarm Optimization (TPAPSO). The aim is to minimize the Makespan, execution costs, and overcoming failures at all workflow processing stages, including when virtual machines are insufficient or tasks fail. The approach proposes three components: initial heuristic list, scheduling tasks with TPAPSO, and implementing performance monitoring with fault tolerance (PMWFT). TPAPSO-PMWFT is simulated using CloudSim 4.0. The experiments indicate that the suggested approach shows superior results compared to existing methods.

Taxonomy and survey of scientific workflow scheduling in infrastructure-as-a-service cloud computing systems

Scientific workflows are groups of scientific application tasks organized in oriented graphs. These scientific workflows are characterized by a large number of tasks requiring sufficient resources for their execution. Tasks with all available input data can be computed simultaneously. Cloud computing is an appropriate environment for the implementation of scientific workflows. Although the cloud computing environment has unlimited resources and can run some scientific workflow tasks simultaneously, scheduling scientific workflow tasks using pay-as-you-go cloud computing resources is an NP-complete problem. This difficulty is due to constraints on the part of the cloud resource provider and the part of the user (customer). The algorithm tries to find efficient schedules that take into account several requirements of client service (QoS) such as deadlines, budgets, and resource providers’ profits, i.e., the minimization of energy consumption and many others. There have been several papers recommending effective solutions to workflow schedule problems. This paper reviews existing and more recent papers on the plan of scientific workflows in pay-as-you-go IaaS cloud computing environments, focusing on future directions for algorithms that can improve the optimal solution.

Combinatorial metaheuristic methods to optimize the scheduling of scientific workflows in green DVFS-enabled edge-cloud computing

A significant challenge in high-performance computing is to ensure the even distribution of applications across computational resources, preventing issues such as resource fragmentation and network congestion. While cloud computing offers advantages, it introduces scheduling delays caused by data transmission. To address this issue, edge computing has emerged as an alternative to traditional cloud systems, aiming to minimize latency. While various methods have been proposed to address this challenge, they often prioritize one aspect at the expense of overall system performance. In this paper, we present a novel algorithm utilizing ant colony optimization to compute a fitness function and prioritize multiple objectives in scheduling. The algorithm effectively determines how to distribute applications between edge and cloud servers to enhance computational efficiency. This entails a delicate balance between scheduling delays and energy consumption in two distinct phases. Initially, the algorithm identifies applications sensitive to delays and ensures their execution on local edge servers. Subsequently, it identifies applications that require intensive computation and migrates them to the cloud layer, where cloud servers can process them. The results demonstrate that this approach reduces delay costs by 21.19% and decreases energy consumption by 13.76%.

HEPGA: A new effective hybrid algorithm for scientific workflow scheduling in cloud computing environment

Cloud data center comprises various physical and virtual machines, alongside storage datacenter services provided by cloud providers. Effectively mapping tasks to optimize resource utilization and load balance is essential for efficient task scheduling. This process, referred to as scheduling constraints, can significantly enhance overall efficiency. However, to harness the benefits of this scheduling, one must address the challenges arising during task execution. The interdependencies between tasks and the diverse resources available in the datacenter pose significant hurdles to efficient resource allocation. To address these challenges, this paper introduces the Hybrid HEFT-PSO-GA algorithm (HEPGA), aiming to efficiently allocate tasks to available resources across the datacenter. The HEPGA algorithm builds upon prior research by integrating the strengths of PSO (Particle Swarm Optimization) and GA (Genetic Algorithm) to optimize task scheduling in cloud computing environments. Through the fusion of PSO, GA, and HEFT-based Initialization, the algorithm strives to efficiently allocate tasks to processors, thereby minimizing the makespan. This approach capitalizes on parallel processing capabilities to further enhance resource utilization in the cloud environment. By varying the weights of the fitness function and considering the resources within the datacenter, we meticulously analyze the algorithm's performance concerning both makespan and Resource Utilization (RU). The results of these tests underscore the algorithm's consistent and robust resource utilization across diverse weight configurations, highlighting its adaptability to varying priorities. Moreover, the observed variations in makespan performance based on different weights emphasize the algorithm's potential for excellence when tailored to specific optimization goals.

Adaptive golden eagle optimization based multi-objective scientific workflow scheduling on multi-cloud environment

Adaptive golden eagle optimization based multi-objective scientific workflow scheduling on multi-cloud environment

A two-stage preference driven multi-objective evolutionary algorithm for workflow scheduling in the Cloud

The workflow scheduling problem considered difficult in the Cloud becomes even more challenging when multiple scheduling criteria are used for optimization. It is much harder to maximize the conflicting interests of users, service providers and the environment, and to obtain “triple win” solutions. This paper presents a two-stage preference driven multi-objective evolutionary algorithm (tsp-MOEA) to address the workflow scheduling problem involving multiple roles. We formulate it as a multi-objective optimization problem with preferences for the first time by considering the special preferences of different participants in decision-making. In particular, a preference distance strategy is introduced in the first stage to speed up the arrival of solutions around the decision makers’ region of interest (ROI); and a special preference region ranking strategy is also designed to focus on searching in the ROI in the second stage. Moreover, the two-stage transition is completed adaptively, which can more fully find a variety of elite solutions. In addition, an elite learning strategy is adopted to guide the global evolution of the population to further enhance the quality of the solution. Extensive experiments demonstrate that the proposed tsp-MOEA approach outperforms recent state-of-the-art algorithms, and can provide significantly better solutions for scientific workflow scheduling in the Cloud.

Precise makespan optimization via hybrid genetic algorithm for scientific workflow scheduling problem

AbstractTask scheduling in scientific workflows represents an NP-hard problem due to the number of interdependent tasks, data transfers, and the possible execution infrastructure assignments in cloud computing. For this reason, metaheuristics are one of the most widely applied optimisation techniques. Makespan is one of the main objectives in this problem. However, this metric needs to be complemented with a quality measure with respect to the actual execution time in order to avoid incurring more costs than expected by using an over-optimistic approximation. This research applies a new enhanced disk-network-computing evaluation model, that takes into account the communication among the storage devices involved, which plays an important role in actual schedules. The model is implemented in a genetic algorithm and the well-known heuristic HEFT. We propose different hybridisation metaheuristics in conjunction with a new accuracy metric to measure the difference between the makespan approximations and the real one. The new evaluation model is able to improve accuracy with respect to the standard model, and the proposed hybrid methods significantly improve makespan in the case of heterogeneous infrastructures.

The Application of Hybrid Krill Herd Artificial Hummingbird Algorithm for Scientific Workflow Scheduling in Fog Computing

The Application of Hybrid Krill Herd Artificial Hummingbird Algorithm for Scientific Workflow Scheduling in Fog Computing

A Multi-Objective Cloud Workflow Scheduling Strategy Based on the Modified HEFT Algorithm

Aiming at the issues of poor system resource utilization and the challenge of ensuring the quality of service (QoS) in scientific workflow scheduling in the current cloud computing environment. By combining the existing IaaS cloud resource model with the scientific workflow task model, the article proposed a cloud workflow scheduling model framework. Firstly, proposed multi-objective QoS constraints based on the execution time and costs in scientific workflow scheduling. Then, modified the HEFT algorithm which was a typical scientific workflow scheduling algorithm. Based on the modified HEFT algorithm makes full use of the idle time of resources and maximizes the multi-objective QoS requirements of users. The comparative experiments show that the scheduling strategy of the modified HEFT algorithm can effectively optimize the costs and execution time of scientific workflow scheduling and achieve higher system resource utilization.

Chaotic hybrid multi-objective optimization algorithm for scientific workflow scheduling in multisite clouds

A cloud is made up of many data centers, with its own set of data and resources. The reasons for employing several cloud sites to operate a workflow are that the data is already dispersed, the required resources surpass the constraints of a single site. This paper presents a hybrid multi-objective optimization algorithm denoted as HSOS-SOA, achieved by combining the Symbiotic Organisms Search and Seagull Optimization Algorithm. The HSOS-SOA uses chaotic maps to generate random numbers and performs a good trade-off between exploration and exploitation, resulting in a higher convergence rate. HSOS-SOA is used to solve scientific workflow scheduling problems in multisite cloud computing by taking into consideration elements such as makespan, cost, and reliability. A solution is chosen from the Pareto front using the knee-point approach in this approach. Extensive analyses are performed out in Microsoft Azure multisite cloud and the results exhibited that the HSOS-SOA can outperform other algorithms in terms of metrics such as IGD, Coverage Ratio, and so on. Experimental results of experiments reveal that the results in makespan improvement in the range of 5.72–28.61%, cost in the range of 5.16–45.16%, and reliability in the range of 3.11–25% over well-known metaheuristic algorithms.

HGPSO: An efficient scientific workflow scheduling in cloud environment using a hybrid optimization algorithm

Cloud technology has raised significant prominence providing a unique market economic approach for resolving large-scale challenges in heterogeneous distributed systems. Through the use of the network, it delivers secure, quick, and profitable information storage with computational capability. Cloud applications are available on-demand to meet a variety of user QoS standards. Due to a large number of users and tasks, it is important to achieve efficient scheduling of tasks submitted by users. One of the most important and difficult non-deterministic polynomial-hard challenges in cloud technology is task scheduling. Therefore, in this paper, an efficient task scheduling approach is developed. To achieve this objective, a hybrid genetic algorithm with particle swarm optimization (HGPSO) algorithm is presented. The scheduling is performed based on the multi-objective function; the function is designed based on three parameters such as makespan, cost, and resource utilization. The proper scheduling system should minimize the makespan and cost while maximizing resource utilization. The proposed algorithm is implemented using WorkflowSim and tested with arbitrary task graphs in a simulated setting. The results obtained reveal that the proposed HGPSO algorithm outperformed all available scheduling algorithms that are compared across a range of experimental setups.

Comments on “IPPTS: An Efficient Algorithm for Scientific Workflow Scheduling in Heterogeneous Computing Systems”

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IPPTS</i> (Improved Predict Priority Task Scheduling) is a list scheduling algorithm that schedules task graphs on fully connected heterogeneous distributed systems, with an objective of minimizing the overall makespan (i.e., schedule length). With respect to the literature on list scheduling techniques for task graphs, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IPPTS</i> improves the task prioritization by considering the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">“out-degree”</i> of a task. However, we have observed that the IPPTS algorithm contains an ambiguity which introduces <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the possibility of assigning higher priority to a task compared to its predecessors in a task graph</i> . This priority inversion <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">may lead to the generation of an incorrect schedule</i> due to the violation of precedence-constraints among tasks. In this note, we first highlight this issue using a counter example. Then, we discuss two possible ways to fix the ambiguity in the algorithm.

Cost effective hybrid genetic algorithm for scheduling scientific workflows in cloud under deadline constraint

Cloud has emerged as a convenient platform for executing complicated scientific applications from multiple disciplines by providing on-demand and scalable infrastructure on rental basis. Research and scientific community often opt for workflows to model these scientific applications Workflow scheduling has been extensively studied for decades with regard to grid and cluster computing, but few initiatives have been tailored for cloud. What's more, the previous work fails to incorporate the basic principles of IaaS clouds like pay-as-you-go model, elasticity, heterogeneity, dynamic provisioning and issues of VM's performance variation and acquisition delay besides other QoS requirements. This paper proposes a resource provisioning and scheduling strategy using genetic algorithm with the aim to optimise the overall execution cost while staying below the given deadline. The performance is further enhanced by using a high quality seed generated by predict earliest finish time (PEFT) algorithm which acts as a catalyst and helps the algorithm to converge faster. The proposed approach is simulated in WorkflowSim and evaluated using various well-known different sized realistic scientific workflows. The results validate the better performance of our approach over numerous state-of-art-algorithms.

Multi-criteria scheduling of scientific workflows in the Workflow as a Service platform

The increasing use of workflows in various fields of science has led to the development of the Workflow as a Service (WaaS) platform, which provides workflow execution as a service to the users. The WaaS platform must use appropriate resource provisioning, sharing, and scheduling, to properly allocate the workflow tasks to the Cloud resources and fulfill the scheduling objectives. In this paper, we have proposed a novel workflow scheduling approach for the WaaS platform that reduces the average makespan of the workflows, improves energy efficiency, and reduces the resource renting cost of the Cloud resources. Our scheduling model consists of containers, virtual machines (VMs), and hosts. We have also proposed a suitable scaling policy for the Cloud resources to improve resource utilization. Extensive simulations with real-world workflows and comparison with some state-of-the-art algorithms show the efficacy of our approach in improving performance and energy efficiency while reducing the monetary cost.

An improved discrete harris hawk optimization algorithm for efficient workflow scheduling in multi-fog computing

Workflow scheduling in fog computing is an NP-hard problem that tries to allocate the best possible set of resources for the workflows considering various objectives such as deadlines, costs, energy, and Quality of Service (QoS). However, fog providers may be under heavy loads from the IoT networks, leading to Service Level Agreement (SLA) violations. This article considers an architecture consisting of multiple fog computing providers and provides a Hidden Markov Model (HMM) model for predicting the availability of each fog computing provider regarding factors such as the number of incoming requests to each fog, deadline-missed workflows, and the offloaded tasks from the fogs to the cloud computing. This HMM model is trained using the unsupervised Baum-Welch algorithm, and the availability probability of each fog is computed using the Viterbi algorithm. Then, the fog provider availability probability is used to select a fog computing provider and schedule IoT workflows on it. Additionally, we have improved Harris Hawks Optimization (HHO) algorithm and presented a Discrete Opposition-based of this algorithm, denoted as DO-HHO, for scientific workflow scheduling. The results of extensive experiments conducted using iFogSim demonstrate that our proposed scheme can significantly reduce offloaded tasks on cloud computing, deadline-missed workflows, and SLA violations, outperforming state-of-the-art works.