Job Scheduling In Grid Research Articles

Teaching learning-based optimisation for job scheduling in computational grids

Teaching learning-based optimisation for job scheduling in computational grids

An Enhanced Firefly Algorithm for Time ‎‎Shared Grid Task ‎Scheduling‎

ABSTRACT Grid computing is a computational paradigm that emerged to ‎‎handle the increasing demand for ‎computational resources. Several metaheuristics methods ‎‎have been applied ‎to tackle the grid task scheduling problem. ‎‎These metaheuristics generally generate good but not optimal ‎‎task ‎schedules. The aim of this paper is to design and ‎‎implement a grid task scheduling mechanism to map clients’ tasks to ‎‎ ‎available resources in order to finish the submitted tasks ‎‎within the optimal execution time. The paper proposes ‎an ‎‎enhanced time shared metaheuristics mechanism based on ‎‎Firefly Algorithm to ‎‎improve the grid job scheduling process. The proposed mechanism utilizes the Smallest Position ‎Value (SPV) technique to handle the scheduling problem as ‎permutations. Experiments using ‎‎simulations and real workload traces were ‎conducted to study ‎‎the performance of the proposed enhanced time shared ‎‎metaheuristic scheduling mechanism. ‎Empirical results revealed ‎‎that the proposed timed shared ‎metaheuristic algorithm can efficiently reduce the makespan time to 1851 compared with 3482, 3185 for Tabu search and genetic algorithm, respectively.

Parallel Scheduling of Grid Jobs on Quadcore Systems using Grouping Methods

As Grid computing continues to make inroads into different spheres of our lives and multicore computers become ubiquitous, the need to leverage the gains of multicore computers for the scheduling of Grid jobs becomes a necessity. Most Grid schedulers remain sequential in nature and are inadequate in meeting up with the growing data and processing need of the Grid. Also, the leakage of Moore’s dividend continues as most computing platforms still depend on the underlying hardware for increased performance. Leveraging the Grid for the data challenge of the future requires a shift away from the traditional sequential method. This work extends the work of [1] on a quadcore system. A random method was used to group machines and the total processing power of machines in each group was computed, a size proportional to speed method is then used to estimates the size of jobs for allocation to machine groups. The MinMin scheduling algorithm was implemented within the groups to schedule a range of jobs while varying the number of groups and threads. The experiment was executed on a single processor system and on a quadcore system. Significant improvement was achieved using the group method on the quadcore system compared to the ordinary MinMin on the quadcore. We also find significant performance improvement with increasing groups. Thirdly, we find that the MinMin algorithm also gained marginally from the quadcore system meaning that it is also scalable.

Enhancing Parallel Scheduling of Grid Jobs in a Multicored Environment

The computing Grid has emerged as a platform to solve the complex and ever-increasing processing need of man and advances in computing technology have birthed the multicore era aimed for high throughput and efficient parallel computing. However, most systems still rely on the underlying hardware for parallelism despite the hard evidence that sequential algorithms do not optimally exploit parallel systems. This research seeks to harness the benefits of multicore systems using job and machine grouping methods to enhance parallelism in the scheduling of Grid jobs. The paper presents the result of two separate experiments on a method that parallelize scheduling algorithm on two multicore platforms. An arbitrary method was employed to group machines; a summation of the total processing power of machines in each group was made. To ensure load balancing, jobs were allocated to machine groups based on the ratio of the total processing power of the machines in each group. The MinMin Grid scheduling algorithm was implemented independently within the groups using a range of threads varied in powers of two. Also, the numbers of groups were varied between 2, 4, and 8. The same experiment was executed on a single processor computer; a duocore machine and a quadcore machine. A performance improvement of 16% to 85% was recorded by the group method against the best ordinary MinMin results and an improvement of 50% to 84% was recorded by the group method against the ordinary MinMin on corresponding machines. We prove that an increase in the number of groups results in improved performance on corresponding machines (approximately 2 times using 2 groups, approximately 3 times using four groups, and approximately 6 times using 8 groups). And most importantly, we established that as the number of processors increases, the grouping method makes more significant improvements over the ordinary MinMin scheduling algorithm executed on the multicore systems.

Performance-aware energy-efficient parallel job scheduling in HPC grid using nature-inspired hybrid meta-heuristics

High-Performance Computing (HPC) systems offer massive computation strength to execute large-scale applications. However, the availability of thousands of CPU cores in the HPC Systems has also triggered a significant increase in the associated energy consumption translating to higher energy expenses of system providers and carbon emissions in the environment. Therefore efficient job schedulers, which can trade-off between user-desired performance and conflicting energy-efficiency objectives simultaneously, are the need of the hour and must nowadays. Job scheduling in HPC systems is a known NP-Hard problem for which meta-heuristics may provide a near-to-optimal solution. Cuckoo search (CS) is a well-known robust swarm-intelligence based meta-heuristic, which has been applied extensively in many optimization problems due to the strong searching efficiency and requirement of very few tuning parameters. However, it suffers from the likelihood of trapping in the local minima and lack of solution diversity towards the end of the algorithm. These drawbacks could result in unacceptable results when the CS algorithm applies to the parallel job scheduling problem. To overcome these limitations and improve the searching efficiency of the traditional CS, we have proposed a multi-objective hybrid scheduling algorithm called MOHCSFA to optimally schedule the batch of parallel jobs in HPC Grid. The proposed MOHCSFA policy combines the solution search mechanisms of both Cuckoo Search (CS) and Firefly algorithm (FA) during each generation. Our proposed policy is further integrated with efficient resource allocation (ERA) heuristic to improve job scheduler performance by effectively using multi-site resource allocation. The experiments are conducted on the GridSim simulator and the benchmarking of the proposed algorithm is done using real data-sets extracted from two supercomputing workload logs. The simulation results showed that the proposed MOHCSFA policy outperforms many heuristics and meta-heuristic scheduling policies for different test cases for both performance and energy-efficiency objectives. Specifically, in the case of Unilu-Gaia workloads, the MOHCSFA obtained 5.87–24.05%, 3.46–28.50%, and 7.06–26.76% performance improvement for the makespan, energy consumption and avg. flowtime, respectively over other tested scheduling policies. The statistical tests validated the stability and robustness of the proposed policy over other scheduling policies.

Application-aware deadline constraint job scheduling mechanism on large-scale computational grid.

Recently, computational Grids have proven to be a good solution for processing large-scale, computation intensive problems. However, the heterogeneity, dynamics of resources and diversity of applications requirements have always been important factors affecting their performance. In response to these challenges, this work first builds a Grid job scheduling architecture that can dynamically monitor Grid computing center resources and make corresponding scheduling decisions. Second, a Grid job model is proposed to describe the application requirements. Third, this paper studies the characteristics of commercial interconnection networks used in Grids and forecast job transmission time. Fourth, this paper proposes an application-aware job scheduling mechanism (AJSM) that includes periodic scheduling flow and a heuristic application-aware deadline constraint job scheduling algorithm. The rigorous performance evaluation results clearly demonstrate that the proposed application-aware job scheduling mechanism can successful schedule more Grid jobs than the existing algorithms. For successful scheduled jobs, our proposed AJSM method is the best algorithm for job average processing time and makespan.

Job Scheduling in Computational Grid Using a Hybrid Algorithm Based on Particle Swarm Optimization and Extremal Optimization

Grid computing has been used as a new paradigm for solving large and complex scientific problems using resource sharing mechanism through many distributed administrative domains. One of the most challenging issues in computational Grid is efficient scheduling of jobs, because of distributed heterogeneous nature of resources. In other words, the job scheduling in computational Grid is an NP-hard problem. Thus, the use of meta-heuristic is more appropriate option in obtaining optimal results. In this article, the authors propose a novel hybrid scheduling algorithm which combines intelligently the exploration ability of Particle Swarm Optimization (PSO) with the exploitation ability of Extremal Optimization (EO) which is a recently developed local-search heuristic method. The hybrid PSO-EO reduces the schedule makespan, processing cost, and job failure rate and improves resource utilization. The proposed hybrid algorithm is compared with the standard PSO, population-based EO (PEO) and standard Genetic Algorithm (GA) methods on all these parameters. The comparison results exhibit that the proposed algorithm outperforms other three algorithms.

The Impact of the Implementation Cost of Replication in Data Grid Job Scheduling

Data Grids deal with geographically-distributed large-scale data-intensive applications. Schemes scheduled for data grids attempt to not only improve data access time, but also aim to improve the ratio of data availability to a node, where the data requests are generated. Data replication techniques manage large data by storing a number of data files efficiently. In this paper, we propose centralized dynamic scheduling strategy-replica placement strategies (CDSS-RPS). CDSS-RPS schedule the data and task so that it minimizes the implementation cost and data transfer time. CDSS-RPS consists of two algorithms, namely (a) centralized dynamic scheduling (CDS) and (b) replica placement strategy (RPS). CDS considers the computing capacity of a node and finds an appropriate location for the job. RPS attempts to improve file access time by using replication on the basis of number of accesses, storage capacity of a computing node, and response time of a requested file. Extensive simulations are carried out to demonstrate the effectiveness of the proposed strategy. Simulation results demonstrate that the replication and scheduling strategies improve the implementation cost and average access time significantly.

A Novel Hybrid Algorithm Based on Firefly Algorithm and Differential Evolution for Job Scheduling in Computational Grid

Scheduling jobs in computational Grids is considered as NP-complete problem owing to the heterogeneity of shared resources. The resources belong to many distributed administrative domains that enforce various management policies. Therefore, the use of meta-heuristics are more appropriate option in obtaining optimal results. In this article, a novel hybrid population-based global optimization algorithm, called the Hybrid Firefly Algorithm and the Differential Evolution (HFA-DE), is proposed by combining the merits of both the Firefly Algorithm and Differential Evolution. The Firefly Algorithm and the Differential Evolution are executed in parallel to support information sharing amongst the population and thus enhance searching efficiency. The proposed HFA-DE algorithm reduces the schedule makespan, processing cost, and improves resource utilization. The HFA-DE is compared with the standard Firefly Algorithm, the Differential Evolution and the Particle Swarm Optimization algorithms on all these parameters. The comparison results exhibit that the proposed algorithm outperforms the other three algorithms.

Decentralised grid scheduling approach based on multi‐agent reinforcement learning and gossip mechanism

As an important class of resource allocation approaches, decentralised job scheduling in large-scale grids has to deal with the difficulties in acquiring timely model information and improving performance by autonomous coordination. In this study, a gossip-based reinforcement learning (GRL) method is proposed for decentralised job scheduling in grids. In the GRL method, a decentralised scheduling architecture based on multi-agent reinforcement learning is presented to improve the scalability and adaptability of job scheduling. A gossip mechanism is designed to realise autonomous coordination among the decentralised schedulers. Simulation results show that the proposed GRL-based schedulers can complete the task of grid job scheduling effectively and achieve load balancing efficiently.

Teaching Learning Based Optimization for Job Scheduling in Computational Grids

Teaching Learning Based Optimization for Job Scheduling in Computational Grids

An improved ant colony optimization algorithm with fault tolerance for job scheduling in grid computing systems.

The Grid scheduler, schedules user jobs on the best available resource in terms of resource characteristics by optimizing job execution time. Resource failure in Grid is no longer an exception but a regular occurring event as resources are increasingly being used by the scientific community to solve computationally intensive problems which typically run for days or even months. It is therefore absolutely essential that these long-running applications are able to tolerate failures and avoid re-computations from scratch after resource failure has occurred, to satisfy the user’s Quality of Service (QoS) requirement. Job Scheduling with Fault Tolerance in Grid Computing using Ant Colony Optimization is proposed to ensure that jobs are executed successfully even when resource failure has occurred. The technique employed in this paper, is the use of resource failure rate, as well as checkpoint-based roll back recovery strategy. Check-pointing aims at reducing the amount of work that is lost upon failure of the system by immediately saving the state of the system. A comparison of the proposed approach with an existing Ant Colony Optimization (ACO) algorithm is discussed. The experimental results of the implemented Fault Tolerance scheduling algorithm show that there is an improvement in the user’s QoS requirement over the existing ACO algorithm, which has no fault tolerance integrated in it. The performance evaluation of the two algorithms was measured in terms of the three main scheduling performance metrics: makespan, throughput and average turnaround time.

Job scheduling in computational grid based on an improved cuckoo search method

Grid computing is a high performance distributed computing platform to solve complex and large-scale scientific problems. It consists of heterogeneous computing resources connected by a network across dynamic and geographically distributed organisations to create a distributed high performance computing infrastructure. Job scheduling in computational Grid is known as NP-complete problem owing to the problem complexity and intractable nature of the problem. Such a problem could be solved using heuristic algorithms. These types of algorithms have the ability to find a near optimal solution in reasonable time rather than the optimal solution in a very long processing time. The primary objective of the scheduling is to minimise the makespan of the system. In this paper, an Improved Cuckoo Search (ICS) optimisation method has been proposed for scheduling user-jobs to available resources so that various performance metrics are optimised. Here, our attention has been focused on the improvement of computational Grid performance in terms of makespan and completion time. The study reveals that the proposed ICS algorithm provides better results in comparison with Cuckoo Search (CS) and Simulated Annealing (SA).

Job scheduling in computational grid based on an improved cuckoo search method

Job scheduling in computational grid based on an improved cuckoo search method

Reliability Driven Constrained Job Scheduling on Volunteer Grid Resources

The job scheduling in grids is a NP-hard problem. Many attempts has been made to solve the job scheduling using different methods. Constraint based job scheduling is evolving in volunteer grid computing environments. The constraint based scheduling provides information about the job requirements like time, cost etc. The paper proposes a job scheduling heuristic to maximize the schedule reliability. The heuristic first choose the capable resources and will assign one of the resources from set which can satisfy the job constraint based on the reliability of schedule. The reliability of schedule has been derived from the closeness of deadline. The simulation results show the significance of closeness of deadline for finding the reliable schedule using the proposed reliability driven job scheduling heuristic. © 2016 American Scientific Publishers. All rights reserved.

Proactive Fault Tolerance Algorithm for Job Scheduling in Computational Grid

Grid use huge number of dynamic, distributed & heterogeneous resources under different organizations domain to executing user applications. Working with these resources, applications face problems due to occurrence of faults. There are number of different faults that occur due to many problems. Therefore grid computing needs an efficient fault tolerance mechanism. This fault tolerance makes the grid system capable to work correctly even in the presence of faults. In this paper we have study various kinds of faults and reasons for occurrence. With this, also study various kinds of existing fault tolerance techniques. We devise a strategy, based on proactive fault tolerance and take appropriate step, if any possibility for fault occurrence. Scheduling of task on available resources, which is provided by GIS consider previous history of these resources. This will decrease the probability of faults, execution time, and increase the execution rate. Proposed strategy also uses technique of check pointing to reduce execution cost. To implement proposed strategy GridSim toolkit is used for simulation

A hierarchical approach for job scheduling in grid computing based on resource prediction and meta-heuristic algorithms

The computational grids as a distributed system are hardware and software infrastructures that are capable of solving large-scale issues, and they use heterogeneous or homogeneous resources scattered around the globe by a high-speed network. Scheduling is a critical and prominent issue in grid computing. An appropriate prediction method for allocating jobs to resources may significantly affect quality of service parameters. In this paper, a hierarchical approach is presented for job scheduling in computational grid utilizing a resource prediction method based on the scoring system. It is inspired by meta-heuristic algorithms in order to improve parameters such as makespan, load balancing and the rate of meeting deadlines. To evaluate the proposed method, GridSim toolkit is exploited. According to the simulation results and comparison with some recent wellknown methods, this approach has been successful in improving the mentioned parameters.

Multicore job scheduling in the Worldwide LHC Computing Grid

After the successful first run of the LHC, data taking is scheduled to restart in Summer 2015 with experimental conditions leading to increased data volumes and event complexity. In order to process the data generated in such scenario and exploit the multicore architectures of current CPUs, the LHC experiments have developed parallelized software for data reconstruction and simulation. However, a good fraction of their computing effort is still expected to be executed as single-core tasks. Therefore, jobs with diverse resources requirements will be distributed across the Worldwide LHC Computing Grid (WLCG), making workload scheduling a complex problem in itself. In response to this challenge, the WLCG Multicore Deployment Task Force has been created in order to coordinate the joint effort from experiments and WLCG sites. The main objective is to ensure the convergence of approaches from the different LHC Virtual Organizations (VOs) to make the best use of the shared resources in order to satisfy their new computing needs, minimizing any inefficiency originated from the scheduling mechanisms, and without imposing unnecessary complexities in the way sites manage their resources. This paper describes the activities and progress of the Task Force related to the aforementioned topics, including experiences from key sites on how to best use different batch system technologies, the evolution of workload submission tools by the experiments and the knowledge gained from scale tests of the different proposed job submission strategies.

An energy-efficient reliable grid scheduling model using NSGA-II

Job scheduling in computational grid is a complex problem and various heuristics and meta-heuristics have been proposed for the same. These approaches usually optimize specific characteristic parameters while allocating the jobs on the grid resources. Many a times, it is desired to optimize multiple parameters during job scheduling. Non-dominated sorting genetic algorithm (NSGA-II) has been observed to be the best meta-heuristic to solve such multi-objective optimization problem. The proposed work applies NSGA-II for job scheduling in computational grid with three conflicting objectives: maximizing reliability of the system for job allocation, minimizing energy consumption and balancing the load on the system. Performance study of the proposed model is done by simulating it on some real data. The result indicates that the proposed model performs well with multiple objectives.

Multiobjective Variable Neighborhood Search algorithm for scheduling independent jobs on computational grid

Grid computing solves high performance and high-throughput computing problems through sharing resources ranging from personal computers to super computers distributed around the world. As the grid environments facilitate distributed computation, the scheduling of grid jobs has become an important issue. In this paper, an investigation on implementing Multiobjective Variable Neighborhood Search (MVNS) algorithm for scheduling independent jobs on computational grid is carried out. The performance of the proposed algorithm has been evaluated with Min–Min algorithm, Simulated Annealing (SA) and Greedy Randomized Adaptive Search Procedure (GRASP) algorithm. Simulation results show that MVNS algorithm generally performs better than other metaheuristics methods.