Efficient Job Scheduling Research Articles

Efficient Job Scheduling in Cloud Environments using Reinforcement Learning Actor-Critic Models

Efficient Job Scheduling in Cloud Environments using Reinforcement Learning Actor-Critic Models

Quality of Service Based on Scheduling in Cloud/Fog Environment

In the area of fog and cloud computing, efficient work scheduling is critical for optimising resource allocation and raising Quality of Service (QOS) levels. Research in this area has resulted in a variety of ways for dealing with this problem. One line of research focuses on lowering data latency and improving QOS in fog-cloud systems by precise job scheduling. This requires intelligently dividing jobs across fog and cloud resources to reduce data processing and delivery delays, ultimately improving the overall user experience. Furthermore, the investigation encompasses novel methodologies such as hybrid evolutionary algorithms and heuristic strategies. These approaches seek to create a compromise between competing objectives, such as cost and energy efficiency, while also achieving QOS criteria. Researchers aim to develop scheduling systems that optimise resource utilisation and reduce operating costs without sacrificing service quality by applying evolutionary concepts or combining heuristic methods. Furthermore, ongoing discussions focus on improving resource scheduling approaches to achieve optimal QOS results. This entails continual evaluation and adaption of scheduling algorithms to changing environmental conditions, workload fluctuations, and user requests. Furthermore, developing methodologies such as the CODA approach provide intriguing avenues for efficient job scheduling in fog computing settings, with the potential to revolutionise future resource allocation and QOS enhancement efforts.

Optimized efficient job scheduling resource (OEJSR) approach using cuckoo and grey wolf job optimization to enhance resource search in cloud environment

Optimized efficient job scheduling resource (OEJSR) approach using cuckoo and grey wolf job optimization to enhance resource search in cloud environment

Big Data Scheduling with Order Acceptance Consideration in Supply Chain

The efficient job scheduling schemes can make full use of resources, and then achieve different goals, such as maximizing efficiency, minimizing cost and saving energy. In supply chain, there are a lot of members and enormous data. Therefore, a suitable scheduling scheme has become the most common and effective method to optimize the execution of big data in supply chain. A scheduling problem with production and delivery consideration, which is usually in supply chain has been considered. For the reason of highly time emergency and random coming orders in quick production and delivery system, the effective algorithms for order acceptance scheduling problem are required. This paper addressed the production and delivery problem which has one manufacturer and multiple customers. There is single machine for order production at the manufacturer. For the consideration of order acceptance or not, the manufacturer need to choose order set to be accepted for processing. The paper aims at finding a balance between orders profit, delivery cost and time-based cost to maximize the total revenue. We consider three main objective functions in scheduling theory, analyze the problem complexity. The complexity of two of the problems are weakly NP hard, and the other one is strongly NP hard. For three weakly NP hard problems, we give pseudo-polynomial time optimal algorithms.

Rock-hyrax: An energy efficient job scheduling using cluster of resources in cloud computing environment

In a cloud computing environment, job scheduling allows the service provider to schedule resources based on demand. Job scheduling must also ensure QoS, end-user satisfaction, and the efficient usage of resources. Cloud computing vendors assign virtualized computing resources to end-users based on job requirements that are dynamically scalable and pay-per-use. The assignment of jobs requires proper investigation and mapping of available resources. In this paper, we have proposed a novel job scheduling scheme based on Rock Hyrax. Our Rock Hyrax approach uses objective functions to map jobs to available resources. The objective function considers a variety of QoS parameters like makespan, response time and energy efficiency. Our method employs two key QoS parameters: makespan and energy consumption. The node behavior and characteristics, such as processing power, storage, and network connectivity to cluster similar resources, have also been considered for scheduling. An experimental setup is created for a thorough study of the proposal using CloudSim simulator. For both the jobs and virtual machines, static and dynamic scenarios for performance evaluation have been developed. To compare our work with existing scheduling algorithms like ACO, PSO, BFO, and ABC has been considered and we have found that the proposal reduces makespan by 2–9% as increased in jobs. Furthermore, the proposed method reduces total energy consumption in data centers by 7–23% as jobs request increases. The findings support the claim that the proposed method surpasses the existing methods and significantly shortens the time needed to determine the resource required for the job.

Multi-objective elitist spotted hyena resource optimized flexible job shop scheduling

The job shop scheduling problem (JSSP) has drained a lot of consideration since it is one of the most important optimization problems in the manufacturing domain. The scheduling method is crucial for optimizing the objective of minimizing makespan among thousands of jobs, but evaluating machine capacity for achieving this goal remains challenging despite the development of various population-based optimization algorithms for job shop scheduling problems. To improve the efficiency of Job shop scheduling, a novel Multi-objective Elitist Spotted Hyena Monotonic Scheduling (MESHS) technique is introduced. The proposed MESHS technique includes two major processes: machine selection and operation sequences. The number of jobs is considered for solving the scheduling problem. First, the machine selection is performed by applying the Multi-objective Elitist Spotted Hyena optimization technique. The optimization technique selects the optimal machines parallelly based on multiple objective functions such as energy consumption, CPU utilization, and job completion time. The fitness of every machine is calculated based on these multiple objective functions using Levenberg–Marquardt method. Then the Elitist strategy is applied to select the optimal machine based on fitness. After the machine selection, the rate-monotonic preemptive scheduling is modeled to provide a robust operation sequence by assigning high-priority jobs to the optimal machines. As a result, efficient job scheduling is achieved with minimum time. Finally, the experimental valuation is carried out using a benchmark OR-Library dataset with different factors such as job shop scheduling efficiency, job scheduling time, makespan, and memory consumption concerning a number of jobs.

Improved QoS with Fog computing based on Adaptive Load Balancing Algorithm

As the number of sensing devices rises, traffic on the cloud servers is boosting day by day. When a device connected to the IoTwants access to data, cloud computing encourages the pairing of fog & cloud nodes to provide that information. One of the key needs in a fog-based cloud system, is efficient job scheduling to decrease the data delay and improve the QoS (Quality of Service). The researchers have used a variety of strategies to maintain the QoS criteria. However, because of the increased service delay caused by the busty traffic, job scheduling is impacted which leads to the unbalanced load on the fog environment. The proposed work uses a novel model which curates the features and working style of Genetic algorithm and the optimization algorithm with the load balancing scheduling on the fog nodes. The performance of the proposed hybrid model is contrasted with the other well-known algorithms in contrast to the fundamental benchmark optimization test functions. The proposed work displays better results in sustaining the task scheduling process when compared to the existing algorithms, which include Round Robin (RR) method, Hybrid RR, Hybrid Threshold based and Hybrid Predictive Based models, which ensures the efficacy of the proposed load balancing model to improve the quality of service in fog environment.

Cuckoo search optimization-based energy efficient job scheduling approach for IoT-edge environment

Recently developed IoT devices are capable of gathering, storing, and processing more data than ever before. This calls for the need for scalability. Through the use of edge computing, more processing functions can be relocated closer to where the data is gathered through the IoT devices. Here, processing tasks may be placed in the edge computing units (ECUs). Each ECU may host a cloudlet consisting of a number of virtual machines, where tasks could be executed. Due to the need to ensure near real-time response of the jobs, efficient job scheduling is required. Few recent works addressed this issue of job scheduling at the edges. However, many important constraints such as job dependency, job conflict along with heterogeneous edge infrastructure are not found to be considered. Accordingly, in this paper, we have proposed an optimal job scheduling approach based on the cuckoo search algorithm to handle these challenges subject to energy efficiency and resource utilization. The proposed algorithm is simulated and found to work notably well as compared to state-of-the-art edge-computing-based job scheduling techniques, especially when the jobs have conflict or dependency among them. The work is reported to achieve above 85% resource utilization even in the presence of job conflicts and dependencies.

Time-Aware Data Partition Optimization and Heterogeneous Task Scheduling Strategies in Spark Clusters

Abstract The Spark computing framework provides an efficient solution to address the major requirements of big data processing, but data partitioning and job scheduling in the Spark framework are the two major bottlenecks that limit Spark’s performance. In the Spark Shuffle phase, the data skewing problem caused by unbalanced data partitioning leads to the problem of increased job completion time. In response to the above problems, a balanced partitioning strategy for intermediate data is proposed in this article, which considers the characteristics of intermediate data, establishes a data skewing model and proposes a dynamic partitioning algorithm. In Spark heterogeneous clusters, because of the differences in node performance and task requirements, the default task scheduling algorithm cannot complete scheduling efficiently, which leads to low system task processing efficiency. In order to deal with the above problems, an efficient job scheduling strategy is proposed in this article, which integrates node performance and task requirements, and proposes a task scheduling algorithm using greedy strategy. The experimental results prove that the dynamic partitioning algorithm for intermediate data proposed in this article effectively alleviates the problem that data skew leads to the decrease of system task processing efficiency and shortens the overall task completion time. The efficient job scheduling strategy proposed in this article can efficiently complete the job scheduling tasks under heterogeneous clusters, allocate jobs to nodes in a balanced manner, decrease the overall job completion time and increase the system resource utilization.

A reinforcement learning based job scheduling algorithm for heterogeneous computing environment

Efficient job scheduling for heterogeneous computing environments has attracted widespread attention, jobs are usually modeled as directed acyclic graphs(DAG). Optimizing scheduling can improve system throughput. We propose a two-stage scheduling algorithm, which calculates task selection and processor allocation respectively. In task selection stage, we utilize a bidirectional graph convolution network to learn DAG structural features, and a fully-connected network to generate proper scheduling scheme. In the processor allocation stage, we propose a heuristic based on optimistic cost table(OCT) and task duplication, which trade-off scheduling allocation better. Experiments of various scheduling scenarios have been conducted, and the results show that the proposed algorithm has better scheduling performance than the compared heterogeneous DAG scheduling algorithms.

Efficient job scheduling paradigm based on hybrid sparrow search algorithm and differential evolution optimization for heterogeneous cloud computing platforms

The job scheduling paradigms include dispatching Internet of Things (IoT) critical services onto processing nodes. Here most energy is consumed in finding suitable virtual machines (VMs) that can execute IoT tasks without resource fragments. Therefore, a significant problem is minimizing energy consumption through efficient task placement that leads to load balance and minimizes resource leakage. To resolve this problem, we proposed a dual-phase metaheuristic algorithm called CSSA-DE. First, we conduct a clustering approach to group computing nodes into effective clusters. Each node is trained at different utilization levels, and the one that can yield the highest Performance-to-Power Ratio (PPR) is selected as the mega cluster head (MCH). Then, we integrated the sparrow search algorithm (SSA) with the differential evolution (DE) algorithm to expand the high search efficiency of finding an appropriate pair task-VM combination. Further, the integration phase can exploit the count of overloaded and underloaded VMs, reducing resource fragments. The performance of CSSA-DE is highly competitive and relatively better in multiple cases compared to state-of-the-art algorithms.

A Novel Resource Scheduler for Resource Allocation and Scheduling in Big Data Using Hybrid Optimization Algorithm at Cloud Environment

Big Medical Data (BMD) is generated by cellular telephones, clinics, academics, suppliers, and organizations. Collecting, finding, analyzing, and managing the big data to make people's lives better, comprehending novel illnesses, and treatments, predicting results at initial phases, and making real-time choices are the actual issues in healthcare systems. Dealing with big medical data in resource scheduling is a major issue that aims to offer higher quality healthcare services. Hadoop MapReduce has been widely used for parallel processing of large data tasks and efficient job scheduling. The number of big data tasks is constantly growing; it is becoming more essential to minimize their energy usage to reduce the environmental effect and operating expenses. Hence to overcome these disadvantages, we propose a novel resource scheduler for big data using a Hybrid 2-GW Optimization Algorithm (H2-GWOA). We employ the Improved GlowWorm Swarm Optimization Algorithm (IGSOA) and Mean GreyWolf Optimization Algorithm (MGWOA) for optimizing the MapReduce framework in heterogeneous big data. The CloudSim platform was used for the simulations. The performance of the proposed scheduler is proved to be better than the conventional methods in terms of metrics like latency, makespan, resource utilization, skewness, and Central Processing Unit (CPU) consumption.

Enhancing the job scheduling procedure to develop an efficient cloud environment using near optimal clustering algorithm

In this internet era, cloud computing and there are various problems in the cloud computing, where the consumers as well as the service providers facing in their day to day cloud activities. Job scheduling problem plays a vital role in the cloud environment. To provide an efficient job scheduling environment, it is necessary to perform efficient resource clustering. In this regard, the proposed system, concentrated on the resource clustering methodology by proposing an efficient resource clustering algorithm named identicalness split up periodic node size (ISPNS) in the cloud environment. The algorithm proposed helps in forming resource clusters with the help of cloud environment. The proposed system is compared with the existing systems to justify the performance of the proposed resource clustering algorithm and it produces near optimal solution for the resource clustering problem which helps to provide an efficient job scheduling in cloud environment.

Addressing Performance and Scalability Issues in Large-Scale Jenkins Deployments

Jenkins, an open-source automation server, has become a cornerstone in modern software development pipelines, facilitating continuous integration and delivery (CI/CD) processes. However, as the scale of Jenkins deployments increases, organizations encounter significant challenges related to performance and scalability. This review paper delves into the various strategies and approaches aimed at mitigating these challenges in large-scale Jenkins deployments. The paper begins by identifying the key performance bottlenecks and scalability limitations inherent in Jenkins at scale. It explores the impact of factors such as increased workload, extensive plugin usage, and complex pipeline configurations on Jenkins’ performance. Subsequently, the review examines existing research and industry practices in addressing these issues. Several techniques have been proposed and implemented to enhance the performance and scalability of Jenkins deployments. These include optimizing resource utilization through efficient job scheduling algorithms, leveraging distributed architectures to distribute workloads across multiple nodes, and employing caching mechanisms to reduce build times and alleviate strain on Jenkins masters. Furthermore, the paper discusses the role of infrastructureas-code (IaC) practices and containerization technologies in streamlining Jenkins deployments and improving scalability. Moreover, the paper highlights the importance of monitoring and instrumentation in identifying performance bottlenecks and optimizing Jenkins deployments continuously. It discusses various monitoring tools and techniques for gathering performance metrics and diagnosing issues in large-scale Jenkins environments. Overall, this paper provides valuable insights into the challenges faced by organizations deploying Jenkins at scale and offers a comprehensive overview of strategies and best practices for addressing performance and scalability issues. By adopting these approaches, organizations can ensure the efficient operation of their Jenkins infrastructure, enabling smoother CI/CD workflows and enhancing overall software delivery capabilities.

Efficient scheduling of jobs on dissimilar parallel machines using heuristic assisted metaheuristic techniques

This work proposes a novel no-wait time heuristic mechanism for identifying optimal schedules for scheduling problems using metaheuristic techniques. The heuristic mechanism helps to reduce the idle time in a machine by shifting the starting time of the job near its release date. The proposed mechanism can be directly integrated with metaheuristic techniques without performing any modifications to the techniques. The proposed heuristic mechanism is employed with fifteen metaheuristic techniques and tested the efficacy by solving twelve scheduling problems with minimization of total processing cost as the objective. The performance of all metaheuristic techniques was analyzed, with and without incorporating the heuristic mechanism. It is observed that the heuristic mechanism helped metaheuristic techniques to determine better schedules, whereas, without the heuristic mechanism, many of these techniques were unable to obtain even one feasible schedule in a complex problem. The inclusion of the proposed heuristic mechanism has provided up to 10% improvement in the fitness value of the best schedules. It also helped to identify multiple realizations of the optimal schedules with zero idle time. This study also analyzed the effect of tightening constraints for solving the scheduling model using metaheuristic techniques. The performance analysis of the techniques validated that the inclusion of horizon constraints does not benefit the metaheuristic techniques in determining better solutions.

Parallel Scheduling of Machines, Tool Transporter and Tools in a Multi Machine FMS with Alternative Routing Using Flower Pollination Algorithm

Scheduling jobs and tools is a significant problem for manufacturing systems. Inefficient job scheduling and tool loading planning may result in the underutilization of capital-intensive machines and a high machine idle time. Tools are transferred by a tool transporter (TT) between machines. Therefore, efficient scheduling of jobs, TT, and the tools with alternate machines enables a manufacturing system to increase machines’ utilization and decrease their idle times. This paper addresses machines, TT, and tools concurrent scheduling with alternate machines and only one copy of every tool variety is made available where the tools are costly, in a multi-machine flexible manufacturing system, taking into account tool transfer times for makespan (MS) minimization. The tools are placed in a central tool magazine, which shares and serves them to many machines to cut down the price of duplicating the tools in each machine. The problem is to assign machines from alternate machines, tools, and corresponding TT trips, including the deadheading trip and loaded trip, to job operations for MS minimization. This paper uses a nonlinear mixed-integer programming framework to present the problem, and a flower pollination algorithm (FPA) is employed to solve it. The results show that FPA outperforms the Jaya algorithm, and the usage of alternate machines for the operations reduces the MS. Reduction in MS indicates an improvement in utilization of resources.

Efficient Scheduling of Jobs and Allocation of Resources in Cloud Computing

Due to the drastic utilization of clouds, a Proper and proficient allocation of resources in dynamically working environment of cloud systems turns into the challenging task. Different promising mechanisms have been created to work on the effectiveness of process of resource allocation. Yet at the same time there is some incompetency as far as resource allocation and job scheduling, when the systems become highly loaded. Hence, an effective algorithm for scheduling of jobs is needed to work on the proficiency of resource allocation activities. In this paper a advanced technique for scheduling of jobs is proposed for effective and unique process of allocation of resources in cloud computing. By making use of prediction-based techniques and mechanism of updating resource tables in dynamic manner, we achieve, better allocation of resources in the form of response time and completion of jobs. The experimental results demonstrate the effective outcomes compared to existing techniques, by achieving exactness in values for resource table updation.

Simultaneous Scheduling of Machines and Tools in a Multi-Machine FMS with Alternate Machines Using Crow Search Algorithm

Scheduling jobs and tools is a significant problem for manufacturing systems. Inefficient job scheduling and tool loading planning may result in under utilization of capital intensive machines and a high level of machine idle time. Therefore, efficient scheduling of jobs and tools enables a manufacturing system to increase machines’ utilization and decrease their idle times. This paper addresses machines’ and tools’ joint scheduling with alternate machines in a multimachine flexible manufacturing system (FMS) to minimize makespan (MSN). Only one copy of each type of tool is made available in FMS where tools are expensive. The tools are stored in the central tool magazine (CTM), which shares and serves them to several machines in order to reduce the cost of duplicating the tools in every machine. The problem is to select machines from alternate machines for job-operations, allocation of tools to the job-operations and job-operations’ sequencing on machines for MSN minimization. This paper presents nonlinear mixed integer programming (MIP) formulation to model this simultaneous scheduling problem and crow search algorithm (CSA) built on the crows’ intelligent behavior for solving this problem. The results show that CSA is providing better solutions than Jaya algorithm and the usage of alternate machines for the operations can reduce MSN.

A Bio-Inspired and Heuristic-Based Hybrid Algorithm for Effective Performance With Load Balancing in Cloud Environment

In a cloud computing environment, effective scheduling policies and load balancing have always been the aim. An efficient task scheduler must be proficient in a dynamically distributed environment and to the policy of efficient scheduling of jobs based upon the workload. In this research, a novel hybrid heuristic algorithm is developed for balancing the load among cloud nodes. This is achieved by hybridizing the existing ant colony optimization (ACO), artificial bee colony algorithm (ABC), and AHP (analytical hierarchy process) algorithm. The AHP algorithm and the artificial bee colony (ABC) algorithm is used for figuring out the best servers suitable for a particular job, and the ant colony algorithm is used to find the most efficient path to that particular server. The proposed algorithm is better in resource utilization. It also performs better load balancing, which keeps on improving with time. The result analysis shows better average response time and better average makespan time compared to other two existing algorithms.

An Agent-based Adaptive Mechanism for Efficient Job Scheduling in Open and Large-scale Environments

An Agent-based Adaptive Mechanism for Efficient Job Scheduling in Open and Large-scale Environments