High Priority Jobs Research Articles

An efficient two-stage optimization algorithm for a flexible job shop scheduling problem with worker shift arrangement

Due to the involvement of workers, scheduling production jobs necessitates consideration of worker shifts in most production activities. In this study, we address a flexible job shop scheduling problem with worker shift arrangement, considering constraints such as job priority, limited resources, and resource unavailability. To minimize the overdue days of low-priority jobs and ensure the timely delivery of high-priority jobs, we establish a mixed-integer programming model to allocate production resources, process sequencing, and schedule worker shifts. An improved differential evolution algorithm is proposed and designed such that overdue days and worker overtime of all jobs are calculated. Furthermore, we develop a two-stage intelligent optimization algorithm. First, we design a two-segment chromosome encoding and decoding method. Then, we propose generation strategies that follow the urgency of the priority rule to generate high-quality initial chromosomes. In adaptive worker shift adjustment, we prioritize high-priority jobs to align with delivery times. We conducted experiments to validate our model and algorithm by comparing them against four well-known intelligent optimization algorithms. Our improved algorithm proves to be highly beneficial in job and worker scheduling as it effectively minimizes overdue days and arranges worker overtime.

Multi-objective elitist spotted hyena resource optimized flexible job shop scheduling

The job shop scheduling problem (JSSP) has drained a lot of consideration since it is one of the most important optimization problems in the manufacturing domain. The scheduling method is crucial for optimizing the objective of minimizing makespan among thousands of jobs, but evaluating machine capacity for achieving this goal remains challenging despite the development of various population-based optimization algorithms for job shop scheduling problems. To improve the efficiency of Job shop scheduling, a novel Multi-objective Elitist Spotted Hyena Monotonic Scheduling (MESHS) technique is introduced. The proposed MESHS technique includes two major processes: machine selection and operation sequences. The number of jobs is considered for solving the scheduling problem. First, the machine selection is performed by applying the Multi-objective Elitist Spotted Hyena optimization technique. The optimization technique selects the optimal machines parallelly based on multiple objective functions such as energy consumption, CPU utilization, and job completion time. The fitness of every machine is calculated based on these multiple objective functions using Levenberg–Marquardt method. Then the Elitist strategy is applied to select the optimal machine based on fitness. After the machine selection, the rate-monotonic preemptive scheduling is modeled to provide a robust operation sequence by assigning high-priority jobs to the optimal machines. As a result, efficient job scheduling is achieved with minimum time. Finally, the experimental valuation is carried out using a benchmark OR-Library dataset with different factors such as job shop scheduling efficiency, job scheduling time, makespan, and memory consumption concerning a number of jobs.

PQ-Mist: Priority Queueing-Assisted Mist–Cloud–Fog System for Geospatial Web Services

The IoT and cloud environment renders enormous quantities of geospatial information. Fog and mist computing is the scaling technology that handles geospatial data and sends it to the cloud storage system through fog/mist nodes. Installing a mist–cloud–fog system reduces latency and throughput. This mist–cloud–fog system has processed different types of geospatial web services, i.e., web coverage service (WCS), web processing services (WPS), web feature services (WFS), and web map services (WMS). There is an urgent requirement to increase the number of computer devices tailored to deliver high-priority jobs for processing these geospatial web services. This paper proposes a priority-queueing assisted mist–cloud–fog system for efficient resource allocation for high- and low-priority tasks. In this study, WFS is treated as high-priority service, whereas WMS is treated as low-priority service. This system dynamically allocates mist nodes and is determined by the load on the system. In addition to that, the assignment of tasks is determined by priority. Not only does this classify high-priority tasks and low-priority tasks, which helps reduce the amount of delay experienced by high-priority jobs, but it also dynamically allocates mist devices within the network depending on the computation load, which helps reduce the amount of power that is consumed by the network. The findings indicate that the proposed system can achieve a significantly lower delay for higher-priority jobs for more significant rates of task arrival when compared with other related schemes. In addition to this, it offers a technique that is both mathematical and analytical for investigating and assessing the performance of the proposed system. The QoS requirements for each device demand are factored into calculating the number of mist nodes deployed to satisfy those requirements.

A combined priority scheduling method for distributed machine learning

Algorithms and frameworks for distributed machine learning have been widely used in numerous artificial intelligence engineering applications. A cloud platform provides a large number of resources at a lower cost and is a more convenient method for such applications. With the rapid development of containerization, native cloud combinations based on Docker and Kubernetes have provided effective resource support for distributed machine learning. However, native Kubernetes does not provide efficient priority or fair resource scheduling strategies for distributed machine learning in computationally intensive and time-consuming jobs, which easily leads to resource deadlock, resource waste, and low job execution efficiency. Therefore, to utilize the execution order between multiple jobs in distributed machine learning as well as the dependencies between multiple tasks for the same job, considering intra- and inter-group scheduling priorities, a combined priority scheduling method is proposed for distributed machine learning based on Kubernetes and Volcano. Considering the user priority, task priority, longest wait time, task parallelism, and affinity and non-affinity between the parameter server and worker nodes, a combined priority scheduling model of inter- and intra-job priority is proposed, which is mapped into a scheduling strategy of inter- and intra-group priorities of pods, enabling the efficient scheduling and training of distributed machine learning. The experiment results show that the proposed method achieves preferential resource allocation for urgent, high parallelism, and high-priority jobs with high-priority users and improves the job execution efficiency. The affinity and anti-affinity settings among pods reduce the time of information interaction between the parameter server and worker nodes to a certain extent, thereby improving the job completion efficiency. This group scheduling strategy alleviates the problems of resource deadlock and waste caused by insufficient resources in cloud computing.

Ergodicity of Two Class Priority Queues with Preemptive Priority

Well-known results on the ergodicity of queues with preemptive priority were obtained under the assumption that jobs arrive according to the Poisson process. This assumption, however, does not always hold true in practice. In our work we find sufficient ergodicity conditions for queues with two priority classes with a single server, where interarrival times of high-priority jobs have either Erlang or hyperexponential distribution and interarrival times of low-priority jobs and service times of jobs of both classes have arbitrary continuous distributions. We present results in disciplines with loss of an interrupted job and with retrial of an interrupted job.

A Theory of Auto-Scaling for Resource Reservation in Cloud Services

We consider a distributed server system consisting of a large number of servers, each with limited capacity on multiple resources (CPU, memory, disk, etc.). Jobs with different rewards arrive over time and require certain amounts of resources for the duration of their service. When a job arrives, the system must decide whether to admit it or reject it, and if admitted, in which server to schedule the job. The objective is to maximize the expected total reward received by the system. This problem is motivated by control of cloud computing clusters, in which, jobs are requests for Virtual Machines or Containers that reserve resources for various services, and rewards represent service priority of requests or price paid per time unit of service by clients. We study this problem in an asymptotic regime where the number of servers and jobs' arrival rates scale by a factor L, as L becomes large. We propose a resource reservation policy that asymptotically achieves at least 1/2, and under certain monotone property on jobs' rewards and resources, at least 11/4 of the optimal expected reward. The policy automatically scales the number of VM slots for each job type as the demand changes, and decides in which servers the slots should be created in advance, without the knowledge of traffic rates. It effectively tracks a low-complexity greedy packing of existing jobs in the system while maintaining only a small number, g(L) = w(logL), of reserved VM slots for high priority jobs that pack well.

A competency-driven staff assignment approach to improving employee scheduling robustness

Presented paper concerns the competency-driven staff assignment and scheduling approach to the management of project portfolios subject to perturbations caused by employee absences and/or unexpected arrival of high priority jobs. Proactive strategy is considered, which exploits the concept of employee substitutability to improve the robustness of personnel allocation in the case of occurrence of specific types of disruptions. Solutions obtained using the model of a constraint satisfaction problem developed in this study are validated in series quantitative and qualitative experiments. With a view to future implementation in a Decision Support Systems dedicated to prototyping of proactive personnel allocation, a methodology employing the concept of a competency framework-based robustness measure is proposed. Implemented in a declarative framework, the proposed approach allows one to find a redundant competency framework robust to a given set of disruptions.

Coordinative Scheduling of Computation and Communication in Data-parallel Systems

For many data-parallel computing systems like Spark, a job usually consists of multiple computation stages and inter-stage communication (i.e., coflows). Many efforts have been done to schedule coflows and jobs independently. The simple combination of coflow scheduling and job scheduling, however, would prolong the average job completion time (JCT) due to the conflict. For this reason, we propose a new abstraction of scheduling unit, named coBranch, which takes the dependency between computation stages and coflows into consideration, to schedule coflows and jobs jointly. Besides, mainstream coflow schedulers are order-preserving, i.e., all coflows of a high-priority job are prioritized than those of a low-priority job. We observe that the order-preserving constraint incurs low inter-job parallelism. To overcome the problem, we employ an urgency-based mechanism to schedule coBranches, which aims to decrease the average JCT by enhancing the inter-job parallelism. We implement the urgency-based coBranch Scheduling (BS) method on Apache Spark, conduct prototype-based experiments, and evaluate the performance of our method against the shortest-job-first critical-path method and the FIFO method. Results show that our method achieves around 10 and 15 percent reduction in the average JCT, respectively. Large-scale simulations based on the Google trace show that our method performs better and reduces JCT by 23 and 35 percent, respectively.

The English Auction Method for Scheduling Jobs in a Distributed Network of Supercomputer Centers

The article describes the problem of scheduling jobs with absolute priorities in a geographically distributed network of supercomputer centers (GDN). In this case English auction method can be efficiently applied. Classic market model considers computational resources as the goods (subject of auction trades), and resources’ owners act as sellers. Users act as buyers who participate in the auction on purpose to purchase computing resources for the execution of their jobs. This model assumes that customers have certain budgets in nominal or real money. The priority of the job is actually determined by the price, which the user can pay to finish the job by a certain time. The GDN model investigated by the authors differs from the known ones in that the jobs priorities are absolute and assigned according to uniform rules. The main goal is the earliest execution of high-priority jobs. In this case, the concept of the user’s budget becomes meaningless, and the classic auction models do not work. The authors propose a new approach where the jobs act as the goods and buyers are resource owners who paying for jobs with available idle supercomputing resources. For this approach, the authors investigate the features and characteristics of English auction, as the most preferred method for scheduling jobs with absolute priorities in GDN.

Cuckoo-inspired Job Scheduling Algorithm for Cloud Computing

Cloud computing offers a pool of virtual resources to users on demand. Among the main challenge facing cloud computing is job scheduling which is an NP-hard problem. Usually, scheduling should consider job priority, where high-priority jobs should be immediately allocated required resources, whereas low-priority jobs can wait. In this paper, we propose a scheduling algorithm that imitates the parasitic behaviour of the cuckoo bird. Cuckoos reproduce by exploiting the nests of other birds with eggs similar to their own. We have evaluated the proposed cuckoo scheduling algorithm under varying number of nodes and percentage of high priority jobs. The experimental results demonstrate the superiority of our proposed algorithm over the benchmarks in terms of average CPU utilization and average turnaround time for each type of job.

Towards robustness of response times: minimising the maximum inter-completion time on parallel machines

Motivated by scheduling practices that require a response to unplanned high-priority jobs as soon as possible without preempting any in-processing jobs, this paper considers a deterministic identical parallel machine scheduling problem to achieve robustness with regard to a worst-case response time. To the best of our knowledge, this paper is the first to study the objective of minimising the maximum inter-completion time, i.e. the maximum time difference between any two consecutive completion times of jobs. For this novel scheduling problem, we first show its NP-hardness, and then propose an integer linear programming formulation and three heuristic approaches. Numerical experiments demonstrate the efficiency of our solution methods.

Process quality improvement through collaboration in synchronized individual production companies

The synchronized individual production combines a high level of standardization with the necessary flexibility to manufacture customized products. However, it is not able to properly react to disturbances, like high priority jobs. To address this and other challenges, the company employees should be more actively involved into decision making processes, especially when it comes to production planning and control. Therefore, this paper presents a wiki-platform based approach to match the expert knowledge of the company's employees with the daily challenges in the production and an evaluation scheme to measure the process quality improvements.

Модели, методы и алгоритмы планирования в грид и облачных вычислениях

Certain intricate problems, e.g., processing the results of physical experiments in the large hadron collider at the CERN, utilize significant distributed computing resources, part of which is shared with their owners. This factor, even within the framework of virtual organizations, causes competition for utilization of resources between both independent users and global (users’) and local job flows of computing resource owners. This circumstance adds much difficulty to the problem of providing the required quality of service in scalable computing. The presently known scheduling algorithms, their combinations and heuristics do not offer tools for producing efficient schedules under the conditions of heterogeneous distributed environments with a dynamically changing composition of computing resources. Under such conditions, so called economic models for allocating non-dedicated resources and for scheduling distributed computing turn out to be highly efficient in fields like grid and cloud computing, and multiagent systems. The article analyzes the current state of investigations in the field of methods and algorithms for efficiently scheduling competing flows of structured and parallel jobs in distributed heterogeneous computing environments, namely, in grid infrastructures and in cloud services. Primary attention is paid to methods and tools for scheduling intricate jobs and their flows with due regard to the preferences of stakeholders (users, resource owners, and administrators), and specific features of applications. With these factors duly taken into account, more efficient utilization of the resources in distributed computing environments may be achieved. The article presents an approach for comprehensively solving the problem of scheduling intricate jobs and their flows. Execution of a user job involves the need to allocate a set of simultaneously accessible computing resources (slot windows). The problem of selecting suitable resources is complicated by such factors as the availability of local high-priority jobs, the dynamics of resources utilization, and advanced reservations. The computing node load schedules are kept in line with the actual demands by means of a scheduling horizon. Online rescheduling implies the need to minimize the computational complexity of the relevant algorithms. Competition among the multitude of different job flows is a factor that degrades the level of resource accessibility and decreases the number of alternative scheduling options. One possible approach for solving this problem is to use anticipation scheduling, a method that makes it possible to increase the number of possible alternative schedules by means of special slot selection mechanisms. The anticipation scheduling procedure opens the possibility to evaluate the efficiency of different resource combinations. The obtained data serve as a basis for producing a feasible job flow execution schedule close to the optimal solution.

TWO-CLASS ROUTING WITH ADMISSION CONTROL AND STRICT PRIORITIES

We consider the problem of routing and admission control in a loss system featuring two classes of arriving jobs (high-priority and low-priority jobs) and two types of servers, in which decision-making for high-priority jobs is forced, and rewards influence the desirability of each of the four possible routing decisions. We seek a policy that maximizes expected long-run reward, under both the discounted reward and long-run average reward criteria, and formulate the problem as a Markov decision process. When the reward structure favors high-priority jobs, we demonstrate that there exists an optimal monotone switching curve policy with slope of at least −1. When the reward structure favors low-priority jobs, we demonstrate that the value function, in general, lacks structure, which complicates the search for structure in optimal policies. However, we identify conditions under which optimal policies can be characterized in greater detail. We also examine the performance of heuristic policies in a brief numerical study.

A SURVEY ON MACHINE LEARNING APPROACH TO MAINFRAME ANALYSIS

Mainframe system processing includes a “Batch Cycle” that approximately spans in regular interval on a daily basis. The core part of the cycle completes in the middle of the regular interval with key client deliverables associated with the end times of certain jobs are tracked by service delivery. There are single and multi-client batch streams, a QA stream which includes all clients, and about huge batch jobs per day that execute. Despite a sophisticated job scheduling software and automated system workload management, operator intervention is required. The outcome of our proposed work is to bring out the high priority job first. According to our method, the jobs are re-prioritized the schedules so that prioritized jobs can get theavailable system resources. Furthermore, the characterization, analysis, and visualization of the reasons for a manual change in the schedule are to be considered. This work requires extensive data preprocessing and building machine learning models for the causal relationship between various system variables and the time of manual changes.

Flexible job shop scheduling problem for parallel batch processing machine with compatible job families

Flexible Job-Shop Scheduling Problem (FJSP) with Parallel Batch processing Machine (PBM) is studied. First, a Mixed Integer Programming (MIP) formulation is proposed for the first time. In order to address an NP-hard structure of this problem, the formulation is modified to selectively schedule jobs. Although there are many jobs on a given floor, semiconductor manufacturing is most challenged by priority jobs that promise a significant amount of financial compensation in exchange for an expedited delivery. This modification could leave some non-priority jobs unscheduled. However, it vastly expedites the discovery of improving solutions by first branching on integer variables with higher priority jobs. This study then turns job-dependent processing times into job-independent ones by assuming a machine has an equal processing time on different jobs. This assumption is roughly true or acceptable for the sake of the reduced computational time in the industry. These changes significantly reduce computational time compared to the original model when tested on a set of common problem instances from the literature. Computational results show that this proposed model can generate an effective schedule for large problems. Author encourages other researchers to propose an improved MIP model.

Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints

With the popularization and development of cloud computing, lots of scientific computing applications are conducted in cloud environments. However, current application scenario of scientific computing is also becoming increasingly dynamic and complicated, such as unpredictable submission times of jobs, different priorities of jobs, deadlines and budget constraints of executing jobs. Thus, how to perform scientific computing efficiently in cloud has become an urgent problem. To address this problem, we design an elastic resource provisioning and task scheduling mechanism to perform scientific workflow jobs in cloud. The goal of this mechanism is to complete as many high-priority workflow jobs as possible under budget and deadline constraints. This mechanism consists of four steps: job preprocessing, job admission control, elastic resource provisioning and task scheduling. We perform the evaluation with four kinds of real scientific workflow jobs under different budget constraints. We also consider the uncertainties of task runtime estimations, provisioning delays, and failures in evaluation. The results show that in most cases our mechanism achieves a better performance than other mechanisms. In addition, the uncertainties of task runtime estimations, VM provisioning delays, and task failures do not have major impact on the mechanism's performance.

Distributed Hybrid Fault Tolerant Model with Fault Recovery for Mobile Grid

This paper proposes a distributed hybrid fault tolerant model for mobile grid systems. It consists of proactive and reactive fault tolerant approaches. Before resource allocation, all the jobs are sorted and assigned to different level based on precedence. The reliability and failure probability of each resource is estimated. In case of proactive approach, the high priority jobs are assigned to the resources with high reliabilities. In reactive approach, a backup node with low failure probability will have the replica of jobs assigned for the resources with high failure probability. If a failure is detected in the primary resource, the backup resource will perform failure recovery by executing the remaining job.

M/M/c Queue with Two Priority Classes

This paper provides the first exact analysis of a preemptive M/M/c queue with two priority classes having different service rates. To perform our analysis, we introduce a new technique to reduce the two-dimensionally infinite Markov chain (MC), representing the two class state space, into a one-dimensionally infinite MC, from which the generating function (GF) of the number of low-priority jobs can be derived in closed form. (The high-priority jobs form a simple M/M/c system and are thus easy to solve.) We demonstrate our methodology for the c = 1, 2 cases; when c > 2, the closed-form expression of the GF becomes cumbersome. We thus develop an exact algorithm to calculate the moments of the number of low-priority jobs for any c ≥ 2. Numerical examples demonstrate the accuracy of our algorithm and generate insights on (i) the relative effect of improving the service rate of either priority class on the mean sojourn time of low-priority jobs; (ii) the performance of a system having many slow servers compared with one having fewer fast servers; and (iii) the validity of the square root staffing rule in maintaining a fixed service level for the low-priority class. Finally, we demonstrate the potential of our methodology to solve other problems using the M/M/c queue with two priority classes, where the high-priority class is completely impatient.

Dynamic Resource Allocation Scheme in Cloud Computing

Cloud Computing environment provisions the supply of computing resources on the basis of demand, as and when needed. It builds upon advances of virtualisation and distributed computing to support cost efficient usage of computing resources, emphasizing on resource scalability and on-demand services. It allows business outcomes to scale up and down their resources based on needs. Managing the customer demand creates the challenges of on-demand resource allocation. Virtual Machine (VM) technology has been employed for resource provisioning. It is expected that using virtualized environment will reduce the average job response time as well as executes the task according to the availability of resources. Hence VMs are allocated to the user based on characteristics of the job. Effective and dynamic utilization of the resources in cloud can help to balance the load and avoid situations like slow run of systems. This paper mainly focuses on allocation of VM to the user, based on analyzing the characteristics of the job. Main principle of this work is that low priority jobs (deadline of the job is high) should not delay the execution of high priority jobs (deadline of the job is low) and to dynamically allocate VM resources for a user job within deadline