Batch Jobs Research Articles

Novel mathematical formulations for parallel-batching processing machine scheduling problems

We study mathematical formulations for batch-processing machine scheduling problems (BPMPs), which are the challenging issues in the machine scheduling literature where machines are capable of processing a batch of jobs simultaneously if jobs with non-identical sizes can be packed in a capacitated machine. In this paper, we tackle single- and parallel-machine BPMPs, and other interesting problem variants that aim at minimizing the makespan. We develop novel formulations along with valid inequalities and an algorithm framework that makes use of dual information and bounding techniques to achieve efficiency when instances are intractable. Extensive computational experiments on benchmark instances show that our approaches achieve state-of-the-art results and prove the optimality of intractable instances in the literature.

A double-Q network collaborative multi-objective optimization algorithm for precast scheduling with curing constraints

Precast scheduling is a special case of optimization problem, in which the curing stage requires parallel processing of a batch of jobs. In this study, distributed flexible job shop scheduling with crane transportation, fuzzy processing, and curing constraints is modeled for precast scheduling. Three objectives are considered simultaneously, namely, fuzzy makespan, energy consumption, and economic cost. To solve this complex problem, a bi-level collaborative multi-objective optimization evolution algorithm (BCMOEA) is developed. First, a double-Q network is designed considering different groups of features to generate valuable solutions for the second-level components. Then, a dynamic-adjusted reference point set is embedded to divide the current population into three sub-populations. Furthermore, a collaboration mechanism is developed to learn different knowledge from these sub-populations to balance the convergence and diversity abilities. Moreover, a knowledge driven curing batching heuristic is designed to improve the fuzzy makespan. Finally, a set of instances generated based on the realistic precast process is tested, and detailed comparisons with the state-of-the-art algorithms show the competitive performances of the proposed algorithm.

Actimize: A Batch Job Real Time Dashboard

Our Compliance partners do not have clear visibility into real time status of day to day surveillance and supervision alert batch jobs. They rely on manual communication to get notified in case of job delays or failures. To avoid any regulatory implications,it is crucial that alerts are generated on time so that they can act on a timely basis to prevent SLA breaches that may lead to further delays. So,it is important that they remain updated on the daily alerting job runs so that they can plan their work according. Index Terms—component, formatting, style, styling, insert

Batch Job Active-Active Resiliency System and Method

This manuscript presents a robust solution for orchestrating batch job workflows in an active-active resiliency mode across multiple regions. The proposed system abstracts the complexities of job and step/task state management, provides automated and customizable retries, and ensures auditability and observability. The system maintains operational continuity and fault tolerance, leveraging lock marker files and a configurable scheduler, even during regional failures. This approach addresses the challenges of asynchronous workflow management, offering enhanced visibility, control, and efficiency in processing batch jobs.

Asynchronous modeling workflows in CyberWater with on-demand HPC/Cloud access

Workflow management systems (WfMSs) are commonly used to organize/automate sequences of tasks as workflows to accelerate scientific discoveries. During complex workflow modeling, a local interactive workflow environment is desirable, as users usually rely on their rich, local environments for fast prototyping and refinements before they consider using more powerful computing resources. However, existing WfMSs do not simultaneously support local interactive workflow environments and HPC resources. In this paper, we present a mechanism for on-demand access to remote HPC resources from desktop/laptop-based workflow management software to compose, monitor, and analyze scientific workflows in the CyberWater project. CyberWater is an open-data and open-modeling software framework for environmental and water communities. In this work, we extend the open-model, open-data design of CyberWater with on-demand HPC accessing capacity. In particular, we design and implement the LaunchAgent and JobManager, which can be integrated into a local desktop/Laptop environment to allow on-demand usage of remote HPC resources for computational modeling workflows effectively and efficiently. LaunchAgent manages user authentication to remote resources, prepares computation-intensive or data-intensive tasks as batch jobs, submits jobs to remote resources, and monitors quality of services for users. LaunchAgent interacts seamlessly with other components in CyberWater, providing advantages of user-friendly feature-rich desktop software experience and increased computing power through on-demand HPC/Cloud usage. In our evaluations, we demonstrate how a hydrological modeling workflow that consists of both local and remote tasks can be constructed and show that our new on-demand HPC/Cloud usage helps speed up hydrology workflows while allowing asynchronous HPC/Cloud access from workflows using a desktop graphical user interface.

On the Performance of Malleable APGAS Programs and Batch Job Schedulers

Malleability—the ability for applications to dynamically adjust their resource allocations at runtime—presents great potential to enhance the efficiency and resource utilization of modern supercomputers. However, applications are rarely capable of growing and shrinking their number of nodes at runtime, and batch job schedulers provide only rudimentary support for such features. While numerous approaches have been proposed to enable application malleability, these typically focus on iterative computations and require complex code modifications. This amplifies the challenges for programmers, who already wrestle with the complexity of traditional MPI inter-node programming. Asynchronous Many-Task (AMT) programming presents a promising alternative. In AMT, computations are split into many fine-grained tasks, which are processed by workers. This makes transparent task relocation via the AMT runtime system possible, thus offering great potential for enabling efficient malleability. In this work, we propose an extension to an existing AMT system, namely APGAS for Java. We provide easy-to-use malleability programming abstractions, requiring only minor application code additions from programmers. Runtime adjustments, such as process initialization and termination, are automatically managed by our malleability extension. We validate our malleability extension by adapting a load balancing library handling multiple benchmarks. We show that both shrinking and growing operations cost low execution time overhead. In addition, we demonstrate compatibility with potential batch job schedulers by developing a prototype batch job scheduler that supports malleable jobs. Through extensive real-world job batches execution on up to 32 nodes, involving rigid, moldable, and malleable programs, we evaluate the impact of deploying malleable APGAS applications on supercomputers. Exploiting scheduling algorithms, such as FCFS, Backfilling, Easy-Backfilling, and one exploiting malleable jobs, the experimental results highlight a significant improvement regarding several metrics for malleable jobs. We show a 13.09% makespan reduction (the time needed to schedule and execute all jobs), a 19.86% increase in node utilization, and a 3.61% decrease in job turnaround time (the time a job takes from its submission to completion) when using 100% malleable job in combination with our prototype batch job scheduler compared to the best-performing scheduling algorithm with 100% rigid jobs.

Component-distinguishable Co-location and Resource Reclamation for High-throughput Computing

Cloud service providers improve resource utilization by co-locating latency-critical (LC) workloads with best-effort batch (BE) jobs in datacenters. However, they usually treat multi-component LCs as monolithic applications and treat BEs as “second-class citizens” when allocating resources to them. Neglecting the inconsistent interference tolerance abilities of LC components and the inconsistent preemption loss of BE workloads can result in missed co-location opportunities for higher throughput. We present Rhythm , a co-location controller that deploys workloads and reclaims resources rhythmically for maximizing the system throughput while guaranteeing LC service’s tail latency requirement. The key idea is to differentiate the BE throughput launched with each LC component, that is, components with higher interference tolerance can be deployed together with more BE jobs. It also assigns different reclamation priority values to BEs by evaluating their preemption losses into a multi-level reclamation queue. We implement and evaluate Rhythm using workloads in the form of containerized processes and microservices. Experimental results show that it can improve the system throughput by 47.3%, CPU utilization by 38.6%, and memory bandwidth utilization by 45.4% while guaranteeing the tail latency requirement.

An improved deep Q-learning algorithm for a trade-off between energy consumption and productivity in batch scheduling

The single-batch machine, commonly found in industrial manufacturing, can concurrently process a group of jobs in variable-speed batches, leading to fluctuating levels of both energy consumption and processing time. Identifying an optimal balance between total energy consumption and makespan presents a challenge due to the intricate nature of their relationship in real-world scenarios. This study introduces a mixed-integer programming model designed to determine optimal machine operating states for diverse workloads, meeting customer requirements while concurrently achieving energy savings for sustainability. An energy-aware batch scheduling deep Q-learning network (EBSDQN) framework has been created, encompassing sequencing policies, batching rules, and speed adjustment policies. This framework is explicitly crafted to tackle the NP-hard problem, a challenge that commonly perplexes commercial solvers like Gurobi when seeking optimal solutions within a 360-second time frame for small-scale instances. The EBSDQN design is fortified with efficient action decoding policies, refined reward assessments, exploitative neighborhood rules, and a streamlined buffer training process. This approach not only conserves training time but also adapts dynamically in accordance with the principles of the Markov Decision Process (MDP) and deep neural network. The developed algorithm exhibits impressive robustness, reaching convergence after 600 episodes of training. In separate comparisons with a commercial solver, two single-objective algorithms, and two multi-objective algorithms, our algorithm consistently demonstrates superior overall performance across the same instances. In the worst-case analysis, further examination delves into the influences of job features. In summary, this study underscores the high sensitivity of both TEC and Cmax to job processing time distribution, suggesting the prioritization of TEC over Cmax in optimization for sustainable planning and operations. Furthermore, this research has the potential to accelerate the integration of artificial intelligence into the manufacturing sector.

Batch Scheduling for Work Centers with Multiple Manufacturing Machines under the Parallel-Sequence-transfer Mode

This paper, based on the parallel-sequence-transfer mode of batch jobs, aimed to implement research on the batch scheduling for work centers with multiple manufacturing machines in full detail. In order to arrange a precise product planning with batch jobs, obtain the maximum completing time with batch jobs, schedule manufacturing machines in each work center accurately, and get the actual number of manufacturing machines used to processing batch jobs in each work center, a new processing procedure of the processing time model under the parallel-sequence-transfer mode was designed. The new designed processing procedure was tested on a numeric example. The results show that the research in this thesis could not only arrange a precise production planning under the parallel-sequence-transfer mode, but also schedule manufacturing machines for each work center with multiple manufacturing machines accurately.

CPU Performance Study for HEP Workloads with Respect to the Number of Single-core Slots

CPU Performance Study for HEP Workloads with Respect to the Number of Single-core Slots

A Metaheuristic Framework for Energy-Intensive Industries With Batch Processing Machines

Batch processing machines, which operate multiple jobs at a time, are commonly used in energy-intensive industries. A significant amount of energy can be saved in such industries using production scheduling as an approach to enhance efficiency. This study deals with an energy-aware scheduling problem for parallel batch processing machines with incompatible families and job release times. In such an environment, a machine may need to wait until all the jobs in the next batch become ready. During waiting time, a machine can be switched <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">off</small> or kept <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">on</small> standby for more energy-efficient scheduling. We first present a mixed-integer linear programming (MILP) model to solve the problem. However, the presented MILP model can only solve small problem instances. We therefore propose an energy-efficient tabu search (ETS) algorithm for solving larger problem instances. The proposed solution framework incorporates multiple neighborhood methods for efficient exploration of the search space. An energy-related heuristic is also integrated into the ETS for minimizing energy consumption during the waiting time. The performance of our proposed ETS algorithm is validated by comparing it with CPLEX for small problem instances and with two other heuristic algorithms for larger problem instances. The contribution of different components in ETS is also established in our experimental studies. The proposed solution framework is expected to bring many benefits in energy-intensive industries both economically and environmentally.

A grid site reimagined: Building a fully cloud-native ATLAS Tier 2 on Kubernetes

The University of Victoria (UVic) operates an Infrastructure-asa-Service scientific cloud for Canadian researchers, and a Tier 2 site for the ATLAS experiment at CERN as part of the Worldwide LHC Computing Grid (WLCG). At first, these were two distinctly separate systems, but over time we have taken steps to migrate the Tier 2 grid services to the cloud. This process has been significantly facilitated by basing our approach on Kubernetes, a versatile, robust, and very widely adopted automation platform for orchestrating containerized applications. Previous work exploited the batch capabilities of Kubernetes to run grid computing jobs and replace the conventional grid computing elements by interfacing with the Harvester workload management system of the ATLAS experiment. However, the required functionality of a Tier 2 site encompasses more than just batch computing. Likewise, the capabilities of Kubernetes extend far beyond running batch jobs, and include for example scheduling recurring tasks and hosting long-running externally-accessible services in a resilient way. We are now undertaking the more complex and challenging endeavour of adapting and migrating all remaining services of the Tier 2 site — such as APEL accounting and Squid caching proxies, and in particular the grid storage element — to cloud-native deployments on Kubernetes. We aim to enable fully comprehensive deployment of a complete ATLAS Tier 2 site on a Kubernetes cluster via Helm charts, which will benefit the community by providing a streamlined and replicable way to install and configure an ATLAS site. We also describe our experience running a high-performance self-managed Kubernetes ATLAS Tier 2 cluster at the scale of 8 000 CPU cores for the last two years, and compare with the conventional setup of grid services.

A web workbench system for the Slurm cluster at IHEP

Slurm REST APIs are released since version 20.02. With those REST APIs one can interact with slurmctld and slurmdbd daemons in a REST- ful way. As a result, job submission and cluster status query can be achieved with a web system. To take advantage of Slurm REST APIs, a web workbench system is developed for the Slurm cluster at IHEP. The workbench system con- sists with four subsystems including dashboard, tomato, jasmine and cosmos. The dashboard subsystem is used to display cluster status including nodes and jobs. The tomato subsystem is developed to submit special HTCondor glidein jobs in the Slurm cluster. The jasmine system is used to generate and submit batch jobs based on workload parameters. The cosmos subsystem is an ac- counting system, which not only generates statistical charts but also provides REST APIs to query jobs. This paper presents design and implementation de- tails of the Slurm workbench. With the help of workbench, administrators and researchers can get their work done in an effective way.

Uniformity of Batch Job Recommendations and Real-time Recommendation

Collaborative filtering recommends items based on similarity measures between users and/or items. The basic assumption behind the algorithm is that users with similar interests have common preferences. There are a lot of applications where websites collect data from their users and use that data to predict the likes and dislikes of their users. This allows them to recommend the content that they like. Recommender systems are a way of suggesting similar items and ideas to a user’s specific way of thinking

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

The unprecedented prosperity of the Industrial Internet of Things (IIoT) promotes the traditional industry transforming into intelligent manufacturing so that the whole production process can be comprehensively controlled to achieve flexible production. Intelligent scheduling, as one of the key enabling techniques, is desired to allocate the production of several machines by an efficient solution with minimum makespan. Existing approaches adopt a fixed search paradigm based on expert knowledge to seek satisfactory solutions. However, considering the varying data distribution and large-sized of the practical problems, these methods fail to guarantee the quality of the obtained solution under the real-time requirement. To address this challenge, we formulate the production scheduling problem as a Markov decision process (MDP) and specifically design a job scheduling model made up of a job batching module for the hybrid flow-shop scheduling problem on batch processing machines (HFSP-BPM). Our proposed model consists of an actor network that learns the action under different conditions and a critic network that evaluates the action of the actor. We analyze the convergence of the model under different parameter settings to determine the optimal parameter. Extensive numerical experiments on both publicly available dataset and real steel plant production dataset demonstrate that the proposed deep reinforcement learning (DRL) approach compared with other baselines, more than 6% average improvements can be observed in many instances.

Configuration-Driven No-Code Data Ingestion Engine with RuleBased Processing

The efficient management of batch processes is critical for modern data-driven organizations, especially within the context of complex data ingestion and processing workflows. This abstract explores the essential components and functionalities required to streamline batch job execution within such environments. Firstly, a robust batch management system is essential, providing capabilities for job scheduling, dependency management, and error handling. Such a system orchestrates the execution of batch jobs, ensuring they run in the appropriate sequence and handle dependencies effectively. Additionally, real-time event-driven triggers enable immediate batch job initiation upon file drops, ensuring timely processing of incoming data and swift response to business needs. Conversely, scheduled batch executions provide predictability and automation, allowing organizations to optimize resource utilization and maintain operational efficiency by running batches at predetermined intervals or specific times. Furthermore, the abstract delves into the necessity of a comprehensive configuration setup to support various data formats, transformation requirements, and validation criteria. A rich configuration engine allows for seamless mapping, transformation, and validation of incoming data, enabling flexibility and adaptability to diverse data sources and requirements. It also emphasizes the importance of containerization for batch job execution, ensuring scalability, resource isolation, and efficient utilization of computing resources. Lastly, the provision of job IDs facilitates seamless invocation of batch jobs by external systems, enabling streamlined interaction and integration across different platforms. Together, these components form a cohesive framework for efficient batch job management, empowering organizations to optimize data processing workflows and derive actionable insights from their data assets.

Accelerating Machine Learning Inference with GPUs in ProtoDUNE Data Processing

We study the performance of a cloud-based GPU-accelerated inference server to speed up event reconstruction in neutrino data batch jobs. Using detector data from the ProtoDUNE experiment and employing the standard DUNE grid job submission tools, we attempt to reprocess the data by running several thousand concurrent grid jobs, a rate we expect to be typical of current and future neutrino physics experiments. We process most of the dataset with the GPU version of our processing algorithm and the remainder with the CPU version for timing comparisons. We find that a 100-GPU cloud-based server is able to easily meet the processing demand, and that using the GPU version of the event processing algorithm is two times faster than processing these data with the CPU version when comparing to the newest CPUs in our sample. The amount of data transferred to the inference server during the GPU runs can overwhelm even the highest-bandwidth network switches, however, unless care is taken to observe network facility limits or otherwise distribute the jobs to multiple sites. We discuss the lessons learned from this processing campaign and several avenues for future improvements.

Managing asynchronous workloads in a multi‐tenant microservice enterprise environment

AbstractA multi‐tenant microservice architecture involving components with asynchronous interactions and batch jobs requires efficient strategies for managing asynchronous workloads. This article addresses this issue in the context of a leading company developing tax software solutions for many national and multi‐national corporations in Brazil. A critical process provided by the company's cloud‐based solutions encompasses tax integration, which includes coordinating complex tax calculation tasks and needs to be supported by asynchronous operations using a message broker to guarantee order correctness. We explored and implemented two approaches for managing asynchronous workloads related to tax integration within a multi‐tenant microservice architecture in the company's context: (i) a polling‐based approach that employs a queue as a distributed lock (DL) and (ii) a push‐based approach named single active consumer (SAC) that relies on the message broker's logic to deliver messages. These approaches aim to achieve efficient resource allocation when dealing with a growing number of container replicas and tenants. In this article, we evaluate the correctness and performance of the DL and SAC approaches to shed light on how asynchronous workloads impact the management of multi‐tenant microservice architectures from delivery and deployment perspectives.

Just-in-Time Precast Production Scheduling Using Dominance Rule-Based Genetic Algorithm.

The objective of this study is to provide a model and method to assist fabricators in making appropriate precast production plans, coordinating factories prefabrication and on-site assembly in construction based on the just-in-time (JIT) philosophy. We propose a JIT precast production scheduling model for the precast production of steel box girders in the Hong Kong-Zhuhai-Macau (HZM) bridge construction project. In order to minimize the total early/tardy costs for this JIT scheduling model, we first present a job batching algorithm (JBA) to group series of jobs into several batches. Then, we model the batch cost function as a piecewise linear convex function and derive its optimal solution, which guides us to propose an optimal shifting algorithm (OSA) to locate the optimal starting time of each batch and minimize the batch cost. In order to get the best job sequence for minimizing the total early/tardy costs and improve search efficiency, we introduce a dominance rule for early/tardy scheduling problem and propose a dominance rule-based genetic algorithm (DBGA) embedded with an optimal timing algorithm, which can find the best job sequence as well as the corresponding optimal schedule. The real-world case study of HZM bridge project demonstrates that our proposed model and algorithm can assist construction practitioners to make better decision on precast production scheduling compared to empirical rule in current engineering practice. In addition, numerical studies prove that our proposed algorithm has a better performance on both effectiveness and efficiency due to the dominance rule.

A deep reinforcement learning-based optimization method for long-running applications container deployment

Unlike the short execution cycles of batch jobs, intelligent algorithmic applications typically run in long-cycle containers in the cloud(Long-Running Applications, LRA). Both need to meet strict SLO (service level objective) requirements, consider performance scaling to cope with peak load demands, and face issues such as I/O dependencies and resource contention and interference from coexisting containers. The above greatly complicates container deployment and can easily lead to performance bottlenecks. Therefore, the optimization of LRA-like container deployment is one of the key issues that cannot be avoided and needs to be addressed in the cloud computing model. This research uses deep reinforcement learning (DRL) to optimize the deployment of LRAs class containers. The proposed non-generic model is able to customize a dedicated model for each container group, providing high-quality placement and low training complexity; meanwhile, the proposed batch deployment scheme is able to optimize various scheduling objectives that are not directly supported by existing constraint-based schedulers, such as minimizing SLO violations. The experimental results show that the performance of the DRL deployment algorithm improves by 56.2% compared to the average RPS of the baseline, indicating that the manual deployment scheme can only meet the basic requirements but cannot cover the complex interactions between containers under constraints from a global perspective. This limitation severely limits the performance of the whole pod. Meanwhile, based on previous experience, the time consumption of a single deployment scheme is about 1 hour, while the time consumption of the DRL deployment algorithm may be less than 7.5 minutes.