Pipeline Scheduling Research Articles

Deep reinforcement learning optimization scheduling algorithm for continuous production line

In order to improve the scheduling effect of continuous production lines and enhance the processing efficiency of production lines, a deep reinforcement learning optimization scheduling algorithm for continuous production lines is proposed. The Monte Carlo algorithm and Bayesian evaluation method are combined to reduce the data complexity of the continuous production line pipeline problem; the deep neural network model is used to optimize the pipeline scheduling parameters, evaluate and encode them; the iterative greedy algorithm is combined with the deep reinforcement learning method to implement the model solution for the scheduling data problem and realize the scheduling of continuous production lines. The experimental results show that the optimal comprehensive evaluation results of the scheduling results of the proposed algorithm are all higher than 0.9531, the process delay is optimized to less than 5 min, the collection speed is fast, and the processing efficiency of the production line is improved.

Multi-Stream Scheduling of Inference Pipelines on Edge Devices - a DRL Approach

Low-power edge devices equipped with Graphics Processing Units (GPUs) are a popular target platform for real-time scheduling of inference pipelines. Such application-architecture combinations are popular in Advanced Driver-assistance Systems for aiding in the real-time decision-making of automotive controllers. However, the real-time throughput sustainable by such inference pipelines is limited by resource constraints of the target edge devices. Modern GPUs, both in edge devices and workstation variants, support the facility of concurrent execution of computation kernels and data transfers using the primitive of streams , also allowing for the assignment of priority to these streams. This opens up the possibility of executing computation layers of inference pipelines within a multi-priority, multi-stream environment on the GPU. However, manually co-scheduling such applications while satisfying their throughput requirement and platform memory budget may require an unmanageable number of profiling runs. In this work, we propose a Deep Reinforcement Learning (DRL)-based method for deciding the start time of various operations in each pipeline layer while optimizing the latency of execution of inference pipelines as well as memory consumption. Experimental results demonstrate the promising efficacy of the proposed DRL approach in comparison with the baseline methods, particularly in terms of real-time performance enhancements, schedulability ratio, and memory savings. We have additionally assessed the effectiveness of the proposed DRL approach using a real-time traffic simulation tool IPG CarMaker.

Two-stage optimization model for scheduling multiproduct pipeline network with multi-source and multi-terminal

The multiproduct pipeline network with multi-source and multi-terminal encompasses multi-point injection and multi-point distribution. Upon reorganization of batches at transfer depots, the intricacies of precise batch tracking become increasingly complex, thereby rendering the optimization of multiproduct pipeline network scheduling particularly arduous. In this paper, we introduce a dynamic index batch numbering method. By delineating the boundaries of intermediate injection points, the issue is bifurcated into two sub-problems: the optimization of batch planning for multiproduct pipeline with single-source and multi-terminal and the optimization of resource allocation at intermediate oil depots. The proposed algorithm is validated on two genuine multiproduct pipeline networks in China, demonstrating its capability to generate feasible solutions within a computational constraint of 18,000 CPU seconds. The findings reveal that this methodology holistically considers the entire pipeline network system, encompassing upstream oil intake, downstream oil supply, and the tank capacities of transfer depots, thereby establishing a coherent link between pipeline scheduling and depot planning. This paper may serve as a reference for the automated formulation of multiproduct pipeline network scheduling strategies.

LUAEMA: A Loop Unrolling Approach Extending Memory Accessing for Vector Very-Long-Instruction-Word Digital Signal Processor with Multiple Register Files

Loop unrolling can provide more instruction-level parallelism opportunities for code and enables a greater range of instruction pipeline scheduling. In high-performance very-long-instruction-word (VLIW) digital signal processors (DSPs), there are special registers to address. To further improve the instruction-level parallelism of code for such DSPs by making full use of these registers, in this paper, we propose a more effective loop unrolling approach through extending memory accessing (LUAEMA). In this approach, the final unrolling factor is computed by a model in which every register kind and every memory accessing operation are considered. For basic digital signal processing algorithms, the unrolling factor under the LUAEMA is larger than that under the conventional loop unrolling approach. We also provide the opportunity to reduce the number of instructions in a loop during the code transformation of loop unrolling. The experimental results show that the loop unrolling approach proposed in this paper can achieve an average speedup ratio ranging from 1.14 to 1.81 compared with the conventional loop unrolling approach. For some algorithms, the peak speedup ratio is up to 2.11.

Optimizing Job Offloading Schedule for Collaborative DNN Inference

Deep Neural Networks (DNNs) have been widely deployed in mobile applications. DNN inference latency is a critical metric to measure the service quality of those applications. Collaborative inference is a promising approach for latency optimization, where partial inference workloads are offloaded from mobile devices to cloud servers. Model partition problems for collaborative inference have been well studied. However, little attention has been paid to optimizing offloading pipeline for multiple DNN inference jobs. In practice, mobile devices usually need to process multiple DNN inference jobs simultaneously. We propose to jointly optimize the DNN partitioning and pipeline scheduling for multiple inference jobs. We theoretically analyze the optimal scheduling conditions for homogeneous chain-structure DNNs. Based on the analysis, we proposed near-optimal partitioning and scheduling methods for chain-structure DNNs. We also extend those methods for general-structure DNNs. In addition, we extend our problem scenario to handle heterogeneous DNN inference jobs. A layer-level scheduling algorithm is proposed. Theoretical analyses show that our proposed method is optimal when computation graphs are tree-structure. Our joint optimization methods are evaluated in a real-world testbed. Experiment results show that our methods can significantly reduce the overall inference latency of multiple inference jobs compared to partition-only or schedule-only approaches.

Establishment of General Oil Pipeline Scheduling Model and Research on Interface Tracking Calculation Method

In order to understand the establishment of oil pipeline scheduling model and interface tracking calculation method, a research on the establishment of a universal product oil pipeline scheduling model and interface tracking calculation method has been proposed. Establish an automatic formulation and optimization model for the scheduling plan of finished oil pipelines with the objective of minimizing the consumption cost of the entire pump unit. The model mainly adopts a centralized distribution method, considering the hydraulic constraints of the pipeline, and adopts a genetic algorithm to optimize and solve the model. This article provides a detailed introduction to the encoding method of genetic algorithm for optimizing the scheduling plan of finished oil pipelines, the selection of main operators in the algorithm, the establishment of fitness functions, and the introduction of penalty functions. Based on the established mathematical model, an automatic programming and optimization program for product oil pipeline scheduling plan was developed using C language in Windows 7 environment. Secondly, through practical applications, the actual pipeline operation data is basically consistent with the software calculation results, with an error of no more than 5%; As the running time changes, the pump cost fluctuates greatly; Using a larger throughput can reduce the amount of mixed oil; Therefore, this mathematical model can better meet the actual needs of pipeline operation, facilitate the rapid identification of the most sensitive parameter indicators, and adjust the scheduling plan in a timely manner, reduce operational economic losses, and improve the efficiency of product oil pipeline scheduling and operation.

Multichannel Pipelined Scheduling for Raw Data Convergecast in Sensor-Cloud

Multichannel Pipelined Scheduling for Raw Data Convergecast in Sensor-Cloud

Scalable and Efficient Full-Graph GNN Training for Large Graphs

Graph Neural Networks (GNNs) have emerged as powerful tools to capture structural information from graph-structured data, achieving state-of-the-art performance on applications such as recommendation, knowledge graph, and search. Graphs in these domains typically contain hundreds of millions of nodes and billions of edges. However, previous GNN systems demonstrate poor scalability because large and interleaved computation dependencies in GNN training cause significant overhead in current parallelization methods. We present G3, a distributed system that can efficiently train GNNs over billion-edge graphs at scale. G3 introduces GNN hybrid parallelism which synthesizes three dimensions of parallelism to scale out GNN training by sharing intermediate results peer-to-peer in fine granularity, eliminating layer-wise barriers for global collective communication or neighbor replications as seen in prior works. G3 leverages locality-aware iterative partitioning and multi-level pipeline scheduling to exploit acceleration opportunities by distributing balanced workload among workers and overlapping computation with communication in both inter-layer and intra-layer training processes. We show via a prototype implementation and comprehensive experiments that G3 can achieve as much as 2.24x speedup in a 16-node cluster, and better final accuracy over prior works.

Merak: An Efficient Distributed DNN Training Framework With Automated 3D Parallelism for Giant Foundation Models

Foundation models are in the process of becoming the dominant deep learning technology. Pretraining a foundation model is always time-consuming due to the large scale of both the model parameter and training dataset. Besides being computing-intensive, the pretraining process is extremely memory- and communication-intensive. These challenges make it necessary to apply 3D parallelism, which integrates data parallelism, pipeline model parallelism, and tensor model parallelism, to achieve high training efficiency. However, current 3D parallelism frameworks still encounter two issues: i) they are not transparent to model developers, requiring manual model modification to parallelize training, and ii) their utilization of computation resources, GPU memory, and network bandwidth is insufficient. We propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Merak</i> , an automated 3D parallelism deep learning training framework with high resource utilization. Merak automatically deploys 3D parallelism with an automatic model partitioner, which includes a graph-sharding algorithm and proxy node-based model graph. Merak also offers a non-intrusive API to scale out foundation model training with minimal code modification. In addition, we design a high-performance 3D parallel runtime engine that employs several techniques to exploit available training resources, including a shifted critical path pipeline schedule that increases computation utilization, stage-aware recomputation that makes use of idle worker memory, and sub-pipelined tensor model parallelism that overlaps communication and computation. Experiments on 64 GPUs demonstrate Merak's capability to speed up training performance over state-of-the-art 3D parallelism frameworks of models with 1.5, 2.5, 8.3, and 20 billion parameters by up to 1.42, 1.39, 1.43, and 1.61×, respectively.

Enabling Organizations to Participate in the Ad Hoc Scheduling of a Cross-Organizational Data Pipeline

Enabling Organizations to Participate in the Ad Hoc Scheduling of a Cross-Organizational Data Pipeline

FPGA Acceleration of Probabilistic Sentential Decision Diagrams with High-level Synthesis

Probabilistic Sentential Decision Diagrams (PSDDs) provide efficient methods for modeling and reasoning with probability distributions in the presence of massive logical constraints. PSDDs can also be synthesized from graphical models such as Bayesian networks (BNs) therefore offering a new set of tools for performing inference on these models (in time linear in the PSDD size). Despite these favorable characteristics of PSDDs, we have found multiple challenges in PSDD’s FPGA acceleration. Problems include limited parallelism, data dependency, and small pipeline iterations. In this article, we propose several optimization techniques to solve these issues with novel pipeline scheduling and parallelization schemes. We designed the PSDD kernel with a high-level synthesis (HLS) tool for ease of implementation and verified it on the Xilinx Alveo U250 board. Experimental results show that our methods improve the baseline FPGA HLS implementation performance by 2,200X and the multicore CPU implementation by 20X. The proposed design also outperforms state-of-the-art BN and Sum Product Network (SPN) accelerators that store the graph information in memory.

An efficient GRASP-like algorithm for the multi-product straight pipeline scheduling problem

The present work deals with the multi-product straight pipeline scheduling problem. The considered straight pipeline system is used to transport refined petroleum products from a single source (refinery with storage tanks) to a single destination (distribution center). The main objective is to find a sequence of batches which aims to maximize the total volume to be transported via the pipeline, while meeting the daily customer demands over a fixed time horizon. Each batch contains only one product with a volume between its upper and lower bounds. Constraints related to inventory levels, batch settling periods and forbidden sequences between pairs of products must be respected, and pipeline stoppage periods should also be handled. A Mixed Integer Linear Programming (MILP) model and a Greedy Randomized Adaptive Search Procedure (GRASP)-like algorithm are proposed to tackle the problem under study. The MILP model is based on the discharging time axis with continuous representation for both time and volume formulation. The GRASP-like algorithm is composed of a construction method and an improvement procedure. The construction method generates a new sequence of batches in a random way with the use of a repair process to obtain an initial solution that satisfies all demands. Then, the improvement procedure is attempted to increase the fill rate of pipeline using volume optimization operators without change the sequence of products. A set of instances was generated from a real case study in order to validate the proposed approach. The performance of our approach is benchmarked against the MILP model (solved using Gurobi Solver), and the numerical experiments proved that the proposed approach obtains very competitive results both in term of solution quality and CPU time.

A novel two-phase decomposition-based algorithm to solve MINLP pipeline scheduling problem

A novel two-phase decomposition-based algorithm to solve MINLP pipeline scheduling problem

Sensory Data Aggregation in Internet of Things: Period-Driven Pipeline Scheduling Approach

The Smart City contexts and the adoption of Industry 4.0 are being upgraded with the latest technologies throughout all systems. New data are continuously collected in huge amounts; therefore, an efficient data aggregation scheduling scheme is highly demanded. This paper addresses the minimum time aggregation scheduling problem in duty-cycled sensor networks. Existing solutions schedule the sensory data through predefined routing structures, which limit the utilization of diverse active time slots of sensors. We propose a period-driven pipeline scheduling approach, namely PDA, that simultaneously grows the aggregation tree and assigns a transmission schedule for each node being added to the tree. Particularly, this process is performed in a top-down manner. In each iteration, corresponding to a time slot, PDA uses a multi-level ranking strategy to schedule several <inline-formula><tex-math notation="LaTeX">$\langle sender, receiver \rangle$</tex-math></inline-formula> pairs, so that in a working period ahead, the possibility to pipeline as many transmissions as possible is high. Intensive simulation results show that the proposed scheme notably works better than the best known recent algorithms by having up to 35 percent shorter aggregation time, as well as having a significantly improved network throughput and time utilization.

BaPipe: Balanced Pipeline Parallelism for DNN Training

The size of deep neural networks (DNNs) grows rapidly as the complexity of the machine learning algorithm increases. Distributed deep learning based on model parallelism has been widely used to satisfy the requirements of DNN training related to computation and memory. In this paper, we propose a training framework for pipeline parallelism called BaPipe (Balanced Pipeline) that can automatically explore methods to schedule pipeline parallelism and balanced partition strategies for DNN training on heterogeneous accelerator clusters. In BaPipe, each accelerator calculates the forward and backward propagation for the assigned partition of networks to implement an intra-batch pipeline parallelism strategy. By considering the parameters of DNN models as well as the computation, memory, and communication resources of each accelerator, BaPipe automatically selects the most suitable method of pipeline scheduling from among multiple proposed scheduling modes. It also uses a novel strategy to automatically investigate load balancing in the context of inter-layer partition, intra-layer partition, and coarse-grained partition. We trained such DNNs as VGG-16, ResNet-50, and Google’s Neural Machine Translation (GNMT) on GPU clusters, and simulated the training-related performance of FPGA clusters. Compared with the state-of-the-art frameworks for data parallelism (DP) and pipeline parallelism, BaPipe provides a speedup of [Formula: see text] and [Formula: see text] of memory reduction on various homogeneous and heterogeneous platforms.

Accelerating Video Captioning on Heterogeneous System Architectures

Video captioning is a core technology to many important applications, such as AI-assisted medical diagnosis, video question answering, storytelling through videos, and lip-reading. Video captioning employs a hybrid CNN + RNN model. Accelerating such a hybrid model on a heterogeneous system is challenging for two reasons. First, CNN and RNN exhibit very different computing behaviors, making the mapping between computation and heterogeneous devices difficult. Second, data dependency exists between the CNN and RNN within a video frame and between adjacent RNNs across video frames. These data dependencies prohibit the full parallelization of the hybrid model. The issues also include the utilization of accelerator resources, which is critical to maximizing the performance. In this work, we propose a fine-grained scheduling scheme for mapping computation and devices within a video frame, and a pipeline scheduling scheme for exploiting maximum parallelism between the execution of the video frames. In addition, we propose two capacity-guided scheduling methods. On the server, the concurrent kernel execution mechanism is exploited for improving GPU utilization. On the edge platform, we rearrange CNN computation among the CPU and EdgeTPUs guided by the EdgeTPU’s SRAM capacity so that balanced computation is achieved and off-chip memory overhead is minimized. Experimental results show that our scheduling scheme improves video captioning performance by up to 3.24 \( \times \) with CPU + GPU collaboration over the GPU-only execution. On an edge platform with an ARM CPU and two EdgeTPUs, our CPU + EdgeTPU scheduling exhibits outstanding performance, which achieves up to 54.9 \( \times \) speedup compared to using ARM CPU only and can perform video captioning of 59 frames per second.

AMPS: An Automated Mesochronous Pipeline Scheduler and Design Space Explorer for High Performance Digital Circuits

In the Mesochronous Pipeline (MP), the clock period is the maximal of differences between the maximum/minimum delays of all stages. This value is less than the maximum delay between the memory elements, so MP is operating faster than the conventional pipeline (CP). To take full advantages of MP, in this paper, Automated Mesochronous Pipeline Scheduler (AMPS) for high performance digital circuits, is proposed which provides all allowed partitioning options generated by register allocation procedures obtained with varying maximum delay differences of stages. This allows a designer to select from design space produced by AMPS based on the desired specification. Unlike the traditional MP, AMPS utilizes automated scripts to extract the timing characteristics of the synchronizers and gates. This enables a designer to explore trade-offs between latency, power, and frequency. To evaluate our toolset, an 8-bit Carry Increment Adder (CIA), a 4-bit Binary-Coded Decimal (BCD) adder, a 4-bit ALU, and a set of circuits from ISCAS85 and ISCAS89 benchmarks are considered. All circuit level simulations are performed in the 65nm CMOS standard technology node. AMPS improves the frequency for its best solutions about 127.94% (on average) in comparison to the conventional pipeline scheme.

Solution] End-to-end Scheduling of Real-time Task Pipelines on Multiprocessors

Task pipelines are common in today’s embedded systems, as data moves from source to sink in sensing-processing-actuation task chains. A real-time task pipeline is constructed by connecting a series of periodic tasks with data buffers. In a time-critical system, end-to-end timing and data-transfer properties of a task pipeline must be guaranteed. A guarantee could be mathematically expressed by assigning constraints to the tasks of a pipeline. However, deriving task scheduling parameters to meet end-to-end guarantees is an NP-hard constraint optimization problem. Hence, a traditional constraint solver is not a suitable runtime solution.In this paper, we present a heuristic constraint solver algorithm, CoPi, to derive the execution times and periods of pipelined tasks that meet the end-to-end constraints and schedulability requirements. We consider two upper bound constraints on a task pipeline: end-to-end delay and loss-rate. After satisfying these constraints, CoPi schedules a pipeline as a set of asynchronous and data independent periodic tasks, under the rate-monotonic scheduling algorithm. Simulations show that CoPi has a comparable pipeline acceptance ratio and significantly better runtime than open-source MINLPsolvers. Furthermore, we use CoPi to map multiple task pipelines to a multiprocessor system. We demonstrate that a partitioned multiprocessor scheduling algorithm coupled with CoPi accommodates dynamically appearing pipelines, while attempting to minimize task migrations.

A MILP approach for detailed pipeline scheduling and storage management problem in the phosphate industry

Short-term detailed multiproduct pipeline scheduling is a complex problem with many industry-specific constraints. We are interested in solving an operational scheduling problem of a straight pipeline in the phosphate industry. In this article, we propose a mixed-integer linear programming model (MILP) with a discrete-time formulation to provide a detailed scheduling solution for a storage-sensitive problem. The objective is to determine the charging and discharging schedule of the slurry pipeline, to satisfy products demand while respecting all pipeline and storage constraints. The model was tested on different instances, proving to be able to minimize products sold-outs and the total water quantity scheduled.

Discrete-Time Approach to Operational Scheduling of Treelike Pipelines with Multiple Input and Output Nodes

Discrete-Time Approach to Operational Scheduling of Treelike Pipelines with Multiple Input and Output Nodes