Thread Scheduling Research Articles

Pac-Sim: Simulation of Multi-threaded Workloads using Intelligent, Live Sampling

High-performance, multi-core processors are the key to accelerating workloads in several application domains. To continue to scale performance at the limit of Moore’s Law and Dennard scaling, software and hardware designers have turned to dynamic solutions that adapt to the needs of applications in a transparent, automatic way. For example, modern hardware improves its performance and power efficiency by changing the hardware configuration, like the frequency and voltage of cores, according to a number of parameters, such as the technology used or the workload running at the time. With this level of dynamism, it is essential to simulate next-generation multi-core processors in a way that can both respond to system changes and accurately determine system performance metrics. Currently, no sampled simulation platform can achieve these goals of dynamic, fast, and accurate simulation of multi-threaded workloads. In this work, we propose a solution that allows for fast, accurate simulation in the presence of both hardware and software dynamism. To accomplish this goal, we present Pac-Sim, a novel sampled simulation methodology for fast, accurate sampled simulation that requires no upfront analysis of the workload. With our proposed methodology, it is now possible to simulate long-running dynamically scheduled multi-threaded programs with significant simulation speedups, even in the presence of dynamic hardware events. We evaluate Pac-Sim using the SPEC CPU2017, NPB, and PARSEC multi-threaded benchmarks with both static and dynamic thread scheduling. The experimental results show that Pac-Sim achieves a very low sampling error of 1.63% and 3.81% on average for statically and dynamically scheduled benchmarks, respectively. Pac-Sim also demonstrates significant simulation speedups as high as 523.5 × (210.3 × on average) for the training input set of SPEC CPU2017 running eight threads.

Codebase release 1.1 for GAPS

The Single Instruction, Multiple Thread (SIMT) paradigm of GPU programming does not support the branching nature of a parton shower algorithm by definition. However, modern GPUs are designed to schedule threads with diverging processes independently, allowing them to handle such branches. With regular thread synchronisation and careful treatment of the individual steps, one can simulate a parton shower on a GPU. We present a Sudakov veto algorithm designed to simulate parton branching on multiple events in parallel. We also release a CUDA C++ program that generates matrix elements, showers partons and computes jet rates and event shapes for LEP at 91.2 GeV on a GPU. To benchmark its performance, we also provide a near-identical C++ program designed to simulate events serially on a CPU. While the consequences of branching are not absent, we demonstrate that a GPU can provide the throughput of a many-core CPU. As an example, we show that the time taken to shower 10^6106 events on one NVIDIA TESLA V100 GPU is equivalent to that of 295 Intel Xeon E5-2620 CPU cores.

An algorithm to parallelise parton showers on a GPU

The Single Instruction, Multiple Thread (SIMT) paradigm of GPU programming does not support the branching nature of a parton shower algorithm by definition. However, modern GPUs are designed to schedule threads with diverging processes independently, allowing them to handle such branches. With regular thread synchronisation and careful treatment of the individual steps, one can simulate a parton shower on a GPU. We present a Sudakov veto algorithm designed to simulate parton branching on multiple events in parallel. We also release a CUDA C++ program that generates matrix elements, showers partons and computes jet rates and event shapes for LEP at 91.2 GeV on a GPU. To benchmark its performance, we also provide a near-identical C++ program designed to simulate events serially on a CPU. While the consequences of branching are not absent, we demonstrate that a GPU can provide the throughput of a many-core CPU. As an example, we show that the time taken to shower 10^6106 events on one NVIDIA TESLA V100 GPU is equivalent to that of 295 Intel Xeon E5-2620 CPU cores.

Software Model Checking Distributed Applications: A Hybrid Approach

Developing reliable distributed systems poses many challenges such as concurrency, failure handling, and scalability. It is due to the non-deterministic execution of threads within processes, and communication means. Formal verification methods, such as model checking, have been used to ensure the reliability of safety-critical systems. This technique systematically explores the complete behavior of the system under test (SUT), investigating each reachable state with different thread schedules. Recent software model-checking tools, employing cache and centralization, have been applied to distributed systems. The caching technique can only check one process at a time, while the centralization technique verifies all processes simultaneously. In the centralization technique, two "ArrayByteQueue" buffers are utilized to store communication and process data byte-by-byte. However, during the backtracking process, the read and write operations, involving data insertion and removal from the queue, become resource-intensive. As a consequence, existing interprocess communication (IPC) models encounter computational limitations and experience a rapid state space explosion. To address these challenges, our work proposes the remodeling of IPC models by introducing a request and response tree structure to store communication data. Additionally, we employ pointers to navigate through the data during the backtracking process. Through experimental evaluations, the proposed implementation choices have demonstrated significantly improved performance across various metrics. By incorporating the request and response tree, we enhance the efficiency of storing communication data, while the use of pointers optimizes navigation during backtracking. This remodeling of IPC models shows promise in mitigating computational limitations and state space explosion, thereby enhancing the model-checking process in distributed systems. Our research contributes to advancing the field of model checking in distributed systems and offers potential solutions to the challenges associated with resource-intensive read and write operations during the model-checking process.

Power Consumption Prediction in Real-Time Multitasking Systems

The paper presents a practical methodology for the prediction of power consumption of a multitask real-time system. The authors briefly recall the structure of the original multitasking time-predictable system. The system has a regular and scalable architecture. It consists of many cores based on pipeline processing with threads’ interleaving mechanism. In previous works, the authors have presented a number of issues related to the methodology of designing a time-predictable system and the scheduling of hardware threads with different design goals and constraints. The current article presents an accurate method for measuring the dynamic power dissipated in a system for various scenarios and the system configuration. The proposed approach is based on continuous monitoring of the power consumed by the FPGA, transmitting the measurements to the computer from the measuring device, and online analysis of the obtained results. The authors based their power estimation method on the original task frequency factor (TF). A theoretical analysis was made, and a series of practical experiments were carried out. The results obtained enable accurate estimation of the power requirements for the tasks performed in the real-time system. The presented technique can be useful during the design stage of multitasking real-time systems.

Automatic Parallelism Management

On any modern computer architecture today, parallelism comes with a modest cost, born from the creation and management of threads or tasks. Today, programmers battle this cost by manually optimizing/tuning their codes to minimize the cost of parallelism without harming its benefit, performance. This is a difficult battle: programmers must reason about architectural constant factors hidden behind layers of software abstractions, including thread schedulers and memory managers, and their impact on performance, also at scale. In languages that support higher-order functions, the battle hardens: higher order functions can make it difficult, if not impossible, to reason about the cost and benefits of parallelism. Motivated by these challenges and the numerous advantages of high-level languages, we believe that it has become essential to manage parallelism automatically so as to minimize its cost and maximize its benefit. This is a challenging problem, even when considered on a case-by-case, application-specific basis. But if a solution were possible, then it could combine the many correctness benefits of high-level languages with performance by managing parallelism without the programmer effort needed to ensure performance. This paper proposes techniques for such automatic management of parallelism by combining static (compilation) and run-time techniques. Specifically, we consider the Parallel ML language with task parallelism, and describe a compiler pipeline that embeds "potential parallelism" directly into the call-stack and avoids the cost of task creation by default. We then pair this compilation pipeline with a run-time system that dynamically converts potential parallelism into actual parallel tasks. Together, the compiler and run-time system guarantee that the cost of parallelism remains low without losing its benefit. We prove that our techniques have no asymptotic impact on the work and span of parallel programs and thus preserve their asymptotic properties. We implement the proposed techniques by extending the MPL compiler for Parallel ML and show that it can eliminate the burden of manual optimization while delivering good practical performance.

Characterizing Multi-Chip GPU Data Sharing

Multi-chip Graphics Processing Unit (GPU) systems are critical to scale performance beyond a single GPU chip for a wide variety of important emerging applications. A key challenge for multi-chip GPUs, though, is how to overcome the bandwidth gap between inter-chip and intra-chip communication. Accesses to shared data, i.e., data accessed by multiple chips, pose a major performance challenge as they incur remote memory accesses possibly congesting the inter-chip links and degrading overall system performance. This article characterizes the shared dataset in multi-chip GPUs in terms of (1) truly versus falsely shared data, (2) how the shared dataset scales with input size, (3) along which dimensions the shared dataset scales, and (4) how sensitive the shared dataset is with respect to the input’s characteristics, i.e., node degree and connectivity in graph workloads. We observe significant variety in scaling behavior across workloads: some workloads feature a shared dataset that scales linearly with input size, whereas others feature sublinear scaling (following a \(\sqrt {2}\) or \(\sqrt [3]{2}\) relationship). We further demonstrate how the shared dataset affects the optimum last-level cache organization (memory-side versus SM-side) in multi-chip GPUs, as well as optimum memory page allocation and thread scheduling policy. Sensitivity analyses demonstrate the insights across the broad design space.

A scheduling algorithm based on critical factors for heterogeneous multicore processors

SummaryAs the development of chip manufacturing technology slows down, high‐performance processors often have high energy consumption and high heat generation. Therefore, heterogeneous multi‐core processors become more and more popular, and the heterogeneous multi‐core processors is adopted to execute programs. At present, the general program consists of multiple threads. To reach goals of accelerating program execution and reducing energy consumption and heat generation of system, a suitable thread scheduling algorithm for heterogeneous multi‐core processors is needed. In this article, a thread scheduling algorithm based on multiple critical scheduling factors is proposed. First, a prediction model of thread performance and energy consumption is used to predict the core sensitivity of threads. Then, critical threads are judged and accelerated by collecting the synchronization information between threads. Finally, the load balancing method based on the computing power of cores and the core sensitivity of threads is employed to perform system load balancing, which ensures the fairness of the scheduling. Several experiments are provided, and the results show that the proposed algorithm can obtain better performance of thread schedule.

Compilation Optimization of DCU-oriented OpenMP Thread Scheduling

OpenMP is one of the mainstream parallel programming models in recent years. After version 4.0, OpenMP introduced a new target instruction to increase the functionality of heterogeneous programming, called OpenMP Offload. For the domestic heterogeneous platform DCU, the thread scheduling algorithm under OpenMP parallel computing has low performance in the default mode, which does not take the best advantage of GPU parallel computing and has wasted resources. To address this problem, this paper performs algorithm improvement at the compiler level, analyzes the available resources of the system by combining the DCU hardware facilities, then further parses the program based on its array information to get its program iteration number, reallocates the number of threads for different execution modes in OpenMP, and optimizes the thread group increase factor by combining the DCU hardware information to adjust the thread. This paper uses the SPEC ACCEL test set to optimize the number of threads in the DCU. In this paper, we use the SPEC ACCEL test set and Polybench standard test set to test the redistribution of threads and thread groups in two parallel modes using the thread scheduling optimization algorithm. The average speedup ratio of ACCEL was improved by 40%.

A thread scheduling model for highly concurrent IoT platforms

To accept and process the requests many IoT devices give, the server on the IoT platform must open many threads. Enhancing server speed while working with constrained processors, memory, and other resources is extremely difficult. In this paper, we propose a thread scheduling model that abstracts server resources, combines queues to handle resource contention, avoids the issue of frequent switching of the server’s underlying processor context in traditional thread scheduling, and combines with a specific thread weighting mechanism to enable the server to handle more requests from IoT devices with the same overhead. Finally, we carry out tests to show that our model is valid.

Temporal Set Inversion for Animated Implicits

We exploit the temporal coherence of closed-form animated implicit surfaces by locally re-evaluating an octree-like discretization of the implicit field only as and where is necessary to rigorously maintain a global error invariant over time, thereby saving resources in static or slowly-evolving areas far from the motion where per-frame updates are not necessary. We treat implicit surface rendering as a special case of the continuous constraint satisfaction problem of set inversion, which seeks preimages of arbitrary sets under vector-valued functions. From this perspective, we formalize a temporally-coherent set inversion algorithm that localizes changes in the field by range-bounding its time derivatives using interval arithmetic. We implement our algorithm on the GPU using persistent thread scheduling and apply it to the scalar case of implicit surface and swept volume rendering where we achieve significant speedups in complex scenes with localized deformations like those found in games and modelling applications where interactivity is required and bounded-error approximation is acceptable.

Hierarchical Model Parallelism for Optimizing Inference on Many-core Processor via Decoupled 3D-CNN Structure

The tremendous success of convolutional neural network (CNN) has made it ubiquitous in many fields of human endeavor. Many applications such as biomedical analysis and scientific data analysis involve analyzing volumetric data. This spawns huge demand for 3D-CNN. Although accelerators such as GPU may provide higher throughput on deep learning applications, they may not be available in all scenarios. CPU, especially many-core CPU with non-uniform memory access (NUMA) architecture, remains an attractive choice for deep learning inference in many scenarios. In this article, we propose a distributed inference solution for 3D-CNN that targets on the emerging ARM many-core CPU platform. A hierarchical partition approach is claimed to accelerate 3D-CNN inference by exploiting characteristics of memory and cache on ARM many-core CPU. Based on the hierarchical model partition approach, other optimization techniques such as NUMA-aware thread scheduling and optimization of 3D-img2row convolution are designed to exploit the potential of ARM many-core CPU for 3D-CNN. We evaluate our proposed inference solution with several classic 3D-CNNs: C3D, 3D-resnet34, 3D-resnet50, 3D-vgg11, and P3D. Our experimental results show that our solution can boost the performance of the 3D-CNN inference, and achieve much better scalability, with a negligible fluctuation in accuracy. When employing our 3D-CNN inference solution on ACL libraries, it can outperform naive ACL implementations by 11× to 50× on ARM many-core processor. When employing our 3D-CNN inference solution on NCNN libraries, it can outperform the naive NCNN implementations by 5.2× to 14.2× on ARM many-core processor.

Full-Stack Optimizing Transformer Inference on ARM Many-Core CPU

The past several years have witnessed tremendous success of transformer models in natural language processing (NLP), and their current landscape is increasingly diverse. Although GPU gradually becomes the dominating workhorse and de facto standard for deep learning, there are still many scenarios where using CPU remains a prevalent choice. Recently, ARM many-core processor starts emigrating to cloud computing and high-performance computing, which is promising to deploy transformer inference. In this paper, we identify several performance bottlenecks of existing inference runtime on many-core CPU including low-core usage, isolated thread configuration, inappropriate implementation of general matrix multiply (GEMM), and redundant computations for variable-length inputs. To tackle these problems, full-stack and cross-layer optimizations are conducted for these challenges from deep learning service level to the neural network operator level. We explore multi-instance parallelization at the service level to improve CPU core usage. To improve parallel efficiency of the inference runtime, we design NUMA-aware thread scheduling and a look-up table for optimal parallel configurations. The GEMM implementation is tailored for some critical modules such self-attention module to exploit the characteristics of transformer workload. To eliminate redundant computations, a novel storage format is designed and implemented to pack sparse data and a load balancing strategy is proposed for tasks with different sparsity. Experimental results show that our implementation can outperform existing solutions by 1.1x to 6x for different transformer-based models with fixed-length inputs. For variable-length inputs, it achieves 1.9x to 6x speedups on Kungpeng920 processor and 3x to 8x speedups on Ampere Altra processor depending on different sequence lengths and batch sizes.

Improving strong scalability of electronic structure simulations with reduced overhead of communications

Physically realizable semiconductor electronic devices easily include more than several million atoms even though they are in nanoscale, so electronic structure simulations of huge atomic systems are the key to predict performance characteristics of such devices. The empirical tight-binding (TB) model, which uses a finite set of localized bases for descriptions of a single atom with assumption of nearest-neighbor coupling, has been extensively adopted to solve electronic structures of large-scale atomic systems, and the size-capability of TB simulations has been continuously improved with aids of parallel computing. Recently, we have demonstrated a strong scalability of TB simulations up to 2,500 computing nodes for a model problem consisting of 400 million atoms, where the scalability is limited due to the overhead of collective communications that the core numerical method must involve to get a partial eigenspectrum of the sparse Hamiltonian system matrix. To reduce the cost of Message Passing Interface (MPI) communications and its effect on the end-to-end simulation time, here we design an equivalent transformation of the sparse matrix eigensolver, and employ a core dedicated to MPI communications with dynamic scheduling of threads. Through rigorous benchmark tests against model problems of various sizes, it is confirmed that our method improves the strong scalability of TB simulations by more than 10% in a huge computing environment, by reducing the time-cost of both core computations and collective communications.

An algorithm for scheduling of threads for system and application code split approach in dynamic malware analysis

An algorithm for scheduling of threads for system and application code split approach in dynamic malware analysis

SLITS: Sparsity-Lightened Intelligent Thread Scheduling

A diverse set of scheduling objectives (e.g., resource contention, fairness, priority, etc.) breed a series of objective-specific schedulers for multi-core architectures. Existing designs incorporate thread-to-thread statistics at runtime, and schedule threads based on such an abstraction (we formalize thread-to-thread interaction as the Thread-Interaction Matrix). However, such an abstraction also reveals a consistently-overlooked issue: the Thread-Interaction Matrix (TIM) is highly sparse. Therefore, existing designs can only deliver sub-optimal decisions, since the sparsity issue limits the amount of thread permutations (and its statistics) to be exploited when performing scheduling decisions. We introduce Sparsity-Lightened Intelligent Thread Scheduling (SLITS), a general scheduler design for mitigating the sparsity issue of TIM, with the customizability for different scheduling objectives. SLITS is designed upon the key insight that: the sparsity issue of the TIM can be effectively mitigated via advanced Machine Learning (ML) techniques. SLITS has three components. First, SLITS profiles Thread Interactions for only a small number of thread permutations, and form the TIM using the run-time statistics. Second, SLITS estimates the missing values in the TIM using Factorization Machine (FM), a novel ML technique that can fill in the missing values within a large-scale sparse matrix based on the limited information. Third, SLITS leverages Lazy Reschedule, a general mechanism as the building block for customizing different scheduling policies for different scheduling objectives. We show how SLITS can be (1) customized for different scheduling objectives, including resource contention and fairness; and (2) implemented with only negligible hardware costs. We also discuss how SLITS can be potentially applied to other contexts of thread scheduling. We evaluate two SLITS variants against four state-of-the-art scheduler designs. We highlight that, averaged across 11 benchmarks, SLITS achieves an average speedup of 1.08X over the de facto standard for thread scheduler - the Completely Fair Scheduler, under the 16-core setting for a variety of number of threads (i.e., 32, 64 and 128). Our analysis reveals that the benefits of SLITS are credited to significant improvements of cache utilization. In addition, our experimental results confirm that SLITS is scalable and the benefits are robust when of the number of threads increases. We also perform extensive studies to (1) break down SLITS components to justify the synergy of our design choices, (2) examine the impacts of varying the estimation coverage of FM, (3) justify the necessity of Lazy Reschedule rather than periodic rescheduling, and (4) demonstrate the hardware overheads for SLITS implementations can be marginal (<1% chip area and power).

Multiprocessing for the Particle Tracking Model MODPATH.

Particle tracking has several important applications for solute transport studies in aquifer systems. Travel time distribution at observation points, particle coordinates in time and streamlines are some practical results providing information of expected transport patterns and interaction with boundary conditions. However, flow model complexity and simultaneous displacement of multiple particle groups leads to rapid increase of computational requirements. MODPATH is a particle tracking engine for MODFLOW models and source code displays potential for parallel processing of particles. This article addresses the implementation of this feature with the OpenMP library. Two synthetic aquifer applications are employed for performance tests on a desktop computer with increasing number of particles. Speed up analysis shows that dynamic thread scheduling is preferable for highly heterogeneous flows, providing processing adaptivity to the presence of slow particles. In simulations writing particles position in time, thread exclusive output files lead to higher speed up factors. Results show that above a threshold number of particles, simulation runtimes become independent of flow model grid complexity and are controlled by the large number of particles, then parallel processing reduces simulation runtimes for the particle tracking model MODPATH. Article impact statement: Parallel particles processing for MODPATH using the OpenMP library.

Publisher Correction: Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systems

Publisher Correction: Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systems

Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systems

Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systems

Improvement and comparison the performance of fuzzing testing algorithms for applications in Google Thread Sanitizer

It is difficult to imagine modern information systems without the use of multithreading. The use of multithreading can both improve the performance of the system as in whole so as slow down the execution of multithreaded applications due to the occurrence of multithreaded programming errors. To find such errors in C/C++ languages, there exists a Google Thread Sanitizer compiler module. The order of execution of threads can change every time the program is started for execution and can affect the appearance of such errors. To repeatedly change the order of execution of threads during the execution of the program, Google Thread Sanitizer has a fuzzing testing module that allows you to increase the probability of finding errors. But all the thread scheduling algorithms in this module are presented in the form of sequential execution of threads which can lead to a significant slowdown in Google Thread Sanitizer as well as can affect the testing of applications that depends on timers (waiting for network events, deadline for operations, ...). To speed up the work of fuzzing schedulers, a method for parallelizing independent transitions is proposed. From the point of view of multithreaded programming errors, it is only important to change the shared state between threads, and local calculations do not affect the reproduction of multithreaded errors. The changes of shared states themselves occur at synchronization points (places in the code where threads are switched according to the principle of cooperative multitasking). The method suggests ordering only the change of shared states at synchronization points, and performing local calculations in parallel, due to which parallelization is achieved. For the analysis of theoretical complexity of the algorithm, the method of combinatorial counting is used. A new approach to the organization of fuzzing testing based on the method of parallelization of independent transitions is proposed the implementation of which, according to theoretical and practical estimates, shows a noticeable acceleration of the work of fuzzing schedulers. According to the results of the experiment, it was revealed that for the algorithm of iterating through all execution variants, the acceleration of execution reaches 1.25 times for two threads. For an arbitrary number of threads, an estimate is presented in the form of a formula. The proposed approach allows fuzzing tests to cover multithreaded applications for which execution time is important — applications with reference to timers which improve the quality of the software.