Thread Assignment Research Articles

Revisiting thread configuration of SpMV kernels on GPU: A machine learning based approach

Sparse matrix-vector multiplication (SpMV) optimization on GPUs has been challenging due to irregular memory accesses and unbalanced workloads. The majority of existing solutions assign a fixed number of threads to one or more rows of sparse matrices according to empirical formulas. However, this method does not give the optimal thread configuration and results in a significant performance loss. This paper proposes a new machine learning-based thread assignment strategy for SpMV on GPU, predicting the near-optimal thread configuration for matrices. Further, we partition irregular sparse matrices into blocks according to the distribution of non-zero elements and predict the optimal thread configuration for each block. A new SpMV kernel is designed to accelerate the execution of different blocks. Experimental results show that our machine learning-based approach can select the near-optimal thread configuration for most matrices. The efficiency of SpMV for irregular matrices is also improved by matrix partitioning and blockwise prediction. Finally, we dive into the trained model to find out the connection between the features of a sparse matrix and its optimal thread configuration.

Parallel Overlapping Community Detection Algorithm on GPU

Community detection is one of the most representative graph mining applications, which is often assembled as a concurrent graph partition application to explore the maximum modularity (or gained modularity) of each community. However, many branch divergence operations create significant obstacles to unleashing GPU&#x2019;s high throughput and memory bandwidth, which are needed in community detection applications to divide the vertices into different communities. In this paper, we present Lugger, a GPU-based overlapping community detection algorithm that reduces GPU&#x2019;s branch divergence via the customer-designed cache-aware parallel searching technique. In Lugger, we first design a cache-aware parallel searching policy using the B-Tree structure. Then, we set the B-Tree node matches with the GPU cache line to meet the coalesced memory access manner and avoid the branch divergence in warps. Moreover, we design a positive node splitting scheme to reduce the lock operation and idle threads when building the B-Tree structure. In addition, we implement a warp-centric thread assignment strategy to make sure the workloads across threads are balanced. We implement the proposed algorithm on NVIDIA GPU and evaluate the performance on eight large graphs (up to <inline-formula><tex-math notation="LaTeX">$3M$</tex-math></inline-formula> vertices and <inline-formula><tex-math notation="LaTeX">$117M$</tex-math></inline-formula> edges) with ground-truth communities. The experimental results show that Lugger can outperform the state-of-the-art works on scalability and detection quality.

A Novel Mutual Insurance Model for Hedging Against Cyber Risks in Power Systems Deploying Smart Technologies

In this paper, a novel cyber-insurance model design is proposed based on system risk evaluation with smart technology applications. The cyber insurance policy for power systems is tailored via cyber risk modeling, reliability impact analysis, and insurance premium calculation. A stochastic Epidemic Network Model is developed to evaluate the cyber risk by propagating cyberattacks among graphical vulnerabilities. Smart technologies deployed in risk modeling include smart monitoring and job thread assignment. Smart monitoring boosts the substation availability against cyberattacks with preventive and corrective measures. The job thread assignment solution reduces the execution failures by distributing the control and monitoring tasks to multiple threads. Reliability assessment is deployed to estimate load losses convertible to monetary losses. These monetary losses would be shared through a mutual insurance plan. To ensure a fair distribution of indemnity, a new Shapley mutual insurance principle is devised. Effectiveness of the proposed Shapley mutual insurance design is validated via case studies. The Shapley premium is compared with existent premium designs. It is shown that the Shapley premium has high indemnity levels closer to those of Tail Conditional Expectation premium. Meanwhile, the Shapley premium is nearly as affordable as the coalitional premium and keeps a relatively low insolvency probability.

Dynamic thread mapping for power-efficient many-core systems under performance constraints

Efficiently mapping threads to system cores is critical for achieving high performance and power efficiency in multicore systems. Conventional scheduling techniques do not take system heterogeneity and varying voltage/frequency states into account and therefore do not produce efficient mappings. Additionally, since thread behavior varies throughout its execution, a fixed thread-to-core mapping is sub-optimal. Dynamically mapping threads to appropriate core states can significantly improve system throughput and decrease energy consumption. Mapping decisions are of utmost importance in systems that have performance constraints to satisfy. The problem of thread assignment in order to meet performance or power constraints has been shown to be NP-complete and therefore exceedingly expensive to solve online. This paper proposes a heuristic method for dynamically assigning threads to system cores such that performance requirements are met while minimizing the energy consumed. This method can be applied to systems consisting of heterogeneous cores, and systems with dynamic voltage/frequency scaling capabilities. Simulation results show that the proposed heuristic can be used online in hundred-core systems, with runtime overhead of less than 1 millisecond for 256-core systems.

Utility Optimal Thread Assignment and Resource Allocation in Multi-Server Systems

Achieving high performance in many multi-server systems (e.g., web hosting center, cloud) requires finding a good assignment of worker threads to servers and also effectively allocating each server’s resources to its assigned threads. The assignment and allocation components of this problem have been studied extensively but largely separately in the literature. In this paper, we introduce the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">assign and allocate (AA)</i> problem, which seeks to simultaneously find an assignment and allocation that maximizes the total utility of the threads. Assigning and allocating the threads together can result in substantially better overall utility than performing the steps separately, as is traditionally done. We model each thread by a utility function giving its performance as a function of its assigned resources. We first prove that the AA problem is NP-hard. We then present a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 (\sqrt {2}-1) &gt; 0.828$ </tex-math></inline-formula> factor approximation algorithm for concave utility functions, which runs in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(mn^{2} + n (\log mC)^{2})$ </tex-math></inline-formula> time for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> threads and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> servers with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$C$ </tex-math></inline-formula> amount of resources each. We also give a faster algorithm with the same approximation ratio and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(n (\log mC)^{2})$ </tex-math></inline-formula> time complexity. We then extend the problem to two more general settings. First, we consider threads with nonconcave utility functions, and give a 1/2 factor approximation algorithm. Next, we give an algorithm for threads using multiple types of resources, and show the algorithm achieves good empirical performance. We conduct extensive experiments to test the performance of our algorithms on threads with both synthetic and realistic utility functions, and find that they achieve over 92% of the optimal utility on average. We also compare our algorithms with a number of practical heuristics, and find that our algorithms achieve up to 9 times higher total utility.

Parallel Louvain Community Detection Algorithm Based on Dynamic Thread Assignment on Graphic Processing Unit

Background and Objectives: Louvain is a time-consuming community detection algorithm especially in large-scale networks. Using Graphic Processing Unit (GPU) in order to calculate modularity sigma, which is a major processing section in Louvain algorithm, can reduce algorithm execution time and make it practical for large-scale networks.Methods: The proposed algorithm Dynamic CUDA Louvain Method (DCLM) blocks hardware threads dynamically on cores inside GPU. By considering the properties of GPU, this algorithm allocates the maximal number of processing cores to each Stream Multi-Processor (SM) as number of threads in a block. If the number of nodes in the graph is smaller than all physical cores on GPU, number of threads per block Is equal to the ratio number of graph nodes over the number of SMs.Results: The implementation results demonstrated that the proposed algorithm is able to decrease the run time by 15% in comparison with the best past method in the large-scale graph.Conclusion: We have introduced DCLM algorithm based on GPU that accelerates Louvain community detection algorithm. Dynamic allocation of threads to each block has a significant effect on the reduction of algorithm execution time. However, incrementing the number of threads per block alone does not result to acceleration the speed of calculations.

A DFT-Based Running Time Prediction Algorithm for Web Queries

Web search engines are built from components capable of processing large amounts of user queries per second in a distributed way. Among them, the index service computes the top-k documents that best match each incoming query by means of a document ranking operation. To achieve high performance, dynamic pruning techniques such as the WAND and BM-WAND algorithms are used to avoid fully processing all of the documents related to a query during the ranking operation. Additionally, the index service distributes the ranking operations among clusters of processors wherein in each processor multi-threading is applied to speed up query solution. In this scenario, a query running time prediction algorithm has practical applications in the efficient assignment of processors and threads to incoming queries. We propose a prediction algorithm for the WAND and BM-WAND algorithms. We experimentally show that our proposal is able to achieve accurate prediction results while significantly reducing execution time and memory consumption as compared against an alternative prediction algorithm. Our proposal applies the discrete Fourier transform (DFT) to represent key features affecting query running time whereas the resulting vectors are used to train a feed-forward neural network with back-propagation.

Efficiency in Parallel Computing of FDS Model for Compartment Fire Simulation: Shared Memory System

This study evaluates the computational efficiency based on the parallel processing mode and domain decomposition method of the FDS model to enhance the computational performance of fire simulation. A single compartment of dimensions 12.0 m × 3.8 m × 3.0 m is considered along with a rectangular fire source (0.4 m × 0.4 m) fueled by n-Heptane. The computational domain was divided into 136,000 cells forming a grid size of 0.1 m, and the computational efficiency for each calculation was evaluated by the wall clock time for a simulation time of 300 s using a computational framework with 24 cores of a single CPU and a 256 GB shared memory system. The MPI and hybrid mode in FDS parallel offers a greater speed-up capability than the OpenMP mode, and the domain decomposition method used greatly affects the computational efficiency. The maximum speed-up with the OpenMP mode was less than 1.5 for a single computational domain, which indicates that there is an optimal condition for thread assignment and domain decomposition in the OpenMP mode. The present study is expected to contribute toward obtaining effective fire simulation results with limited computing power and time in fire protection engineering.

Bulk execution of the dynamic programming for the optimal polygon triangulation problem on the GPU

SummaryThe bulk execution is to execute some computation for many different inputs in turn or at the same time. The main contribution of this paper is to propose a parallel processing technique for the bulk execution of the dynamic programming using the GPU (Graphics Processing Unit). Especially, we focus on the optimal polygon triangulation problem for a lot of polygons. We consider programming issues of the GPU architecture such as coalesced memory access of the global memory, warp divergence avoidance, and reduction of CUDA kernel calls. In the GPU implementation, we propose two thread assignment methods that efficiently perform the parallel execution with a lot of threads on thousands of cores in the GPU. The experimental results show that our GPU implementation on NVIDIA TITAN V attains a speed‐up factor of up to 106.05 and 26.78 over the single‐thread and 8‐thread CPU implementations on Intel Core i7‐6700K CPU, respectively.

Adaptive parallel Louvain community detection on a multicore platform

Community detection is a demanded technique in analyzing complex and massive graph-based networks. The quality of the detected communities in an acceptable time is an important aspect of an algorithm, which aims at passing through an ultra large scale graph, for instance a social network graph. In this paper, an efficient method is proposed to tackle Louvain community detection problem on multicore systems in the line of thread-level parallelization. The main contribution of this article is to present an adaptive parallel thread assignment for the calculation of adding qualified neighbor nodes to the community. This leads to obtain a better load balancing method for the execution of threads. The proposed method is evaluated on an AMD system with 64 cores, and can reduce the execution time by 50% in comparison with the previous fastest parallel algorithms. Moreover, it was observed in the course of the experiments that our method could find comparably qualified communities.

Hybrid CPU-GPU scheduling and execution of tree traversals

GPUs offer the promise of massive, power-efficient parallelism. However, exploiting this parallelism requires code to be carefully structured to deal with the limitations of the SIMT execution model. In recent years, there has been much interest in mapping irregular applications to GPUs: applications with unpredictable, data-dependent behaviors. While most of the work in this space has focused on ad hoc implementations of specific algorithms, recent work has looked at generic techniques for mapping a large class of tree traversal algorithms to GPUs, through careful restructuring of the tree traversal algorithms to make them behave more regularly. Unfortunately, even this general approach for GPU execution of tree traversal algorithms is reliant on ad hoc , handwritten, algorithm-specific scheduling ( i.e. , assignment of threads to warps) to achieve high performance. The key challenge of scheduling is that it is a highly irregular process, that requires the inspection of thread behavior and then careful sorting of the threads into warps. In this paper, we present a novel scheduling and execution technique for tree traversal algorithms that is both general and automatic. The key novelty is a hybrid approach: the GPU partially executes tasks to inspect thread behavior and transmits information back to the CPU, which uses that information to perform the scheduling itself, before executing the remaining, carefully scheduled, portion of the traversals on the GPU. We applied this framework to five tree traversal algorithms, achieving significant speedups over optimized GPU code that does not perform application-specific scheduling. Further, we show that in many cases, our hybrid approach is able to deliver better performance even than GPU code that uses hand-tuned, application-specific scheduling.

Thread Assignment in Multicore/Multithreaded Processors: A Statistical Approach

The introduction of multicore/multithreaded processors, comprised of a large number of hardware contexts (virtual CPUs) that share resources at multiple levels, has made process scheduling, in particular assignment of running threads to available hardware contexts, an important aspect of system performance. Nevertheless, thread assignment of applications running on state-of-the art processors is an NP-complete problem. Over the years, numerous studies have proposed heuristic-based algorithms for thread assignment. Since the thread assignment problem is intractable, it is in general impossible to know the performance of the optimal assignment, so the room for improvement of a given algorithm is also unknown. It is therefore hard to decide whether to invest more effort and time to improve an algorithm that may already be close to optimal. In this paper, we present a statistical approach to the thread assignment problem. First, we present a method that predicts the performance of the optimal thread assignment, based on the observed performance of each thread assignment in a random sample. The method is based on Extreme Value Theory (EVT), a branch of statistics that analyses extreme deviations from the population mean. We also propose sample pruning, a method that significantly reduces the time required to apply the statistical method by reducing the number of candidate solutions that need to be measured. Finally, we show that, if no suitable heuristic-based algorithm is available, a sample of several thousand random thread assignments is enough to obtain, with high confidence, an assignment with performance close to optimal. The presented approach is architecture and application independent, and it can be used to address the thread assignment problem in various domains. It is especially well suited for systems in which the workload seldom changes. An example is network systems, which typically provide a constant set of services that are known in advance, with network applications performing a similar processing algorithm for each packet in the system. In this paper, we validate our methods with an industrial case study for a set of multithreaded network applications on an UltraSPARC T2 processor. This article is an extension of our previous work [44] , which was published in Proceedings of 17th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-2012).

Tumbler

Schedulers used by modern OSs (e.g., Oracle Solaris 11™ and GNU/Linux) balance load by balancing the number of threads in run queues of different cores. While this approach is effective for a single CPU multicore system, we show that it can lead to a significant load imbalance across CPUs of a multi-CPU multicore system. Because different threads of a multithreaded application often exhibit different levels of CPU utilization, load cannot be measured in terms of the number of threads alone. We propose Tumbler that migrates the threads of a multithreaded program across multiple CPUs to balance the load across the CPUs. While Tumbler distributes the threads equally across the CPUs, its assignment of threads to CPUs is aimed at minimizing the variation in utilization of different CPUs to achieve load balance. We evaluated Tumbler using a wide variety of 35 multithreaded applications, and our experimental results show that Tumbler outperforms both Oracle Solaris 11™ and GNU/Linux.

Energy-Efficient Thread Assignment Optimization for Heterogeneous Multicore Systems

The current trend to move from homogeneous to heterogeneous multicore systems provides compelling opportunities for achieving performance and energy efficiency goals. Running multiple threads in multicore systems poses challenges on meeting limited shared resources, such as memory bandwidth. We propose an optimization approach that includes an Integer Linear Programming (ILP) optimization model and a scheme to dynamically determine thread-to-core assignment. We present simulation analysis that shows energy savings and performance gains for a variety of workloads compared to state-of-the-art schemes. We implemented and evaluated a prototype of our thread assignment approach at user level, leveraging Linux scheduling and performance-monitoring capabilities.

Thread Assignment of Multithreaded Network Applications in Multicore/Multithreaded Processors

The introduction of multithreaded processors comprised of a large number of cores with many shared resources makes thread scheduling, and in particular optimal assignment of running threads to processor hardware contexts to become one of the most promising ways to improve the system performance. However, finding optimal thread assignments for workloads running in state-of-the-art multicore/multithreaded processors is an NP-complete problem. In this paper, we propose BlackBox scheduler, a systematic method for thread assignment of multithreaded network applications running on multicore/multithreaded processors. The method requires minimum information about the target processor architecture and no data about the hardware requirements of the applications under study. The proposed method is evaluated with an industrial case study for a set of multithreaded network applications running on the UltraSPARC T2 processor. In most of the experiments, the proposed thread assignment method detected the best actual thread assignment in the evaluation sample. The method improved the system performance from 5 to 48 percent with respect to load balancing algorithms used in state-of-the-art OSs, and up to 60 percent with respect to a naive thread assignment.

Data rate based adaptive thread assignment solution for combating the SlowPOST denial of service attack

Denial of Service (DoS) attacks represent a major threat to network security, especially in today's networked world. There has been significant research in this area, primarily focused on mitigating and preventing DoS attacks affecting transport layer services. This paper addresses issues arising from a new variation of a DoS attack, namely the SlowPOST attack that affects Application Layer services. In SlowPOST, the malicious clients send data at a slow rate after the connection is established, and the server is left waiting for the data to arrive. These attacks are particularly devastating due to their ability to resist detection due to their protocol compliance. In addition, such attacks do not require the massive resources that DoS attacks normally require, making them easier to launch. Some solutions for this issue have already been deployed in some commercial servers. These solutions are based on either monitoring traffic or enforcing a time limit on the transmission of the protocol headers. In order to achieve reliable detection, the detection parameters need to adapt to the constantly changing traffic. This paper proposes a novel algorithm that uses the data rate of connections to evolve a threshold for determining potential attackers in SlowPOST. This proposed method is tested by subjecting a server to an attack, and it was observed that in the absence of this method, the servicing of legitimate requests is not completed.

Auto-Tuning of Thread Assignment for Matrix-Vector Multiplication on GPUs

Auto-Tuning of Thread Assignment for Matrix-Vector Multiplication on GPUs

Optimal task assignment in multithreaded processors

The introduction of massively multithreaded (MMT) processors, comprised of a large number of cores with many shared resources, has made task scheduling, in particular task to hardware thread assignment, one of the most promising ways to improve system performance. However, finding an optimal task assignment for a workload running on MMT processors is an NP-complete problem. Due to the fact that the performance of the best possible task assignment is unknown, the room for improvement of current task-assignment algorithms cannot be determined. This is a major problem for the industry because it could lead to: (1)~A waste of resources if excessive effort is devoted to improving a task assignment algorithm that already provides a performance that is close to the optimal one, or (2)~significant performance loss if insufficient effort is devoted to improving poorly-performing task assignment algorithms. In this paper, we present a method based on Extreme Value Theory that allows the prediction of the performance of the optimal task assignment in MMT processors. We further show that executing a sample of several hundred or several thousand random task assignments is enough to obtain, with very high confidence, an assignment with a performance that is close to the optimal one. We validate our method with an industrial case study for a set of multithreaded network applications running on an UltraSPARC~T2 processor.

Optimal task assignment in multithreaded processors

The introduction of massively multithreaded (MMT) processors, comprised of a large number of cores with many shared resources, has made task scheduling, in particular task to hardware thread assignment, one of the most promising ways to improve system performance. However, finding an optimal task assignment for a workload running on MMT processors is an NP-complete problem. Due to the fact that the performance of the best possible task assignment is unknown, the room for improvement of current task-assignment algorithms cannot be determined. This is a major problem for the industry because it could lead to: (1)~A waste of resources if excessive effort is devoted to improving a task assignment algorithm that already provides a performance that is close to the optimal one, or (2)~significant performance loss if insufficient effort is devoted to improving poorly-performing task assignment algorithms. In this paper, we present a method based on Extreme Value Theory that allows the prediction of the performance of the optimal task assignment in MMT processors. We further show that executing a sample of several hundred or several thousand random task assignments is enough to obtain, with very high confidence, an assignment with a performance that is close to the optimal one. We validate our method with an industrial case study for a set of multithreaded network applications running on an UltraSPARC~T2 processor.

Task Scheduling Based On Thread Essence and Resource Limitations

Scheduling of threads based on the concept of thread essence is proposed in this paper. Multithreaded applications contain serial phases (single thread) and parallel phases (many threads). We propose a thread assignment mechanism that takes into account the essence of the threads in simultaneously-running applications that grants higher priority to applications during their critical-serial phases, for environments where there are more threads than cores. Furthermore, our proposed scheduler considers the limited resources of the system by reducing the number of context switches when there are more ready threads than cores. Analytic and experimental evaluation of the proposed thread assignment mechanism on both symmetric and emulated asymmetric multiprocessors show throughput improvements by as much as 16%, improved fairness by as much as 26% and reduced jitter by as much as 88%.