Thread Allocation Research Articles

Development of GPU‐based matrix‐free strategies for large‐scale elastoplasticity analysis using conjugate gradient solver

AbstractIn recent years, matrix‐free conjugate gradient (CG) solvers with graphics processing unit (GPU) acceleration have been effectively used to reduce the execution timings of finite element method (FEM)‐based engineering simulations. However, there is not much in the literature that discusses the application of matrix‐free CG solvers for elastoplasticity. The primary challenge is the presence of both elastic and plastic states, which introduce branching issues in the parallel computation of sparse matrix‐vector multiplication (SpMV). The current work proposes an efficient split kernel strategy for elastoplasticity that segregates elements in elastic and plastic zones and allows the application of standard GPU‐based matrix‐free SpMV strategies in the literature to elastoplastic simulation. The proposed strategy avoids branching issues, facilitates coalesced memory access, allows efficient usage of on‐chip memory, and uses minimum storage to realize large‐scale simulation on a GPU with limited memory. We also propose matrix‐free SpMV strategies for GPU implementation based on an element‐by‐element technique that makes efficient use of memory hierarchy available in a GPU. The computational efficiency of the proposed strategies is demonstrated over three large‐scale benchmark examples from elastoplasticity, and the performance results are compared with GPU optimized node‐based and degrees of freedom (DOF)‐based matrix‐free strategies. The results show that the splitting of computation is most suitable for low plasticity problems, where most of the matrix‐free strategies show significant speedups with respect to single kernel strategy for elastoplasticity. The proposed element‐by‐element strategy using node‐wise thread allocation is found to have the best performance, achieving up to 7.16 and 3.35 speedup over node‐based and DOF‐based strategy, respectively. For problems with higher amounts of plasticity, the proposed strategies perform slightly better and achieve modest speedups due to advantages in the initial stages of Newton iterations. In terms of wall‐clock timings, the proposed matrix‐free solver is found to be up to 2 faster than GPU optimized assembly‐based elastoplasticity solver.

TLP Balancer: Predictive Thread Allocation for Multi-Tenant Inference in Embedded GPUs

TLP Balancer: Predictive Thread Allocation for Multi-Tenant Inference in Embedded GPUs

NIOT: A Novel Inference Optimization of Transformers on Modern CPUs

In the machine learning era, model inference efficiency is one of the most important issues for machine learning systems. It is a major challenge to find the optimal configuration in a huge search space as the combinations of kernel fusion, memory tiling, and thread allocation strategies result in highly variable and unpredictable inference performance. The problem is particularly pronounced in models with large parameter matrices such as Transformers. In this paper, we aim to develop a general and powerful framework for inference optimization, called NIOT, to achieve desirable efficiency for the prevailing Transformer-like models on CPUs. To take full advantage of modern CPU features such as SIMD and cache hierarchy, NIOT employs various methods to provide promising strategies tailored to the target Transformer model. Our C++ implementation of NIOT shows significant performance improvements over popular well-optimized model-serving runtimes such as PyTorch and ONNXRuntime.

Neuro-semantic prediction of user decisions to contribute content to online social networks

Understanding at microscopic level the generation of contents in an online social network (OSN) is highly desirable for an improved management of the OSN and the prevention of undesirable phenomena, such as online harassment. Content generation, i.e., the decision to post a contributed content in the OSN, can be modeled by neurophysiological approaches on the basis of unbiased semantic analysis of the contents already published in the OSN. This paper proposes a neuro-semantic model composed of (1) an extended leaky competing accumulator (ELCA) as the neural architecture implementing the user concurrent decision process to generate content in a conversation thread of a virtual community of practice, and (2) a semantic modeling based on the topic analysis carried out by a latent Dirichlet allocation (LDA) of both users and conversation threads. We use the similarity between the user and thread semantic representations to built up the model of the interest of the user in the thread contents as the stimulus to contribute content in the thread. The semantic interest of users in discussion threads are the external inputs for the ELCA, i.e., the external value assigned to each choice.. We demonstrate the approach on a dataset extracted from a real life web forum devoted to fans of tinkering with musical instruments and related devices. The neuro-semantic model achieves high performance predicting the content posting decisions (average F score 0.61) improving greatly over well known machine learning approaches, namely random forest and support vector machines (average F scores 0.19 and 0.21).

A Hybrid MPI/OpenMP Parallelization Scheme Based on Nested FDTD for Parametric Decay Instability

Parametric decay instability (PDI) generated in milliseconds is an important physical phenomenon in ionospheric heating. Usually, numerical simulations are used to study PDI mechanisms. They can intuitively investigate the generation and development process of PDI, which is necessary in experimental studies. When simulating the PDI phenomenon through the explicit finite-difference time-domain (FDTD), the spatial scale spans from kilometers to centimeters, and the time scale needs to meet the Courant–Friedrichs–Lewy condition. Simulating the PDI phenomenon is time-consuming and difficult due to the high spatial resolution and strict restriction on the discrete time step. Although a nested mesh technique can boost the computational efficiency, the application of a parallel strategy is imperative to further improve it. In this study, we present a hybrid Message Passing Interface (MPI)/OpenMP parallelization scheme to solve the above-mentioned problems. This scheme can achieve an adaptive calculation and automatic allocation of MPI tasks and OpenMP threads, proving its flexibility and portability. Under the EISCAT background parameters, the PDI phenomenon was simulated. The results of the wave mode conversion and intense localized turbulence were identical to those of the serial program. Furthermore, a new simulation example and the effect of the cavity depth on electrostatic waves and negative ion density cavity were investigated. By utilizing the proposed parallelization scheme, the simulation time can be reduced from 70 h for the serial program to 3.6 h.

Parallel Louvain Community Detection Algorithm Based on Dynamic Thread Assignment on Graphic Processing Unit

Background and Objectives: Louvain is a time-consuming community detection algorithm especially in large-scale networks. Using Graphic Processing Unit (GPU) in order to calculate modularity sigma, which is a major processing section in Louvain algorithm, can reduce algorithm execution time and make it practical for large-scale networks.Methods: The proposed algorithm Dynamic CUDA Louvain Method (DCLM) blocks hardware threads dynamically on cores inside GPU. By considering the properties of GPU, this algorithm allocates the maximal number of processing cores to each Stream Multi-Processor (SM) as number of threads in a block. If the number of nodes in the graph is smaller than all physical cores on GPU, number of threads per block Is equal to the ratio number of graph nodes over the number of SMs.Results: The implementation results demonstrated that the proposed algorithm is able to decrease the run time by 15% in comparison with the best past method in the large-scale graph.Conclusion: We have introduced DCLM algorithm based on GPU that accelerates Louvain community detection algorithm. Dynamic allocation of threads to each block has a significant effect on the reduction of algorithm execution time. However, incrementing the number of threads per block alone does not result to acceleration the speed of calculations.

Participation Maximization Based on Social Influence in Online Discussion Forums

In online discussion forums, users are more motivated to take part in discussions when observing other users’ participation—the effect of social influence among forum users. In this paper, we study how to utilize social influence for increasing the overall forum participation. To this end, we propose a mechanism to maximize user influence and boost participation by displaying forum threads to users. We formally define the participation maximization problem, and show that it is a special instance of the social welfare maximization problem with submodular utility functions and it is NP-hard. However, generic approximation algorithms is impracticable for real-world forums due to time complexity. Thus we design a heuristic algorithm, named Thread Allocation Based on Influence (TABI), to tackle the problem. Through extensive experiments using a dataset from a real-world online forum, we demonstrate that TABI consistently outperforms all other algorithms in maximizing participation. The results of this work demonstrates that current recommender systems can be made more effective by considering future influence propagations. The problem of participation maximization based on influence also opens a new direction in the study of social influence.

Multithreaded Acceleration of 3D Mathematical Model for Ore Sintering

One of the widely used methods to accelerate a numerical solver is implementation of multithreading. The problem of thread allocation on-demand at runtime is latency, caused by periodical instantiation of threads. The article is devoted to parallelization of solver for 3D mathematical model of ore sintering, based on software threads reusing them during computation. Computational domain is equally shared among available threads. Each thread writes only to own data partition. A looped barrier is proposed for guaranteed synchronization of all threads after iteration. The method allows scaling performance without recompilation of the solver by using similar CPU with more cores. Measurement of solver performance with 220 nodes using different thread count confirms scalability around 95% for double and single precision arithmetics. Presented pictures of perspective view with three slices of temperature field show influence of heat loss from pallets walls. A cross section of temperature field in layer after 16 minutes of sintering is calculated with appearance of two high-temperature regions inside. Comparison of temperature field with literature data gives good correspondence. The computer model takes into account important chemical reactions, such as, coke burning, carbonate dissolution, water vaporization, as well as mass-heat transfer inside the sinter layer and can be used in metallurgical plants to increase effectiveness of sintering.

Implementation and optimization of GPU‐based parallel one‐step leapfrog ADI‐FDTD for far‐field scattering problems

The one-step leapfrog alternative-direction-implicit finite-difference time-domain (ADI-FDTD), free from the Courant-Friedrichs-Lewy (CFL) stability condition and sub-step computations, is efficient when dealing with fine grid problems. However, solution of the numerous tridiagonal systems still imposes a great computational burden and makes the method hard to execute in parallel. In this paper, we proposed an efficient graphic processing unit (GPU)-based parallel implementation of the one-step leapfrog ADI-FDTD for the far-field EM scattering simulation of objects, in which we present and analyze the manners of calculation area division and thread allocation and a data layout transformation of z components is proposed to achieve better memory access mode, which is a key factor affecting GPU execution efficiency. The simulation experiment is carried out to verify the accuracy and efficiency of the GPU-based implementation. The simulation results show that there is a good agreement between the proposed one-step leapfrog ADI-FDTD method and Yee's FDTD in solving the far-field scattering problem and huge benefits in performance were encountered when the method was accelerated using GPU technology.

Fast Non-Local Adaptive In-Loop Filter Optimization on GPU

The non-local adaptive in-loop filter (NALF) for video coding has achieved significant coding gain by exploiting image non-local self-similarity (NSS) to efficiently reduce the compression artifacts. However, the intensive computation of NALF hinders its practical deployment in video standardizations. In this paper, we propose a fast NALF optimization algorithm in parallel-computing framework by leveraging the massive parallel execution resources of GPU. First, the computational complexity of original NALF is analyzed in depth, then the pipelines of computational-intensive modules are re-designed to adapt to the general-purpose GPU with more parallel-friendly consideration. Specifically, we speed up the NALF by optimizing thread allocation to maximize the parallelism degree and elaborately designing the GPU block dimension to avoid access conflict. The group-level and pixel-level parallelization for collaboratively filtering and patch matching modules are designed respectively. To reduce the cost in data transmission, the whole filtering process is implemented on GPU by taking the advantage of low data dependency in NALF. Extensive experimental results show that the proposed fast NALF optimization using GPU architecture achieves high-speeed processing while maintaining the significant coding performance of original NALF, which shows the potential of NALF in the future video coding standard.

Co-Processing Parallel Computation for Distributed Optical Fiber Vibration Sensing

Rapid data processing is crucial for distributed optical fiber vibration sensing systems based on a phase-sensitive optical time domain reflectometer (Φ-OTDR) due to the huge amount of continuously refreshed sensing data. The vibration sensing principle is analyzed to study the data flow of Rayleigh backscattered light among the different processing units. A field-programmable gate array (FPGA) is first chosen to synchronously implement pulse modulation, data acquisition and transmission in parallel. Due to the parallelism characteristics of numerous independent algorithm kernels, graphics processing units (GPU) can be used to execute the same computation instruction by the allocation of multiple threads. As a conventional data processing method for the sensing system, a differential accumulation algorithm using co-processing parallel computation is verified with a time of 1.6 μs spent of the GPU, which is 21,250 times faster than a central processing unit (CPU) for a 2020 m length of optical fiber. Moreover, the cooperation processes of the CPU and GPU are realized for the spectrum analysis, which could shorten substantially the time of fast Fourier transform analysis processing. The combination of FPGA, CPU and GPU can largely enhance the capacity of data acquisition and processing, and improve the real-time performance of distributed optical fiber vibration sensing systems.

Parallelized combined finite‐discrete element (FDEM) procedure using multi‐GPU with CUDA

SummaryThis paper focuses on the efficiency of finite discrete element method (FDEM) algorithmic procedures in massive computers and analyzes the time‐consuming part of contact detection and interaction computations in the numerical solution. A detailed operable GPU parallel procedure was designed for the element node force calculation, contact detection, and contact interaction with thread allocation and data access based on the CUDA computing. The emphasis is on the parallel optimization of time‐consuming contact detection based on load balance and GPU architecture. A CUDA FDEM parallel program was developed with the overall speedup ratio over 53 times after the fracture from the efficiency and fidelity performance test of models of in situ stress, UCS, and BD simulations in Intel i7‐7700K CPU and the NVIDIA TITAN Z GPU. The CUDA FDEM parallel computing improves the computational efficiency significantly compared with the CPU‐based ones with the same reliability, providing conditions for achieving larger‐scale simulations of fracture.

An efficient sparse approximate inverse preconditioning algorithm on GPU

SummaryThe sparse approximate inverse (SPAI) preconditioner has proven to be effective in accelerating the convergence of iterative methods. Recently, accelerating it on the graphics processing unit (GPU) has attracted considerable attention due to the fact that the cost of constructing it is high. This motivates us to investigate how to accelerate the construction of SPAI preconditioners on GPU in this paper. We propose an efficient sparse approximate inverse algorithm on GPU, called SPAI‐Adaptive. For our proposed SPAI‐Adaptive, there are the following novelties: (1) an adaptive thread allocation strategy for SPAI‐Adaptive is proposed to assign the optimal thread number for each column of the preconditioner, and (2) Each component of the preconditioner, which includes finding indices I and J, constructing local submatrix, decomposing the local matrix into QR, and solving the upper triangular linear system, is computed in parallel inside a thread group of GPU. Experimental results show that the proposed SPAI‐Adaptive is effective, and has good performance and high parallelism.

View Priority Based Threads Allocation and Binary Search Oriented Reweight for GPU Accelerated Real-Time 3D Ball Tracking

View Priority Based Threads Allocation and Binary Search Oriented Reweight for GPU Accelerated Real-Time 3D Ball Tracking

Efficient dense matrix‐vector multiplication on GPU

SummaryGiven that the dense matrix‐vector multiplication (Ax or ATx) is of great importance in scientific computations, how to accelerate it is investigated on the graphics processing unit (GPU) in this paper. We present a warp‐based implementation of Ax on the GPU, called GEMV‐Adaptive, and a thread‐based implementation of ATx on the GPU, called GEMV‐T‐Adaptive. For our proposed GEMV‐Adaptive and GEMV‐T‐Adaptive, there are the following novelties: (1) an adaptive warp allocation strategy for GEMV‐Adaptive is proposed to assign the optimal warp number for each matrix row, (2) an adaptive thread allocation strategy for GEMV‐T‐Adaptive is designed to assign the optimal thread number to each matrix row, and (3) several optimization schemes are formulated. Experimental results show that the proposed GEMV‐Adaptive and GEMV‐T‐Adaptive mitigate the performance fluctuations of the implementations in the CUBLAS library, always have high performance, and outperform the most recently proposed GEMV and GEMV‐T kernels by Gao et al, respectively, for all test matrices.

Model-driven scheduling for distributed stream processing systems

Distributed Stream Processing Systems (DSPS) are “Fast Data” platforms that allow streaming applications to be composed and executed with low latency on commodity clusters and Clouds. Such applications are composed as a Directed Acyclic Graph (DAG) of tasks, with data parallel execution using concurrent task threads on distributed resource slots. Scheduling such DAGs for DSPS has two parts—allocation of threads and resources for a DAG, and mapping threads to resources. Existing schedulers often address just one of these, make the assumption that performance linearly scales, or use ad hoc empirical tuning at runtime. Instead, we propose model-driven techniques for both mapping and allocation that rely on low-overhead a priori performance modeling of tasks. Our scheduling algorithms are able to offer predictable and low resource needs that is suitable for elastic pay-as-you-go Cloud resources, support a high input rate through high VM utilization, and can be combined with other mapping approaches as well. These are validated for micro and application benchmarks, and compared with contemporary schedulers, for the Apache Storm DSPS.

Analysis of the allocation of classes, threads and CPU used in embedded systems for Java applications

In order to analyze the volume of classes and threads allocated by Java applications, as well as the CPU used in embedded systems, we performed a comparative study to follow the behavior of several simple applications of different categories and in two VMs. different. After collecting data through monitoring tools, we found that with the VMZ more classes and threads are created, and more CPU is consumed, so more collections are made to free up memory space. This is due to the fact that VMZ is based on Oracle’s Open JDK, it takes a little more memory and CPU to load all the libraries used.

Efficient Thread Allocation Method of Convolutional Neural Network based on GPGPU

Efficient Thread Allocation Method of Convolutional Neural Network based on GPGPU

Adaptive Tuning of Photonic Devices in a Photonic NoC Through Dynamic Workload Allocation

Photonic network-on-chip (PNoC) is a promising candidate to replace traditional electrical NoC in manycore systems that require substantial bandwidths. The photonic links in the PNoC comprise laser sources, optical ring resonators, passive waveguides, and photodetectors. Reliable link operation requires laser sources and ring resonators to have matching optical frequencies. However, inherent thermal sensitivity of photonic devices and manufacturing process variations can lead to a frequency mismatch. To avoid this mismatch, micro-heaters are used for thermal trimming and tuning, which can dissipate a significant amount of power. This paper proposes a novel FreqAlign workload allocation policy, accompanying an adaptive frequency tuning ( AFT ) policy, that is capable of reducing thermal tuning power of PNoC. FreqAlign uses thread allocation and thread migration to control temperature for matching the optical frequencies of ring resonators in each photonic link. The AFT policy reduces the remaining optical frequency difference among ring resonators and corresponding on-chip laser sources by hardware tuning methods. We use a full modeling stack of a PNoC that includes a performance simulator, a power simulator, and a thermal simulator with a temperature-dependent laser source power model to design and evaluate our proposed policies. Our experimental results demonstrate that FreqAlign reduces the resonant frequency gradient between ring resonators by 50%–60% when compared to existing workload allocation policies. Coupled with AFT , FreqAlign reduces localized thermal tuning power by 19.28 W on average, and is capable of saving up to 34.57 W when running realistic loads in a 256-core system without any performance degradation.

Explicit Fourth-Order Runge\u2013Kutta Method on Intel Xeon Phi Coprocessor

This paper concerns an Intel Xeon Phi implementation of the explicit fourth-order Runge–Kutta method (RK4) for very sparse matrices with very short rows. Such matrices arise during Markovian modeling of computer and telecommunication networks. In this work an implementation based on Intel Math Kernel Library (Intel MKL) routines and the authors’ own implementation, both using the CSR storage scheme and working on Intel Xeon Phi, were investigated. The implementation based on the Intel MKL library uses the high-performance BLAS and Sparse BLAS routines. In our application we focus on OpenMP style programming. We implement SpMV operation and vector addition using the basic optimizing techniques and the vectorization. We evaluate our approach in native and offload modes for various number of cores and thread allocation affinities. Both implementations (based on Intel MKL and made by the authors) were compared in respect of the time, the speedup and the performance. The numerical experiments on Intel Xeon Phi show that the performance of authors’ implementation is very promising and gives a gain of up to two times compared to the multithreaded implementation (based on Intel MKL) running on CPU (Intel Xeon processor) and even three times in comparison with the application which uses Intel MKL on Intel Xeon Phi.