Thread Management Research Articles

Computing on many cores

SummaryThis paper presents an alternative method to parallelize programs, better suited to manycore processors than actual operating system–/API‐based approaches like OpenMP and MPI. The method relies on a parallelizing hardware and an adapted programming style. It frees and captures the instruction‐level parallelism (ILP). A many‐core design is presented in which cores are multithreaded and able to fork new threads. The programming style is based on functions. The hardware creates a concurrent thread at each function call. The programming style and the hardware create the conditions to free the ILP, by eliminating the architectural dependences between a call and its continuation after return. We illustrate the method on a sum reduction, a matrix multiplication and a sort. We measure the ILP of the parallel runs and show that it is high enough to feed thousands of cores because it increases with data size. We compare our method to pthread parallelization, showing that (1) our parallel execution is deterministic, (2) our thread management is cheap, (3) our parallelism is implicit, and (4) our method parallelizes functions and loops. Implicit parallelism makes parallel code easy to write and read. Deterministic parallel execution makes parallel code easy to debug.

Mth: Codesigned Hardware/Software Support for Fine Grain Threads

Multi-core processors are ubiquitous in all market segments from embedded to high performance computing, but only few applications can efficiently utilize them. Existing parallel frameworks aim to support thread-level parallelism in applications, but the imposed overhead prevents their usage for small problem instances. This work presents Micro-threads ( Mth ) a hardware-software proposal focused on a shared thread management model enabling the use of parallel resources in applications that have small chunks of parallel code or small problem inputs by a combination of software and hardware: delegation of the resource control to the application, an improved mechanism to store and fill processor's context, and an efficient synchronization system. Four sample applications are used to test our proposal: HSL filter (trivially parallel), FFT Radix2 (recursive algorithm), LU decomposition (barrier every cycle) and Dantzig algorithm (graph based, matrix manipulation). The results encourage the use of Mth and could smooth the use of multiple cores for applications that currently can not take advantage of the proliferation of the available parallel resources in each chip.

Exploiting Data-Parallelism on Multicore and SMT Systems for Implementing the Fractal Image Compressing Problem

This paper presents a parallel modeling of a lossy image compression method based on the fractal theory and its evaluation over two versions of dual-core processors: with and without simultaneous multithreading (SMT) support. The idea is to observe the speedup on both configurations when changing application parameters and the number of threads at operating system level. Our target application is particularly relevant in the Big Data era. Huge amounts of data often need to be sent over low/medium bandwidth networks, and/or to be saved on devices with limited store capacity, motivating efficient image compression. Especially, the fractal compression presents a CPU-bound coding method known for offering higher indexes of file reduction through highly time-consuming calculus. The structure of the problem allowed us to explore data-parallelism by implementing an embarrassingly parallel version of the algorithm. Despite its simplicity, our modeling is useful for fully exploiting and evaluating the considered architectures. When comparing performance in both processors, the results demonstrated that the SMT-based one presented gains up to 29%. Moreover, they emphasized that a large number of threads does not always represent a reduction in application time. In average, the results showed a curve in which a strong time reduction is achieved when working with 4 and 8 threads when evaluating pure and SMT dual-core processors, respectively. The trend concerns a slow growing of the execution time when enlarging the number of threads due to both task granularity and threads management.

A Novel Wavefront-Based High Parallel Solution for HEVC Encoding

With a lot of enhanced coding tools introduced, High Efficiency Video Coding (HEVC) achieves significant improvement in coding efficiency at the cost of increased computational complexity. To efficiently reduce the encoding time of HEVC, a wavefront-based high parallel (WHP) solution integrating novel data-level and task-level methods is proposed in this paper. On data level, optimal single-instruction-multiple-data algorithms are designed for the enhanced coding tools, i.e., replacing the multiplication in motion compensation by add and shift operations with reduced instruction cycles, removing the transpose in transform via realignment of coefficients, and minimizing the memory access in sum of absolute difference/sum of squared differences calculation by fully reusing the registers. On task level, a novel inter-frame wavefront (IFW) method is developed by effectively decreasing the dependence of wavefront parallel processing (WPP). In addition, a coding tree block level parallelism analysis method is presented to prove the superior of IFW method compared with other HEVC representative parallel methods. Besides, a three-level thread management scheme is proposed to best exploit the parallelism of IFW method and achieve corresponding encoding speedup. Extensive experimental results show that, the overall WHP solution can bring up to $57.65\times $ , $65.55\times $ , and $88.17\times $ speedup for HEVC encoding of Wide Video Graphics Array, 720p and 1080p standard test sequences, while maintaining the same coding performance as with WPP. The proposed solution is also applied in several leading video companies in China, providing HEVC video service for more than 1.3 million users everyday.

NestedMP: Enabling cache-aware thread mapping for nested parallel shared memory applications

Abstract It is beneficial to exploit multiple levels of parallelism for a wide range of applications, because a typical server already has tens of processor cores now. As the number of cores in a computer is increasing rapidly, efficient support of nested parallelism will be more and more important. We observe that different task-core mapping schemas may result significant performance difference because modern HPC servers are NUMA multi-core systems. So it is important to control the task-core mapping for nested parallelism. However, the number of threads management mechanism in current parallel programming models, such as OpenMP, does not provide enough information for runtime systems to make optimized decision. As a result, current nested parallel applications often suffer from suboptimal task-core mapping and get significant performance loss. To address this problem, we propose NestedMP, a set of directives which extends OpenMP. NestedMP specifies the number of threads of each nested parallel branch in a declarative way and allows runtime systems to see the whole picture of task trees to make locality-aware task-core mapping. We have implemented NestedMP in GCC 4.8.2 and tested the performance on a 4-way 8-core SandyBridge server. The result shows NestedMP improves the performance significantly over GCC’s OpenMP implementation.

Scheduling a Video Transcoding Server to Save Energy

Recent popular streaming services such as TV Everywhere, N-Screen, and dynamic adaptive streaming over HTTP (DASH) need to deliver content to the wide range of devices, requiring video content to be transcoded into different versions. Transcoding tasks require a lot of computation, and each task typically has its own real-time constraint. These make it difficult to manage transcoding, but the more efficient use of energy in servers is an imperative. We characterize transcoding workloads in terms of deadlines and computation times, and propose a new dynamic voltage and frequency scaling (DVFS) scheme that allocates a frequency and a workload to each CPU with the aim of minimizing power consumption while meeting all transcoding deadlines. This scheme has been simulated, and also implemented in a Linux transcoding server, in which a frontend node distributes transcoding requests to heterogeneous backend nodes. This required a new protocol for communication between nodes, a DVFS management scheme to reduce power consumption and thread management and scheduling schemes which ensure that transcoding deadlines are met. Power measurements show that this approach can reduce system-wide energy consumption by 17% to 31%, compared with the Linux Ondemand governor.

A resource utilization based instruction fetch policy for SMT processors

Simultaneous Multithreading (SMT) architectures are proposed to better explore on-chip parallelism, which capture the essence of performance improvement in modern processors. SMT overcomes the limits in a single thread by fetching and executing from multiple of them in a shared fashion. The long-latency operations, however, still cause inefficiency in SMT processors. When instructions have to wait for data from lower-level memory hierarchy, the dependent instructions cannot proceed, hence continue occupying the shared resources on the chip for an extended number of clock cycles. This introduces undesired inter-thread interference in SMT processors, which further leads to negative impacts on overall system throughput and average thread performance. In practice, instruction fetch policies take the responsibility of assigning thread priority at the fetch stage, in an effort to better distribute the shared resources among threads in the same core to cope with the long-latency operations and other runtime behavior from the thread for better performance.In this paper we propose an instruction fetch policy RUCOUNT, which considers resource utilization of individual thread in the prioritization process. The proposed policy observes instructions in the front-end stages of the pipeline as well as low-level data misses to summarize the resource utilization for thread management. Higher priority is granted to the thread(s) with less utilized resources, such that overall resources are distributed more efficiently in SMT processors. As a result, it has two unique features compared to other policies: one is to observe the hardware resource comprehensively and the other is to monitor limited resource entries. Our experimental results demonstrate that RUCOUNT is 20% better than ICOUNT, 10% than Stall, 8% than DG and 3% than DWarn, in terms of averaged performance. Considering its hardware overhead is at the similar level as ICOUNT and DWarn, our proposed instruction fetch policy RUCOUNT is superior among the studied policies.

Understanding energy behaviors of thread management constructs

Java programmers are faced with numerous choices in managing concurrent execution on multicore platforms. These choices often have different trade-offs (e.g., performance, scalability, and correctness guarantees). This paper analyzes an additional dimension, energy consumption. It presents an empirical study aiming to illuminate the relationship between the choices and settings of thread management constructs and energy consumption. We consider three important thread management constructs in concurrent programming: explicit thread creation, fixed-size thread pooling, and work stealing. We further shed light on the energy/performance trade-off of three ``tuning knobs'' of these constructs: the number of threads, the task division strategy, and the characteristics of processed data. Through an extensive experimental space exploration over real-world Java programs, we produce a list of findings about the energy behaviors of concurrent programs, which are not always obvious. The study serves as a first step toward improving energy efficiency of concurrent programs on parallel architectures.

Scheduling and thread management with RTEMS

The goal of a real-time operating system (RTOS) is to support real-time and embedded system (RT/ES) application development, which differ from general-purpose applications because of the size, weight, and power (SWaP) and timing constraints imposed by embedded applications. Useful RTOS features include real-time thread scheduling, thread communication, synchronization, interrupt handling, memory management, file systems, device drivers, networking, and debugging support. The Real-Time Executive for Multi-processor Systems (RTEMS) is a free and open-source RTOS that supports over a dozen processor architecture families and over 150 embedded system boards. RTEMS is designed to support embedded applications with stringent real-time requirements while being compatible with open standards such as POSIX. RTEMS includes optional services such as TCP/IP networking and file systems while still offering minimum executable sizes under 20 KB in useful configurations. One of the primary functions of an RTOS is to select threads that can obtain access to resources such as shared memory and processor time. RTEMS uses multiple algorithms to manage both waiting threads and those ready to execute. The thread execution schedulers include the traditional RTOS round robin and deterministic priority schedulers, rate monotonic, earliest deadline first (EDF), constant bandwidth server (CBS), and simple SMP scheduling algorithms. The RTEMS scheduling framework allows the application developer to select the thread scheduling algorithm that best meets the application's space and time requirements. ÂăWe will present how this framework can be used by researchers to integrate their own scheduling algorithm into RTEMS and test it using a scheduling simulator before deploying it on target hardware.

The Gamma-Ray Imaging Framework

The Gamma-Ray Imaging Framework (GRIF) is an open source (LGPL) software framework for creating real-time gamma-ray imaging applications. GRIF is written in C++ using Qt, and uses ROOT and the Boost Graph Library. GRIF provides automatic multi-threading and data management to make it easy to quickly develop power gamma-ray imagining applications. The model for application developers is built around the separation of data acquisition (DAQ) and analysis units. Users are expected to use the APIs for the DAQ and analysis units to build their applications. Memory and data are managed by GRIF, so the user does not need to worry about allocating and de-allocating memory for data or thread management and locking schemes. The user only needs to post data to and read data from the GRIF memory manager. GRIF uses XML configuration files for determining data dependences between DAQ and analysis units in the system. We will give an overview of the first release of GRIF, as well as show example applications that have been built using the framework.

GPGPU를 위한 쉐이더 명령어기반 멀티 스레드 관리 기법

본 논문에서는 모바일환경에 최적화 된 멀티 스레드 모바일 GPGPU를 설계하고, 멀티 스레드로 구성된 모바일 프로세서에서의 효과적인 스레드 관리 기법을 검증한다. 스레드의 제어에는 별도의 하드웨어 없이 소프트웨어 명령어를 기반으로 설계하였다. 스레드 관리 기법의 검증은 차선 검출 알고리즘을 구현하여 nVidia의 CUDA Architecture와 설계한 GPGPU의 스레드 관리 효율을 비교 분석한다. 스레드 효율에서는 CUDA와 비교했을 때 최대 2배까지 높은 효율을 보인다. This thesis is intended to design multi thread mobile GPGPU optimized in mobile environment, and to verify an effective thread management method of the multi thread mobile processor. In thread management, there is no management hardware and implement with software instructions. For the verification of the multi thread management method, Lane detection algorithm was implemented to compare nVidia's CUDA Architecture and the designed GPGPU in terms of thread management efficiency. The number of thread is normalized to 48 threads. An implemented Land Detection Algorithm is composed of Gaussian filter algorithm and Sobel Edge Detection algorithm. As a result, the designed GPGPU's thread efficiency is up to 2 times higher than CUDA's thread efficiency.

Parallel implementation of background subtraction algorithms for real-time video processing on a supercomputer platform

Results of evaluation of the background subtraction algorithms implemented on a supercomputer platform in a parallel manner are presented in the article. The aim of the work is to chose an algorithm, a number of threads and a task scheduling method, that together provide satisfactory accuracy and efficiency of a real-time processing of high-resolution camera images, maintaining the cost of resources usage at a reasonable level. Two selected algorithms: the Gaussian mixture models and the Codebook, are presented and their computational complexity is discussed. Various approaches to the parallel implementation, including assigning the image pixels to threads, the task scheduling methods and the thread management systems, are presented. The experiments were performed on a supercomputer cluster, using a single machine with 12 physical cores. The accuracy and performance of the implemented algorithms were evaluated for varying image resolutions and numbers of concurrent processing threads. On a basis of the evaluation results, an optimal configuration for the parallel implementation of the system for real-time video content analysis on a supercomputer platform was proposed.

Dynamically dispatching speculative threads to improve sequential execution

Efficiently utilizing multicore processors to improve their performance potentials demands extracting thread-level parallelism from the applications. Various novel and sophisticated execution models have been proposed to extract thread-level parallelism from sequential programs. One such execution model, Thread-Level Speculation (TLS), allows potentially dependent threads to execute speculatively in parallel. However, TLS execution is inherently unpredictable, and consequently incorrect speculation could degrade performance for the multicore systems. Existing approaches have focused on using the compilers to select sequential program regions to apply TLS. Our research shows that even the state-of-the-art compiler makes suboptimal decisions, due to the unpredictability of TLS execution. Thus, we propose to dynamically optimize TLS performance. This article describes the design, implementation, and evaluation of a runtime thread dispatching mechanism that adjusts the behaviors of speculative threads based on their efficiency. In the proposed system, speculative threads are monitored by hardware-based performance counters and their performance impact is evaluated with a novel methodology that takes into account various unique TLS characteristics. Thread dispatching policies are devised to adjust the behaviors of speculative threads accordingly. With the help of the runtime evaluation, where and how to create speculative threads is better determined. Evaluated with all the SPEC CPU2000 benchmark programs written in C, the dynamic dispatching system outperforms the state-of-the-art compiler-based thread management techniques by 9.4% on average. Comparing to sequential execution, we achieve 1.37X performance improvement on a four-core CMP-based system.

Multithreading on reconfigurable hardware: An architectural approach

In this paper, we address the problem of organization and management of threads on a multithreading custom computing machine composed of a General Purpose Processor (GPP) and Reconfigurable Coprocessors. We target higher portability, flexibility, and performance of the prospective design solutions by means of a strictly architectural approach. Our proposal to improve overall system performance is twofold. First, we provide architectural mechanisms to accelerate applications by supporting computationally intensive kernels with reconfigurable hardware accelerators. Second, we propose an infrastructure capable of facilitating thread management. Besides the architectural and microarchitectural extensions of the reconfigurable computing system, we also propose a hierarchical programming model. The model supports balanced and performance efficient SW/HW co-execution of multithreading applications. We demonstrate that our approach provides better performance–portability and performance–flexibility trade-off characteristics compared to other state-of-the-art proposals. The experimental results, based on real applications, suggest average system speedups between 1.2 and 19.6. Based on singlethreaded synthetic benchmark, we achieve average speedup between 8.5 and 129. For multithreaded synthetic benchmark, the achieved average speedup is between 1.3 and 7.3.

A lightweight and extensible Complex Event Processing system for sense and respond applications

Complex Event Processing (CEP) is considered as a promising technology for enabling the evolution of service-oriented enterprise systems towards operational aware systems. CEP effectively supports the implementation of “sense and respond” behaviours, as it enables to extract meaningful events from raw data streams originated by sensing infrastructures, for enterprise processes and applications consumption. This paper proposes a novel CEP engine conceived with ease of use, extensibility, portability, and scalability requirements in mind. More specifically, we propose a Lightweight Stage-based Event Processor (LiSEP) based on a layered architectural design. Thanks to the adoption of Staged Event-Driven Architecture principles, core event processing logic is decoupled from low-level thread management issues. This results in an easy-to-understand and extensible implementation while testing results show performance scalability. We report on the carrying out of a case study on dangerous goods monitoring in maritime transport. The objective of the case study is to develop a Proof of Concept application leveraging on LiSEP capabilities in sensor and RFID events processing for monitoring and alerting purposes.

PERFORMANCE ANALYSIS OF PARALLEL MATRIX MULTIPLICATION ON A MULTI-CORE COMPUTER USING JAVA THREADS

With the developing multi-core technology, it becomes quite practical to speed up sequential algorithms via parallel running threads. Java language which has built in libraries for the management of threads provides a good environment for developing parallel applications. In the presented study, performance of the parallel implementation of matrix multiplication algorithm that provides a basis for most of the matrix operations is investigated on a multi-core computer. In the experiments where six-core processor computer used, the impact of static and dynamic load management approaches which are common in practice for managing threads on effect of use of computational power are shown. Experiments are realized according to the number of threads and the matrix size to be multiplied. Through the experiments, it has been shown by graphical results that parallel algorithm bring significant speed up and parallel efficiency by comparing it to sequential equivalent.

Compiler-Supported Thread Management for Multithreaded Network Processors

Traditionally, runtime management involving CPU sharing, real-time scheduling, etc., is provided by the runtime environment (typically an operating system) using hardware support such as timers and interrupts. However, due to stringent performance requirements on network processors, neither OS nor hardware mechanisms are typically feasible/available. Mapping packet processing tasks on network processors involves complex trade-offs to maximize parallelism and pipelining. Due to an increase in the size of the code store and complexity of application requirements, network processors are being programmed with heterogeneous threads that may execute code belonging to different tasks on a given micro-engine. Also, most network applications are streaming applications that are typically processed in a pipelined fashion. Thus, the tasks on different micro-engines are pipelined in such a way as to maximize the throughput. Tasks themselves could have different runtime performance demands. In this article, we focus on network processors on which hardware can only schedule threads in a round-robin fashion and no OS assistance is provided. We show that it is very difficult and inefficient for the programmer to meet the constraints of runtime management by coding them statically. Due to the infeasibility of hardware or OS solution (even in the near future), we undertake a compiler approach. We propose a complete compiler solution to automatically insert explicit context switch (ctx) instructions provided on the network processor such that the execution of threads is better manipulated at runtime to meet their constraints. Two approaches are presented that can control programs’ runtime behavior with different applicability and overheads. We show that it is feasible and also opens new application domains that would need heterogeneous thread programming. Such approaches would in general become important for multicore processors. Finally, our experiments show that the runtime constraints are enforced nearly ideally with minimal runtime degradation and small code growth.

Lightweight Chip Multi-Threading (LCMT): Maximizing Fine-Grained Parallelism On-Chip

Irregular and dynamic applications, such as graph problems and agent-based simulations, often require fine-grained parallelism to achieve good performance. However, current multicore processors only provide architectural support for coarse-grained parallelism, making it necessary to use software-based multithreading environments to effectively implement fine-grained parallelism. Although these software-based environments have demonstrated superior performance over heavyweight, OS-level threads, they are still limited by the significant overhead involved in thread management and synchronization. In order to address this, we propose a Lightweight Chip Multi-Threaded (LCMT) architecture that further exploits thread-level parallelism (TLP) by incorporating direct architectural support for an “unlimited” number of dynamically created lightweight threads with very low thread management and synchronization overhead. The LCMT architecture can be implemented atop a mainstream architecture with minimum extra hardware to leverage existing legacy software environments. We compare the LCMT architecture with a Niagara-like baseline architecture. Our results show up to 1.8X better scalability, 1.91X better performance, and more importantly, 1.74X better performance per watt, using the LCMT architecture for irregular and dynamic benchmarks, when compared to the baseline architecture. The LCMT architecture delivers similar performance to the baseline architecture for regular benchmarks.

Parallel Multithreaded Processing for Data Set Summarization on Multicore CPUs

Data mining algorithms should exploit new hardware technologies to accelerate computations. Such goal is difficult to achieve in database management system (DBMS) due to its complex internal subsystems and because data mining numeric computations of large data sets are difficult to optimize. This paper explores taking advantage of existing multithreaded capabilities of multicore CPUs as well as caching in RAM memory to efficiently compute summaries of a large data set, a fundamental data mining problem. We introduce parallel algorithms working on multiple threads, which overcome the row aggregation processing bottleneck of accessing secondary storage, while maintaining linear time complexity with respect to data set size. Our proposal is based on a combination of table scans and parallel multithreaded processing among multiple cores in the CPU. We introduce several database-style and hardware-level optimizations: caching row blocks of the input table, managing available RAM memory, interleaving I/O and CPU processing, as well as tuning the number of working threads. We experimentally benchmark our algorithms with large data sets on a DBMS running on a computer with a multicore CPU. We show that our algorithms outperform existing DBMS mechanisms in computing aggregations of multidimensional data summaries, especially as dimensionality grows. Furthermore, we show that local memory allocation (RAM block size) does not have a significant impact when the thread management algorithm distributes the workload among a fixed number of threads. Our proposal is unique in the sense that we do not modify or require access to the DBMS source code, but instead, we extend the DBMS with analytic functionality by developing User-Defined Functions.

A moving threads processor architecture MTPA

Moving threads is a new kind of approach for multicore processor architectures. Traditionally, each thread stays in the core where it is created, and data is moved from the main memory via caches to each core and thread. In the moving threads approach, each core can access only a certain portion of the main memory via its local memory block, and thus extremely lightweight threads are moved between the cores. As a consequence, all kinds of cache coherence problems and need for read reply messages are eliminated. Also Lamport's sequential consistency of shared memory multiprocessor systems is achieved for free. In this paper, we propose a processor architecture (MTPA) for the moving threads paradigm. We describe the overall structure, operation, instruction set, and thread management mechanism as well as evaluate the proposed architecture with different functional unit settings with simulations and give early silicon area and power consumption estimates.