Specific Parallel Research Articles

Runtime, Speculative On-Stack Parallelization of For-Loops in Binary Programs

Nowadays almost every device has parallel architecture, hence parallelization is almost always desirable. However, parallelizing legacy running programs is very challenging. That is due to the fact that usually source code is not available, and runtime parallelization, without program restarting is challenging. Also, detecting parallelizable code is difficult, due to possible dependencies and different execution paths that are undecidable statically. Therefore, speculation is a typical approach whereby wrongly parallelized code is detected and rolled back at runtime. This paper considers utilizing processes to implement speculative parallelization using on-stack replacement, allowing for generally simple and portable design where forking a new process enters the speculative state, and killing a faulty process simply performs the roll back operation. While the cost of such operations are high, the approach is promising for cases where the parallel section is long and dependency issues are rare. Also, our proposed system performs speculative parallelization on binary code at runtime, without the need for source code, restarting the program, or special hardware support. Initial experiments show about 2× to 3× speedup for speculative execution over serial one, when three fourth of loop iterations are parallelizable. Also, maximum measured speculation overhead over pure parallel execution is 5.8 percent.

Fractal

Most systems that support speculative parallelization, like hardware transactional memory (HTM), do not support nested parallelism. This sacrifices substantial parallelism and precludes composing parallel algorithms. And the few HTMs that do support nested parallelism focus on parallelizing at the coarsest (shallowest) levels, incurring large overheads that squander most of their potential. We present FRACTAL, a new execution model that supports unordered and timestamp-ordered nested parallelism. FRACTAL lets programmers seamlessly compose speculative parallel algorithms, and lets the architecture exploit parallelism at all levels. FRACTAL can parallelize a broader range of applications than prior speculative execution models. We design a FRACTAL implementation that extends the Swarm architecture and focuses on parallelizing at the finest (deepest) levels. Our approach sidesteps the issues of nested parallel HTMs and uncovers abundant fine-grain parallelism. As a result, FRACTAL outperforms prior speculative architectures by up to 88x at 256 cores.

ReduxSTM: Optimizing STM designs for Irregular Applications

The exploitation of optimistic concurrency in modern multicore architectures via Transactional Memory (TM) is becoming a mainstream programming paradigm. TM features can be leveraged to provide support for speculative parallel execution of irregular applications, characterized by a lack of knowledge about data dependences at compile-time. This work is focused on software TM (STM) solutions and how they can be adapted and optimized to deal efficiently with irregular memory access patterns, mainly those caused by reduction operations.With this aim, ReduxSTM is introduced as a specific STM system designed by combining techniques for speculative execution with TM algorithms. ReduxSTM is based on three main design aspects: a transactional commit order mechanism which is available to guarantee sequential semantics when needed; a specific transactional memory primitive defined for expressing commutative and associative operations (reductions) that leverages the underlying TM privatization mechanism to avoid unnecessary transaction aborts caused by reduction memory patterns; and an enhanced conflict resolution mechanism that takes advantage of the two previous features.

A speculative parallel decompression algorithm on Apache Spark

Data decompression is one of the most important techniques in data processing and has been widely used in multimedia information transmission and processing. However, the existing decompression algorithms on multicore platforms are time-consuming and do not support large data well. In order to expand parallelism and enhance decompression efficiency on large-scale datasets, based on the software thread-level speculation technique, this paper raises a speculative parallel decompression algorithm on Apache Spark. By analyzing the data structure of the compressed data, the algorithm firstly hires a function to divide compressed data into blocks which can be decompressed independently and then spawns a number of threads to speculatively decompress data blocks in parallel. At last, the speculative results are merged to form the final outcome. Comparing with the conventional parallel approach on multicore platform, the proposed algorithm is very efficiency and obtains a high parallelism degree by making the best of the resources of the cluster. Experiments show that the proposed approach could achieve 2.6\(\times \) speedup when comparing with the traditional approach in average. In addition, with the growing number of working nodes, the execution time cost decreases gradually, and the speedup scales linearly. The results indicate that the decompression efficiency can be significantly enhanced by adopting this speculative parallel algorithm.

Toward Emotion-Aware Computing: A Loop Selection Approach Based on Machine Learning for Speculative Multithreading

Emotion-aware computing can recognize, interpret, process, and simulate human affects. These programs in this area are compute-intensive applications, so they need to be executed in parallel. Loops usually have regular structures and programs spend significant amounts of time executing them, and thus loops are ideal candidates for exploiting the parallelism of sequential programs. However, it is difficult to decide which set of loops should be parallelized to improve program performance. The existing research is one-size-fits-all strategy and cannot guarantee to select profitable loops to be parallelized. This paper proposes a novel loop selection approach based on machine learning (ML-based) for selecting the profitable loops and paralleling them on multi-core by speculative multithreading (SpMT). It includes establishing sufficient training examples, building and applying prediction model to select profitable loops for speculative parallelization. Using the ML-based loop selection approach, an unseen emotion-aware sequential program can obtain a stable, much higher speedup than the one-size-fits-all approach. On Prophet, which is a generic SpMT processor to evaluate the performance of multithreaded programs, the novel loop selection approach is evaluated and reaches an average speedup of 1.87 on a 4-core processor. Experiment results show that the ML-based approach can obtain a significant increase in speedup, and Olden benchmarks deliver a better performance improvement of 6.70% on a 4-core than the one-size-fits-all approach.

Variable Latency Speculative Parallel Prefix Adders for Unsigned and Signed Operands

A variable latency adder (VLA) reduces average addition time by using speculation: the exact arithmetic function is replaced by an approximated one, that is faster and gives correct results most of the times. When speculation fails, an error detection and correction circuit gives the correct result in the following clock cycle. Previous papers investigate VLAs based on Kogge-Stone, Han-Carlson or carry select topologies, speculating that carry propagation involves only a few consecutive bits. In several applications using 2's complement representation, however, operands have a Gaussian distribution and a nontrivial portion of carry chains can be as long as the adder size. In this paper we propose five novel VLA architectures, based on Brent-Kung, Ladner-Fisher, Sklansky, Hybrid Han-Carlson, and Carry increment parallel-prefix topologies. Moreover, we present a new efficient error detection and correction technique, that makes proposed VLAs suitable for applications using 2's complement representation. In order to investigate VLAs performances, proposed architectures have been synthesized using the UMC 65 nm library, for operand lengths ranging from 32 to 128 bits. Obtained results show that proposed VLAs outperform previous speculative architectures and standard (non-speculative) adders when high-speed is required.

Using the Xeon Phi Platform to Run Speculatively-Parallelized Codes

Intel Xeon Phi accelerators are one of the newest devices used in the field of parallel computing. However, there are comparatively few studies concerning their performance when using most of the existing parallelization techniques. One of them is thread-level speculation, a technique that optimistically tries to extract parallelism of loops without the need of a compile-time analysis that guarantees that the loop can be executed in parallel. In this article we evaluate the performance delivered by an Intel Xeon Phi coprocessor when using a software, state-of-the-art thread-level speculative parallelization library in the execution of well-known benchmarks. We describe both the internal characteristics of the Xeon Phi platform and the particularities of the thread-level speculation library being used as benchmark. Our results show that, although the Xeon Phi delivers a relatively good speedup in comparison with a shared-memory architecture in terms of scalability, the relatively low computing power of its computational units when specific vectorization and SIMD instructions are not fully exploited makes this first generation of Xeon Phi architectures not competitive (in terms of absolute performance) with respect to conventional multicore systems for the execution of speculatively parallelized code.

BFCA+: automatic synthesis of parallel code with TLS capabilities

Parallelization of sequential applications requires extracting information about the loops and how their variables are accessed, and afterwards, augmenting the source code with extra code depending on such information. In this paper we propose a framework that avoids such an error-prone, time-consuming task. Our solution leverages the compile-time information extracted from the source code to classify all variables used inside each loop according to their accesses. Then, our system, called BFCA+, automatically instruments the source code with the necessary OpenMP directives and clauses to allow its parallel execution, using the standard shared and private clauses for variable classification. The framework is also capable of instrumenting loops for speculative parallelization, with the help of the ATLaS runtime system, that defines a new speculative clause to point out those variables that may lead to a dependency violation. As a result, the target loop is guaranteed to correctly run in parallel, ensuring that its execution follows sequential semantics even in the presence of dependency violations. Our experimental evaluation shows that the framework not only saves development time, but also leads to a faster code than the one manually parallelized.

Optimization Strategies Oriented to Loop Characteristics in Software Thread Level Speculation Systems

Thread level speculation provides not only a simple parallel programming model, but also an effective mechanism for thread-level parallelism exploitation. The performance of software speculative parallel models is limited by high global overheads caused by different types of loops. These loops usually have different characteristics of dependencies and different requirements of optimization strategies. In this paper, we propose three comprehensive optimization techniques to reduce different factors of global overheads, aiming at requirements from different types of loops. Inter-thread fetching can reduce the high mis-speculation rate of the loops with frequent dependencies and out-of-order committing can reduce the control overhead of the loops with infrequent dependencies, while enhanced dynamic task granularity resizing can reduce the control overhead and optimize the global overhead of the loops with changing characteristics of dependencies. All these three optimization techniques have been implemented in HEUSPEC, a software TLS system. Experimental results indicate that they can satisfy the demands from different groups of benchmarks. The combination of these techniques can improve the performance of all benchmarks and reach a higher average speedup.

Compiler-Driven Software Speculation for Thread-Level Parallelism

Current parallelizing compilers can tackle applications exercising regular access patterns on arrays or affine indices, where data dependencies can be expressed in a linear form. Unfortunately, there are cases that independence between statements of code cannot be guaranteed and thus the compiler conservatively produces sequential code. Programs that involve extensive pointer use, irregular access patterns, and loops with unknown number of iterations are examples of such cases. This limits the extraction of parallelism in cases where dependencies are rarely or never triggered at runtime. Speculative parallelism refers to methods employed during program execution that aim to produce a valid parallel execution schedule for programs immune to static parallelization. The motivation for this article is to review recent developments in the area of compiler-driven software speculation for thread-level parallelism and how they came about. The article is divided into two parts. In the first part the fundamentals of speculative parallelization for thread-level parallelism are explained along with a design choice categorization for implementing such systems. Design choices include the ways speculative data is handled, how data dependence violations are detected and resolved, how the correct data are made visible to other threads, or how speculative threads are scheduled. The second part is structured around those design choices providing the advances and trends in the literature with reference to key developments in the area. Although the focus of the article is in software speculative parallelization, a section is dedicated for providing the interested reader with pointers and references for exploring similar topics such as hardware thread-level speculation, transactional memory, and automatic parallelization.

Speculative parallel pattern matching using stride-k DFA for deep packet inspection

Modern deep packet inspection (DPI) systems match network traffic against a large set of patterns which are defined using regular expressions. Deterministic finite automata (DFA) is generally preferred to parse these regular expressions. However, packets are mostly scanned one byte at a time which becomes a bottleneck for the DPI systems as they are unable to cope up with higher line rates. In this paper, we present an approach which allows a packet to be split up into two chunks. Furthermore, we scan the bytes of each chunk in parallel using speculation and multi-stride (stride-k) DFA. Stride-k DFAs results in fast processing of bytes but leads to high memory usage. Therefore, we propose a transition compression algorithm using alphabet compression table to limit the memory usage of multi-stride DFA. Experimental results show that the speculative parallel pattern matching using stride-k DFA leads to improvement in terms of speedup and latency over traditional DFA matching.

Speculative Parallelism Characterization Profiling in General Purpose Computing Applications

General purpose computing applications have not yet been thoroughly explored in procedure level speculation, especially in the light-weighted profiling way. This paper proposes a light-weighted profiling mechanism to analyze speculative parallelism characterization in several classic general purpose computing applications from SPEC CPU2000 benchmark. By comparing the key performance factors in loop and procedure-level speculation, it includes new findings on the behaviors of loop and procedure-level parallelism under these applications. The experimental results are as follows. The best gzip application can only achieve a 2.4X speedup in loop level speculation, while the best mcf application can achieve almost 3.5X speedup in procedure level. It proves that our light-weighted profiling method is also effective. It is found that between the loop-level and procedure-level TLS, the latter is better on several cases, which is against the conventional perception. It is especially shown in the applications where their 'hot' procedure body is concluded as 'hot' loops.

New Data Structures to Handle Speculative Parallelization at Runtime

Software-based, thread-level speculation (TLS) is a software technique that optimistically executes in parallel loops whose fully-parallel semantics can not be guaranteed at compile time. Modern TLS libraries allow to handle arbitrary data structures speculatively. This desired feature comes at the high cost of local store and/or remote recovery times: The easier the local store, the harder the remote recovery. Unfortunately, both times are on the critical path of any TLS system. In this paper we propose a solution that performs local store in constant time, while recover values in a time that is in the order of $$T$$T, being $$T$$T the number of threads. As we will see, this solution, together with some additional improvements, makes the difference between slowdowns and noticeable speedups in the speculative parallelization of non-synthetic, pointer-based applications on a real system. Our experimental results show a gain of 3.58$$\times $$× to 28$$\times $$× with respect to the baseline system, and a relative efficiency of up to, on average, 65 % with respect to a TLS implementation specifically tailored to the benchmarks used.

Exploiting Thread-Level Parallelism Based on Balancing Load for Speculative Multithreading

Speculative Multithreading (SpMT) technology is an effective mechanism for automatic parallelization of irregular programs. While speculative parallelization can potentially deliver significant speedup for irregular applications, several speculative parallelization overheads resulting from the factors, especially inter-thread load imbalance, limit these speedups in practice. Most existing thread partitioning methods are mainly based on heuristic rules strategies to generate speculative threads. However, these heuristic rules extracted from people’s experiences cannot estimate quantitatively but qualitatively the overhead. Based on the thorough analysis of the speculative parallelization overhead resulting from inter-thread load imbalance, we propose a novel method to balance the inter-thread load. In this method, we firstly determine a method by which to unroll the loops. Then, we introduce cluster method to search the solution space of speculative threads and finally get the optimal solution. The experimental results show that, the proposed method can effectively reduce the inter-thread load imbalance; and the load imbalance overhead can be effectively reduced by 43.7%. And we can gain 8.8% performance improvement on Olden benchmark suits.

SP-fold Speculative Parallelization for Parallel Algorithm of RNA Secondary Structure Prediction on Multicore

is faced with accelerating increase of data set sizes originating from powerful high-throughput measuring devices. Extensive computational power is the basic requirement for solving problems in bioinformatics. One of the key solutions to time-efficient data processing is the proper implementation of computational intensive tasks using parallel technology. A large number of cores is combined into a single chip to improve the overall performance of the multi- core processors. This depicts the current trends in processor architecture. This paper proposes a new software-only speculative parallelization scheme for implementing RNA Secondary Structure Prediction algorithm in parallel. The scheme is developed after a systematic evaluation of the design options available. It is also shown to be efficient, robust and to outperform previously proposed schemes used for parallel implementation of RNA Secondary Structure Prediction.

An explicit parallelism study based on thread-level speculation


 
 
 Developments in parallel architectures are an important branch in computer science. The success of such architectures derives from their inherent ability to improve the program performances. However, their ability to improve the performance on programs depends on the parallelism extraction strategies, which are always limited by the logic of each sequential program. Speculation is the only known alternative to overcome these constraints and increase the parallelism. In this paper, we study the explicit speculative parallelism using a library of thread-level speculation. We present the design of this library and study different speculative models: speculation of decision structures, speculation of loops, and speculation of critical sections. Our study evaluates different cases taken from SPEC CPU 2000, allowing acceleration of about 1.8x in multicore architectures (four core) with coarse-grained multithreaded.
 
 

Squashing Alternatives for Software-Based Speculative Parallelization

Speculative parallelization is a runtime technique that optimistically executes sequential code in parallel, checking that no dependence violations arise. In the case of a dependence violation, all mechanisms proposed so far either switch to sequential execution, or conservatively stop and restart the offending thread and all its successors, potentially discarding work that does not depend on this particular violation. In this work we systematically explore the design space of solutions for this problem, proposing a new mechanism that reduces the number of threads that should be restarted when a data dependence violation is found. Our new solution, called exclusive squashing, keeps track of inter-thread dependencies at runtime, selectively stopping and restarting offending threads, together with all threads that have consumed data from them. We have compared this new approach with existent solutions on a real system, executing different applications with loops that are not analyzable at compile time and present as much as 10% of inter-thread dependence violations at runtime. Our experimental results show a relative performance improvement of up to 14%, together with a reduction of one-third of the numbers of squashed threads. The speculative parallelization scheme and benchmarks described in this paper are available under request.

Leveraging GPUs using cooperative loop speculation

Graphics processing units, or GPUs, provide TFLOPs of additional performance potential in commodity computer systems that frequently go unused by most applications. Even with the emergence of languages such as CUDA and OpenCL, programming GPUs remains a difficult challenge for a variety of reasons, including the inherent algorithmic characteristics and data structure choices used by applications as well as the tedious performance optimization cycle that is necessary to achieve high performance. The goal of this work is to increase the applicability of GPUs beyond CUDA/OpenCL to implicitly data-parallel applications written in C/C++ using speculative parallelization. To achieve this goal, we propose Paragon : a static/dynamic compiler platform to speculatively run possibly data-parallel portions of sequential applications on the GPU while cooperating with the system CPU. For such loops, Paragon utilizes the GPU in an opportunistic way while orchestrating a cooperative relation between the CPU and GPU to reduce the overhead of miss-speculations. Paragon monitors the dependencies for the loops running speculatively on the GPU and nonspeculatively on the CPU using a lightweight distributed conflict detection designed specifically for GPUs, and transfers the execution to the CPU in case a conflict is detected. Paragon resumes the execution on the GPU after the CPU resolves the dependency. Our experiments show that Paragon achieves 4x on average and up to 30x speedup compared to unsafe CPU execution with four threads and 7x on average and up to 64x speedup versus sequential execution across a set of sequential but implicitly data-parallel applications.

The BonaFide C Analyzer: automatic loop-level characterization and coverage measurement

The advent of multicore technologies has increased the interest in parallelization techniques for existing sequential applications. These techniques include the need of detecting loops that are good candidates for parallelization, and classifying all variables of these loops according to their use, a task surprisingly hard to be carried out manually. In this paper, we introduce the BonaFide C Analyzer, an XML-based framework that combines static analysis of source code with profiling information to generate complete reports regarding all loops in a C application, including loop coverage, loop suitability for parallelization, a classification of all variables inside loops based on their accesses, and other hurdles that restrict the parallelization. This information allows to analyze how particular language constructs are used in real-world applications, and helps the programmer to parallelize the code. To show the features of the framework, we present the results of an in-depth loop characterization of C applications that are part of the SPEC CPU2006 benchmark suite. Our study shows that 47.72 % of loops present in the applications analyzed are potentially parallelizable with existent parallel programming models such as OpenMP, while an additional 37.7 % of loops could be run in parallel with the help of runtime speculative parallelization techniques.

Exploring speculative procedure and loop level parallelism in SPLASH2

How to make use of multicore computing resources to accelerate high performance computing HPC applications has become a common concern problem. However, HPC applications have not yet been explored in thread level speculation TLS thoroughly, especially in the procedure level. This paper proposes a procedure and loop level speculation architecture model for speeding up HPC applications, including its speculative mechanism, analysis method, etc. It also takes several applications from SPLASH2 to analyse their speculative parallel potential together with performance impacting factors. The experimental results show that: 1 the best Barnes application can get a 90.9× speedup in loop level speculation while Lu application can get 40.2× speedup in procedure level speculation; 2 limited parallelism coverage and severe inter-thread data dependence violations badly affect both loop and procedure level speculative parallelism in some HPC applications; 3 It is found that although loop structure is the main source of speculative parallelism, procedure structure can be treated as its important supplement. Especially in applications that their 'hot' iteration body concludes multiple procedure calls, higher speculative procedure level speedup can be achieved than that in loop level speculation.