Speculative Multithreading Research Articles

GbA: A graph‐based thread partition approach in speculative multithreading

SummarySpeculative multithreading is an effective technique to automatically parallelize sequential programs. Conventional thread partition approaches primarily include heuristic rule–based (HR‐based) and machine learning–based. Heuristic rule–based approaches are effective to parallelize one type of programs and can seldom obtain the respective optimal partitions for different programs, and existing machine learning–based approaches usually use vector‐based characterization to represent a program, but easily ignore control information among basic blocks and partitions along other paths except the critical path. This paper proposes a novel graph‐based thread partition approach to overcome these 2 bottlenecks. Our approach characterizes programs with graphs, integrating feature and control informations, extracting good partition scheme successfully, and also applies a machine‐learning algorithm to predict partition for unseen programs. Prophet, which consists of an automatic parallelization compiler and a multicore simulator, evaluates the performance of multithreaded programs. Experiment results reveal that our approach delivers a maximum performance improvement of about 55.49% on an 8 core than HR‐based approach and a maximum 97.67% performance improvement over HR‐based partition for SPEC2000 benchmarks. This result suggests that graph‐based thread partition approach is effective for thread partition in speculative multithreading.

Qinling: A Parametric Model in Speculative Multithreading

Speculative multithreading (SpMT) is a thread-level automatic parallelization technique that can accelerate sequential programs, especially for irregular applications that are hard to be parallelized by conventional approaches. Thread partition plays a critical role in SpMT. Conventional machine learning-based thread partition approaches applied machine learning to offline guide partition, but could not explicitly explore the law between partition and performance. In this paper, we build a parametric model (Qinling) with a multiple regression method to discover the inherent law between thread partition and performance. The paper firstly extracts unpredictable parameters that determine the performance of thread partition in SpMT; secondly, we build a parametric model Qinling with extracted parameters and speedups, and train Qinling offline, as well as apply it to predict the theoretical speedups of unseen applications. Finally, validation is done. Prophet, which consists of an automatic parallelization compiler and a multi-core simulator, is used to obtain real speedups of the input programs. Olden and SPEC2000 benchmarks are used to train and validate the parametric model. Experiments show that Qinling delivers a good performance to predict speedups of unseen programs, and provides feedback guidance for Prophet to obtain the optimal partition parameters.

A hybrid sample generation approach in speculative multithreading

Speculative multithreading (SpMT) is a thread-level automatic parallelization technique to accelerate sequential programs. Machine learning has been successfully brought into SpMT to improve its performance. An appropriate sample set, which plays the role of knowledge provider, is important for machine learning-based (ML-based) thread partition. Conventionally, heuristic rules-based (HR-based) sample generation approach cannot generate adaptive samples. A hybrid sample generation approach can break this bottleneck. With this method, we firstly automatically generate samples, which are MIPS codes consisting of spawning points (SPs) and control quasi-independent points (CQIPs) by heuristic rules; secondly manually adjust the positions of SPs and CQIPs and rebuild pre-computation slice to obtain better performance for every sample; and then build model to ensure that the probability of adjusting to the optimal partition positions is increasing. During the implementation of this approach, three measures: bias weighting, preservation of optimal solutions, summary of greedy rules, are taken. In this way, we enhance the adjustment frequency for subroutines with high called time and preserve the optimal partition positions, so to achieve a stable speedup improvement. On Prophet, which is a generic SpMT processor to evaluate the performance of multithreaded programs, SPEC2000 and Olden benchmarks are used as input. Experiments show that our approach can obtain better sample sets, which deliver a better performance improvement of about 86.9% on a 16 core than the samples generated by HR-based approach. Experiment results also prove that this approach is effective to generate sample sets for ML-based thread partition.

Toward Emotion-Aware Computing: A Loop Selection Approach Based on Machine Learning for Speculative Multithreading

Emotion-aware computing can recognize, interpret, process, and simulate human affects. These programs in this area are compute-intensive applications, so they need to be executed in parallel. Loops usually have regular structures and programs spend significant amounts of time executing them, and thus loops are ideal candidates for exploiting the parallelism of sequential programs. However, it is difficult to decide which set of loops should be parallelized to improve program performance. The existing research is one-size-fits-all strategy and cannot guarantee to select profitable loops to be parallelized. This paper proposes a novel loop selection approach based on machine learning (ML-based) for selecting the profitable loops and paralleling them on multi-core by speculative multithreading (SpMT). It includes establishing sufficient training examples, building and applying prediction model to select profitable loops for speculative parallelization. Using the ML-based loop selection approach, an unseen emotion-aware sequential program can obtain a stable, much higher speedup than the one-size-fits-all approach. On Prophet, which is a generic SpMT processor to evaluate the performance of multithreaded programs, the novel loop selection approach is evaluated and reaches an average speedup of 1.87 on a 4-core processor. Experiment results show that the ML-based approach can obtain a significant increase in speedup, and Olden benchmarks deliver a better performance improvement of 6.70% on a 4-core than the one-size-fits-all approach.

Shared write buffer to boost applications on SpMT architecture

With the trend of growing number of integrated processing cores on Chip Multiprocessors, researchers are working hard to increase the available parallelism of software programs so as to efficiently harness the growing computing power. One noticeable direction among these efforts is speculative multi-threading (SpMT), a.k.a thread level speculation, which aims to extract thread level parallelism by splitting a sequential execution thread into several finer ones and execute them on parallel. A SpMT thread is in speculative status before it “knows” all its input data are correct. A speculative thread needs to write to the L1 cache, but its output might be discarded if the speculation eventually fails. However, another speculative thread may have already read in such speculative output. Therefore, some mechanism is needed to support speculative read and write. And because the SpMT threads are extracted from a single thread, they usually share lots of data, thus there might be intense data coherence among the L1 caches. It would be very complicated to support data coherence and speculation together. This Paper proposes a shared write buffer among the SpMT cores. We are able to confine the speculative read and write in the SWB, thus the speculation will not interference with coherence, and the L1 cache design could be drastically simplified. Experiments show that the SWB can capture a big portion of inter-core data sharing, reduce cache coherence, and drastically improve data access performance of SpMT threads.

プログラム全域を対象としたホットパスベース投機的マルチスレッド処理方式の提案

Nowadays, multicore processor is common presence and available everywhere. However, when we try to speedup non-numerical programs, which are used by ordinary users much especially on both desktop and mobile computers, by introducing parallel computation on multicore processors, it is necessary to strictly keep both control and data dependencies between parallel tasks in order to get the correct computation result. In many cases, parallel computation is difficult for non-numerical programs on multicore processors, since these dependencies are complicated. To solve this problem,we propose a method of speculative thread-level parallel computation along the hot path, the most frequently executed control flow, on entire program, assuming the use of the support hardware of speculative multithreading. This utilizes the general property of program codes, that is, the control flow of program tends to be highly biased to quite a small number of path even if the program code contains many possibilities of paths. Our method divides conventional serial program into thread codes along the hot path, and the thread codes are executed in parallel on each processor core on multicore processor. Preliminary performance evaluation by trace-based simulation using SPEC CINT2000, that is the collection of practical programs for performance valuation purpose, shows that the speedup of 2.48 times at maximum and 1.94 times on average can be expected by using our method, as compared to the serial execution.

Optimization of thread partitioning parameters in speculative multithreading based on artificial immune algorithm

Thread partition plays an important role in speculative multithreading (SpMT) for automatic parallelization of irregular programs. Using unified values of partition parameters to partition different applications leads to the fact that every application cannot own its optimal partition scheme. In this paper, five parameters affecting thread partition are extracted from heuristic rules. They are the dependence threshold (DT), lower limit of thread size (TSL), upper limit of thread size (TSU), lower limit of spawning distance (SDL), and upper limit of spawning distance (SDU). Their ranges are determined in accordance with heuristic rules, and their step-sizes are set empirically. Under the condition of setting speedup as an objective function, all combinations of five threshold values form the solution space, and our aim is to search for the best combination to obtain the best thread granularity, thread dependence, and spawning distance, so that every application has its best partition scheme. The issue can be attributed to a single objective optimization problem. We use the artificial immune algorithm (AIA) to search for the optimal solution. On Prophet, which is a generic SpMT processor to evaluate the performance of multithreaded programs, Olden benchmarks are used to implement the process. Experiments show that we can obtain the optimal parameter values for every benchmark, and Olden benchmarks partitioned with the optimized parameter values deliver a performance improvement of 3.00% on a 4-core platform compared with a machine learning based approach, and 8.92% compared with a heuristics-based approach.

Exploiting Thread-Level Parallelism Based on Balancing Load for Speculative Multithreading

Speculative Multithreading (SpMT) technology is an effective mechanism for automatic parallelization of irregular programs. While speculative parallelization can potentially deliver significant speedup for irregular applications, several speculative parallelization overheads resulting from the factors, especially inter-thread load imbalance, limit these speedups in practice. Most existing thread partitioning methods are mainly based on heuristic rules strategies to generate speculative threads. However, these heuristic rules extracted from people’s experiences cannot estimate quantitatively but qualitatively the overhead. Based on the thorough analysis of the speculative parallelization overhead resulting from inter-thread load imbalance, we propose a novel method to balance the inter-thread load. In this method, we firstly determine a method by which to unroll the loops. Then, we introduce cluster method to search the solution space of speculative threads and finally get the optimal solution. The experimental results show that, the proposed method can effectively reduce the inter-thread load imbalance; and the load imbalance overhead can be effectively reduced by 43.7%. And we can gain 8.8% performance improvement on Olden benchmark suits.

Data Structure Directed Thread Partitioning Method and Execution Model

在对程序进行并行化时,为了保证结果的正确性,并行编译器只能采取一种保守的策略,也就是,如果它不能确定两段代码在并行执行时是否会发生冲突,它就不允许这两段代码并行执行.虽然这种做法保证了正确性,但同时也限制了对并行性的开发.在这种背景下,许多推测多线程方法被提了出来,这些方法通过允许可能冲突的代码段并行执行来把握更多的并行机会,同时,通过从冲突中恢复来保证结果的正确性.然而,传统推测多线程方法所使用的“沿控制流将串行程序划分为多个线程”的做法并不适合不同数据结构上的操作在控制流中相互交错的情况,因为如果沿控制流将程序线性地划分为多个线程,则同一个数据结构上的操作将被分到不同的线程中,从而非常容易发生冲突.为了有效地对这些程序进行并行化,提出了一种基于数据结构的线程划分方法与执行模型.在这种方法中,程序中的对象被划分成多个组,同一组中对象上的操作被分派到同一个线程中去执行,从而降低了在同一个数据结构上发生冲突的可能性.;While parallelizing a program, to ensure the correctness of the result, a parallel compiler can only adopt a conservative pol icy that if it cannot accurately determine whether two pieces of code wi ll conflict while being executed in parallel, it does not permit them to be executed in parallel. Although this approach can ensure correctness, it also limits the amount of parallelism that can be exploited. A number of speculative multithreading (SpMT) approaches gained wide popularity because they can get more parallelism by allowing two pieces of potentially conflicting code to execute in parallel while ensuring the correctness of the results by recovering from conflicts. However, the dividing-along-the-control-flow-path method employed by the traditional SpMT approach is unsuitable for a number of programs in which operations on different data structures are interleaved in the control flow path. If the program is linearly divided into threads along the control flow path, the operations on the same data structure will be divided among the different threads that are doomed to be prone to conflict when being executed concurrently. To effectively parallelize these programs, this paper proposes a data structure centric approach, in which the objects of a program are organized into different groups and the operations on the objects wit hin the same group are dispatched to the same thread for execution, thereby reducing the likelihood of conflict on the same data structure.

A thread partitioning approach for speculative multithreading

Speculative multithreading (SpMT) is a thread-level automatic parallelization technique, which partitions sequential programs into multithreads to be executed in parallel. This paper presents different thread partitioning strategies for nonloops and loops. For nonloops, we propose a cost estimation based on combined run-time effects of various speculation factors to predict the resulting performance of candidate threads to guide the thread partitioning. For loops, we parallelize all the profitable loops that can potentially offer additional performance benefits by multilevel spawning in loop bodies, loop iterations, and inner loops. Then we select a proper thread boundary located in the front of loop branch instruction to reduce invalid spawning threads that waste core resources. Experimental results show that the proposed approach can obtain a significant increase in speedup and Olden benchmarks reach a performance improvement of 6.62 % on average.

Dynamic Selection of Speculative Paths in Two-Path Limited Speculation Method

Speculative multithreading is one of the most promising methods for speeding up the execution of programs in multicore systems. Each loop has numerous possible execution paths; however, in many cases, only a few of them are executed frequently. We focus on a two-path limited speculation method, which speculates on only the two most frequently executed paths based on path-profiling results of the whole program execution using path-profiling. To maximize the performance of the method, this paper discusses “phased behaviour” in program execution. This paper firstly shows actual behaviours in program execution in terms of execution paths. Then, we introduce practical methods of dynamic selection of speculation paths. Preliminary results show that, realistically, the optimal dynamic method increases the speed by 1.8 times at maximum and also that the practical dynamic method can increase the speculation success ratio by up to 30% compared with the static method.

Thread Partitioning Algorithm for Speculative Multithreading Based on Path Optimization

PDF HTML阅读 XML下载 导出引用 引用提醒 一种基于路径优化的推测多线程划分算法 DOI: 10.3724/SP.J.1001.2012.04148 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(61173040); 国家高技术研究发展计划(863)(2008AA01Z136) Thread Partitioning Algorithm for Speculative Multithreading Based on Path Optimization Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:推测多线程(speculative multithreading,简称SpMT)技术是一种实现非规则程序自动并行化的有效途径.然而,基于控制流图和分支预测技术的线程划分方法,不可避免地会受到划分路径上所存在的控制依赖和数据依赖的制约.目前,在传统的线程划分算法中存在的一个重要问题是,在对划分路径进行选取时只考虑了控制依赖影响却不能有效地综合考虑数据依赖的影响,进而导致不能选取最佳的划分路径.因此,针对传统方法中这种依赖评估方法效率低下的问题,设计并实现了一种基于路径优化的线程划分算法.该算法通过引入基于程序切片技术的预计算方法,建立一种路径评估方法来评估程序间的控制和数据依赖.同时,引入控制线程体大小的启发式规则,以便有效地解决负载不平衡的问题.基于Olden 测试集的测试结果表明,所提出的算法可以有效地对非规则程序进行划分,其平均加速比可以达到1.83. Abstract:Speculative multithreading (SpMT) technology is an effective mechanism for automatic parallelization of irregular programs. However, just generating speculative threads based on the control flow graph which only contains branch probability information is not enough. It is inevitable that there may be excessive constraints resulting from control dependence and data dependence in practice. In the traditional thread partitioning algorithms, a key problem is that these existing evaluation methods do not integrate the data dependence with the control speculation in order to obtain the maximum benefit. In order to solve the inefficiency of evaluation for control dependence and data dependence, this paper proposes a path optimization based thread partitioning algorithm. In this algorithm, by introducing pre-computation technique and discussing the trade-offs between different speculative paths the study establishes an evaluation model to evaluate the control dependence and data dependence. Meanwhile, in order to reduce the side-effect of workload imbalance, some heuristics rules are designed. The experimental results show that there are interesting trade-offs between different speculative paths and it is possible to achieve a better performance. On average, the study achieved an average speedup of 1.83 on Olden benchmark suits. 参考文献 相似文献 引证文献

Disjoint out-of-order execution processor

High-performance superscalar architectures used to exploit instruction level parallelism in single-thread applications have become too complex and power hungry for the multicore processors era. We propose a new architecture that uses multiple small latency-tolerant out-of-order cores to improve single-thread performance. Improving single-thread performance with multiple small out-of-order cores allows designers to place more of these cores on the same die. Consequently, emerging highly parallel applications can take full advantage of the multicore parallel hardware without sacrificing performance of inherently serial and hard to parallelize applications. Our architecture combines speculative multithreading (SpMT) with checkpoint recovery and continual flow pipeline architectures. It splits single-thread program execution into disjoint control and data threads that execute concurrently on multiple cooperating small and latency-tolerant out-of-order cores. Hence we call this style of execution Disjoint Out-of-Order Execution (DOE). DOE uses latency tolerance to overcome performance issues of SpMT caused by interthread data dependences. To evaluate this architecture, we have developed a microarchitecture performance model of DOE based on PTLSim, a simulation infrastructure of the x86 instruction set architecture. We evaluate the potential performance of DOE processor architecture using a simple heuristic to fork control independent threads in hardware at the target addresses of future procedure return instructions. Using applications from SpecInt 2000, we study DOE under ideal as well as realistic architectural constraints. We discuss the performance impact of key DOE architecture and application variables such as number of cores, interthread data dependences, intercore data communication delay, buffers capacity, and branch mispredictions. Without any DOE specific compiler optimizations, our results show that DOE outperforms conventional SpMT architectures by 15%, on average. We also show that DOE with four small cores can perform on average equally well to a large superscalar core, consuming about the same power. Most importantly, DOE improves throughput performance by a significant amount over a large superscalar core, up to 2.5 times, when running multitasking applications.

Performance Evaluation of Dynamic Speculative Multithreading with the Cascadia Architecture

Thread-level parallelism (TLP) has been extensively studied in order to overcome the limitations of exploiting instruction-level parallelism (ILP) on high-performance superscalar processors. One promising method of exploiting TLP is dynamic speculative multithreading (D-SpMT), which extracts multiple threads from a sequential program without compiler support or instruction set extensions. This paper introduces Cascadia, a D-SpMT multicore architecture that provides multigrain thread-level support and is used to evaluate the performance of several benchmarks. Cascadia applies a unique sustainable IPC (sIPC) metric on a comprehensive loop tree to select the best performing nested loop level to multithread. This paper also discusses the relationships that loops have on one another, in particular, how loop nesting levels can be extended through procedures. In addition, a detailed study is provided on the effects that thread granularity and interthread dependencies have on the entire system.

SpMT WaveCache: Exploiting Speculative Multithreading for Dataflow Computer

Speculative Multithreading(SpMT) increases the performance by means of executing multithreads speculatively to exploit thread-level parallelism. The authors expanded the transactional memory system of WaveCache by adding extra hardware components,such as Thread Synchronization Unit(TSU),Thread Context Table(TCT) and Thread Memory History(TMH),to improve the performance of SpMT WaveCache which is built on WaveScalar instruction set architecture(ISA). Furthermore,a novel two-levels commit method is proposed to support submitting thread-level transactions. Finally,the SpMT WaveCache is evaluated with 6 real benchmarks selected from SPEC,Mediabench and Mibench benchmarks. According to the experimental results,the SpMT WaveCache outperforms superscalar architecture ranging from 2X to 3X,and it also performs greater gain over original WaveCache and Transactional WaveCache. Therefore,the SpMT WaveCache is a good way to exploit thread-level parallelism of dynamic dataflow computer.

Mitosis: A Speculative Multithreaded Processor Based on Precomputation Slices

This paper presents the Mitosis framework, which is a combined hardware-software approach to speculative multithreading, even in the presence of frequent dependences among threads. Speculative multithreading increases single-threaded application performance by exploiting thread-level parallelism speculatively, that is, executing code in parallel, even when the compiler or runtime system cannot guarantee that the parallelism exists. The proposed approach is based on predicting/computing thread input values via software through a piece of code that is added at the beginning of each thread (the precomputation slice). A precomputation slice is expected to compute the correct thread input values most of the time but not necessarily always. This allows aggressive optimization techniques to be applied to the slice to make it very short. This paper focuses on the microarchitecture that supports this execution model. The primary novelty of the microarchitecture is the hardware support for the execution and validation of precomputation slices. Additionally, this paper presents new architectures for the register file and the cache memory in order to support multiple versions of each variable and allow for efficient rollback in case of misspeculation. We show that the proposed microarchitecture, together with the compiler support, achieves an average speedup of 2.2 for applications that conventional nonspeculative approaches are not able to parallelize at all.

DeLorean

Support for deterministic replay of multithreaded execution can greatly help in finding concurrency bugs. For highest effectiveness, replay schemes should (i) record at production-run speed, (ii) keep their logging requirements minute, and (iii) replay at a speed similar to that of the initial execution. In this paper, we propose a new substrate for deterministic replay that provides substantial advances along these axes. In our proposal, processors execute blocks of instructions atomically, as in transactional memory or speculative multithreading, and the system only needs to record the commit order of these blocks. We call our scheme DeLorean. Our results show that DeLorean records execution at a speed similar to that of Release Consistency (RC) execution and replays at about 82% of its speed. In contrast, most current schemes only record at the speed of Sequential Consistency (SC) execution. Moreover, DeLorean only needs 7.5% of the log size needed by a state-of-the-art scheme. Finally, DeLorean can be configured to need only 0.6% of the log size of the state-of-the-art scheme at the cost of recording at 86% of RC’s execution speed — still faster than SC. In this configuration, the log of an 8-processor 5-GHz machine is estimated to be only about 20GB per day.

Accurate branch prediction for short threads

Multi-core processors, with low communication costs and high availability of execution cores, will increase the use of execution and compilation models that use short threads to expose parallelism. Current branch predictors seek to incorporate large amounts of control flow history to maximize accuracy. However, when that history is absent the predictor fails to work as intended. Thus, modern predictors are almost useless for threads below a certain length. Using a Speculative Multithreaded (SpMT) architecture as an example of a system which generates shorter threads, this work examines techniques to improve branch prediction accuracy when a new thread begins to execute on a different core. This paper proposes a minor change to the branch predictor that gives virtually the same performance on short threads as an idealized predictor that incorporates unknowable pre-history of a spawned speculative thread. At the same time, strong performance on long threads is preserved. The proposed technique sets the global history register of the spawned thread to the initial value of the program counter. This novel and simple design reduces branch mispredicts by 29% and provides as much as a 13% IPC improvement on selected SPEC2000 benchmarks.

Accurate branch prediction for short threads

Multi-core processors, with low communication costs and high availability of execution cores, will increase the use of execution and compilation models that use short threads to expose parallelism. Current branch predictors seek to incorporate large amounts of control flow history to maximize accuracy. However, when that history is absent the predictor fails to work as intended. Thus, modern predictors are almost useless for threads below a certain length. Using a Speculative Multithreaded (SpMT) architecture as an example of a system which generates shorter threads, this work examines techniques to improve branch prediction accuracy when a new thread begins to execute on a different core. This paper proposes a minor change to the branch predictor that gives virtually the same performance on short threads as an idealized predictor that incorporates unknowable pre-history of a spawned speculative thread. At the same time, strong performance on long threads is preserved. The proposed technique sets the global history register of the spawned thread to the initial value of the program counter. This novel and simple design reduces branch mispredicts by 29% and provides as much as a 13% IPC improvement on selected SPEC2000 benchmarks.

Accurate branch prediction for short threads

Multi-core processors, with low communication costs and high availability of execution cores, will increase the use of execution and compilation models that use short threads to expose parallelism. Current branch predictors seek to incorporate large amounts of control flow history to maximize accuracy. However, when that history is absent the predictor fails to work as intended. Thus, modern predictors are almost useless for threads below a certain length. Using a Speculative Multithreaded (SpMT) architecture as an example of a system which generates shorter threads, this work examines techniques to improve branch prediction accuracy when a new thread begins to execute on a different core. This paper proposes a minor change to the branch predictor that gives virtually the same performance on short threads as an idealized predictor that incorporates unknowable pre-history of a spawned speculative thread. At the same time, strong performance on long threads is preserved. The proposed technique sets the global history register of the spawned thread to the initial value of the program counter. This novel and simple design reduces branch mispredicts by 29% and provides as much as a 13% IPC improvement on selected SPEC2000 benchmarks.