Simultaneous Multithreading Processors Research Articles

Efficient resource sharing algorithm for physical register file in simultaneous multi-threading processors

Simultaneous Multi-Threading (SMT) processors increase performance by allowing concurrent execution of multiple independent threads with sharing of key datapath components and better utilization of the resources. An SMT processor usually maintains a shared register file to accommodate multiple threads for register renaming. By supporting inter-thread sharing of the physical registers, an SMT system processor can reduce the number of registers that would have been required in multiple superscalar processors while achieving a comparable throughput. However, congested shared resources due to slower threads can easily lead to inefficient usage of the resources and thus an undesirable performance outcome. In this paper, we propose an allocation algorithm at the architectural level for a better utilization of the shared register file. We show that, by limiting the number of the physical register entries each thread is allowed to occupy at any given time, the overall system throughput is enhanced by a substantial margin. An improvement in IPC of up to 44.6% and 32.7% is observed when the proposed technique is applied to a 4-threaded and a 6-threaded SMT system, respectively. Furthermore, a 4-threaded system with a physical register file of 160 entries can deliver a performance comparable to that of a default system with 256 entries, reflecting a resource saving of 37.5%.

Simultaneous Multithreaded Matrix Processor

This paper proposes a simultaneous multithreaded matrix processor (SMMP) to improve the performance of data-parallel applications by exploiting instruction-level parallelism (ILP) data-level parallelism (DLP) and thread-level parallelism (TLP). In SMMP, the well-known five-stage pipeline (baseline scalar processor) is extended to execute multi-scalar/vector/matrix instructions on unified parallel execution datapaths. SMMP can issue four scalar instructions from two threads each cycle or four vector/matrix operations from one thread, where the execution of vector/matrix instructions in threads is done in round-robin fashion. Moreover, this paper presents the implementation of our proposed SMMP using VHDL targeting FPGA Virtex-6. In addition, the performance of SMMP is evaluated on some kernels from the basic linear algebra subprograms (BLAS). Our results show that, the hardware complexity of SMMP is 5.68 times higher than the baseline scalar processor. However, speedups of 4.9, 6.09, 6.98, 8.2, 8.25, 8.72, 9.36, 11.84 and 21.57 are achieved on BLAS kernels of applying Givens rotation, scalar times vector plus another, vector addition, vector scaling, setting up Givens rotation, dot-product, matrix–vector multiplication, Euclidean length, and matrix–matrix multiplications, respectively. The average speedup over the baseline is 9.55 and the average speedup over complexity is 1.68. Comparing with Xilinx MicroBlaze, the complexity of SMMP is 6.36 times higher, however, its speedup ranges from 6.87 to 12.07 on vector/matrix kernels, which is 9.46 in average.

Dynamic MIPS Rate Stabilization for Complex Processors

Modern microprocessor cores reach their high performance levels with the help of high clock rates, parallel and speculative execution of a large number of instructions, and vast cache hierarchies. Modern cores also have adaptive features to regulate power and temperature and avoid thermal emergencies. All of these features contribute to highly unpredictable execution times. In this article, we demonstrate that the execution time of in-order (IO), out-of-order (OoO), and OoO simultaneous multithreaded processors can be stable and predictable by stabilizing their mega instructions executed per second (MIPS) rate via a proportional, integral, and differential (PID) gain feedback controller and dynamic voltage and frequency scaling (DVFS). Processor cores in idle cycles are continuously consuming power, which is highly undesirable in systems, especially in real-time systems. In addition to meeting deadlines in real-time systems, our MIPS rate stabilization framework can be applied on top of it to reduce power and energy by avoiding idle cycles. If processors are equipped with MIPS rate stabilization, the execution time can be predicted. Because the MIPS rate remains steady, a stabilized processor meets deadlines on time in real-time systems or in systems with quality-of-service execution latency requirements at the lowest possible frequency. To demonstrate and evaluate this capability, we have selected a subset of the MiBench benchmarks with the widest execution rate variations. We stabilize their MIPS rate on a 1GHz Pentium III--like OoO single-thread microarchitecture, a 1.32GHz StrongARM-like IO microarchitecture, and the 1GHz OoO processor augmented with two-way and four-way simultaneous multithreading. Both IO and OoO cores can take advantage of the stabilization framework, but the energy per instruction of the stabilized OoO core is less because it runs at a lower frequency to meet the same deadlines. The MIPS rate stabilization of complex processors using a PID feedback control loop is a general technique applicable to environments in which lower power or energy coupled with steady, predictable performance are desirable, although we target more specifically real-time systems in this article.

Reliability‐aware simultaneous multithreaded architecture using online architectural vulnerability factor estimation

Miniaturisation in modern microprocessors increases susceptibility to soft errors leading to reliability degradation. Recently simultaneous multithreaded (SMT) architecture is utilised to improve fault tolerance. Despite full coverage, redundant checking in such schemes causes significant performance and energy overheads. Fortunately, some of the soft errors can be masked at the architectural level and architectural vulnerability factor (AVF) of a structure represents the portion of soft errors which lead to a failure in the output of a program. In this study, the authors present an infrastructure for online monitoring of AVF of sensitive structures of an SMT processor. Based on estimated AVF, we have introduced partial thread redundancy (PTR) protection scheme for intervals whose AVF is greater than a predefined threshold and the estimated AVF is used for adaptation between reliability improvement or performance enhancement, especially when the processor executes more than one workload. We have utilised SPEC CPU2006 benchmarks for AVF estimation of some important hardware resources such as issue queue, reorder buffer, load/store queue and register file. Experimental results show that the mean absolute error of our AVF estimation method varies from 0.04 to 0.07 and combined online AVF estimation and PTR protection, leads to a reliability aware execution and lower performance overhead.

A resource utilization based instruction fetch policy for SMT processors

Simultaneous Multithreading (SMT) architectures are proposed to better explore on-chip parallelism, which capture the essence of performance improvement in modern processors. SMT overcomes the limits in a single thread by fetching and executing from multiple of them in a shared fashion. The long-latency operations, however, still cause inefficiency in SMT processors. When instructions have to wait for data from lower-level memory hierarchy, the dependent instructions cannot proceed, hence continue occupying the shared resources on the chip for an extended number of clock cycles. This introduces undesired inter-thread interference in SMT processors, which further leads to negative impacts on overall system throughput and average thread performance. In practice, instruction fetch policies take the responsibility of assigning thread priority at the fetch stage, in an effort to better distribute the shared resources among threads in the same core to cope with the long-latency operations and other runtime behavior from the thread for better performance.In this paper we propose an instruction fetch policy RUCOUNT, which considers resource utilization of individual thread in the prioritization process. The proposed policy observes instructions in the front-end stages of the pipeline as well as low-level data misses to summarize the resource utilization for thread management. Higher priority is granted to the thread(s) with less utilized resources, such that overall resources are distributed more efficiently in SMT processors. As a result, it has two unique features compared to other policies: one is to observe the hardware resource comprehensively and the other is to monitor limited resource entries. Our experimental results demonstrate that RUCOUNT is 20% better than ICOUNT, 10% than Stall, 8% than DG and 3% than DWarn, in terms of averaged performance. Considering its hardware overhead is at the similar level as ICOUNT and DWarn, our proposed instruction fetch policy RUCOUNT is superior among the studied policies.

Thread scheduling and memory coalescing for dynamic vectorization of SPMD workloads

We propose a new thread synchronization heuristic called Min-SP/PC.Min-SP/PC handles function calls better than previous algorithms.Many instructions in SPMD programs are identical across threads.Many memory accesses are either uniform or affine across threads. Simultaneous Multi-Threading (SMT) is a hardware model in which different threads share the same processing unit. This model is a compromise between high parallelism and low hardware cost. Minimal Multi-Threading (MMT) is one architecture recently proposed that shares instruction decoding and execution between threads running the same program in an SMT processor, thereby generalizing the approach followed by Graphics Processing Units to general-purpose processors. In this paper we propose new ways to expose redundancies in the MMT execution model. First, we propose and evaluate a new thread reconvergence heuristic that handles function calls better than previous approaches. Our heuristic only inspects the program counter and the stack frame to reconverge threads; hence, it is amenable to efficient and inexpensive hardware implementation. Second, we demonstrate that this heuristic is able to reveal the existence of substantial regularity in inter-thread memory access patterns. We validate our results on data-parallel applications from the PARSEC and SPLASH suites. Our new reconvergence heuristic increases the throughput of our MMT model by 7%, when compared to a previous, and substantially more complex approach, due to Long et al. Moreover, it gives us an effective way to increase regularity in memory accesses. We have observed that over 70% of simultaneous memory accesses are either the same for all the threads, or are affine expressions of the thread identifier. This observation motivates the design of newly proposed hardware that benefits from regularity in inter-thread memory accesses.

Mitigating Voltage Emergency in Simultaneous Multithreading Processor by Memory Level Parallelism Aware Thread Scheduling

Low power design techniques like clock gating incur side-effect of increasing current variation drawn by processors.Current variation produces voltage ripples due to parasitic inductance in power supply networks.Voltage emergency occurs when voltage drops below threshold,which may result in system malfunction because of timing faults.This work characterized the relationship between voltage emergencies and memory access behaviors in simultaneous multithreading processors.And then proposed a thread scheduling strategy utilizing memory level parallelism to mitigate voltage emergencies.Experimental results show that,compared with existing work,the proposed strategy reduces voltage emergencies by 21.7% and 25.2% for 2-thread workloads and 4-thread workloads respectively,and achieves better overall balance between performance and fairness.

Research on Trace Drive Instruction Cache Timing Attack on RSA

指令Cache 攻击是基于获取算法执行路径的一种旁路攻击方式.首先,通过分析原有RSA 指令Cache 计时攻击存在可行性不高且能够获取的幂指数位不足等局限性,建立了新的基于监视整个指令Cache 而不只是监视特定指令Cache 的踪迹驱动计时攻击模型;然后,提出了一种改进的基于SWE 算法窗口大小特征的幂指数分析算法;最后,在实际环境下,利用处理器的同步多线程能力确保间谍进程与密码进程能够同步运行.针对OpenSSLv.0.9.8f 中的RSA算法执行指令Cache 计时攻击实验,实验结果表明:新的攻击模型在实际攻击中具有更好的可操作性;改进的幂指数分析算法能够进一步缩小密钥搜索空间,提高了踪迹驱动指令Cache 计时攻击的有效性.对于一个512 位的幂指数,新的分析算法能够比原有分析算法多恢复出大约50 个比特位.;The I-cache timing attack which exploits the instruction path of a cipher is one type of side channel attack. First, by analyzing the complications in the previous I-cache timing attacks on RSA algorithm because of how hard it has been to put them into practice, and how the number of the inferred bits is insufficient, this paper builds a new trace driven I-cache timing attack model via spying on the whole I-cache, instead targeting the instruction cache to which the special function mapped. Next, an improved analysis algorithm of the exponent based on the characteristic of the side of window in sliding window exponentiation (SWE) algorithm is proposed. Finally, an I-cache timing attack is implemented on RSA of OpenSSL v.0.9.8f in a practical environment, using a simultaneous multithreading processor to insure that the spy process and the cipher process can run in parallel. Experimental results show that the proposed attack model has strong applicability in real environments; the improved analysis algorithm of the exponent can further reduce the search space of the bits of the key, and improve the effectively of the trace driven I-cache timing attack. For a 512-bit exponent, it can recover about 50 bits of exponent more than the previous.

Recalling instructions from idling threads to maximize resource utilization for simultaneous multi-threading processors

Simultaneous Multi-Threading (SMT) has been a very popular design in improving resource utilization by sharing key datapath components among multiple independent threads. However, allowing any of the threads to overwhelm these shared resources not only leads to unfair thread processing but may also result in severely degraded overall performance. How to prevent idling threads from clogging the critical resources in the pipeline becomes a must in sustaining desired system performance. In this paper, we show that, if one can manage to recall instructions of idling threads from the shared Issue Queue (IQ), the system performance is easily enhanced by a significant margin, with up to 20% for some benchmark mixes. An even more noteworthy feature about this technique is that the ensuing hardware overhead is very insignificant and it can also be coupled with other advanced techniques employed in other stages of the SMT pipeline for potentially additive benefits.

Design of High performance and Low power Simultaneous Multi-Threaded Processor

In this paper, we present the design of a High Performance Multi-Threaded Processor. Processing of high quality images is inevitable in applications such as, HD TV, Gaming Multimedia, etc. which require a great processing power with low power consumption. This can be achived with multi-threaded processors which optimally utilises the Functional Units (Fus). The speed of processing is as good as multi-core processors with lesser area. A conflict resolver (CR) is designed for scheduling the instructions, which involves allocation of Fu. The data move instructions are in majority in any of the programs; the corresponding logic blocks are replicated and speed of execution is further improved. We illustrated for two-threaded processorHowever, it is possible to extend the design for any number of threads by suitably redesigning the CR, and also replicate Transfer Logic and CPU Registers. DOI: http://dx.doi.org/10.11591/ijece.v3i3.2530

Adaptive instruction dispatching techniques for Simultaneous Multi-Threading (SMT) processors

Simultaneous Multi-Threading (SMT) provides a technique to improve resource utilization ability by sharing key data-path components among multiple independent threads. When critical resources are shared by multiple threads, effective use of these resources proves to be the most important factor in fully exploiting the system potential. Transient behaviors of various threads in terms of their execution parallelism can easily affect utilization efficiency of these shared resources. To commit more resources to threads that are more active allows for better resource utilization and thus higher throughput. In this paper, we propose a real-time dynamic scheduler for the SMT which dispatches instructions from threads based on thread-activeness information gathered in real time and dynamically adjusts dispatching priorities among threads accordingly. An extensive simulation shows a significant gain in system throughput by this technique. The performance of the proposed dispatching technique is evaluated on different workload mixtures created based on instruction-level parallelism available in each thread. An average of 6.5% and maximum of 15% performance improvement is observed with the proposed dispatching technique.

Probabilistic modeling for job symbiosis scheduling on SMT processors

Symbiotic job scheduling improves simultaneous multithreading (SMT) processor performance by coscheduling jobs that have “compatible” demands on the processor's shared resources. Existing approaches however require a sampling phase, evaluate a limited number of possible coschedules, use heuristics to gauge symbiosis, are rigid in their optimization target, and do not preserve system-level priorities/shares. This article proposes probabilistic job symbiosis modeling, which predicts whether jobs will create positive or negative symbiosis when coscheduled without requiring the coschedule to be evaluated. The model, which uses per-thread cycle stacks computed through a previously proposed cycle accounting architecture, is simple enough to be used in system software. Probabilistic job symbiosis modeling provides six key innovations over prior work in symbiotic job scheduling: (i) it does not require a sampling phase, (ii) it readjusts the job coschedule continuously, (iii) it evaluates a large number of possible coschedules at very low overhead, (iv) it is not driven by heuristics, (v) it can optimize a performance target of interest (e.g., system throughput or job turnaround time), and (vi) it preserves system-level priorities/shares. These innovations make symbiotic job scheduling both practical and effective. Our experimental evaluation, which assumes a realistic scenario in which jobs come and go, reports an average 16% (and up to 35%) reduction in job turnaround time compared to the previously proposed SOS (sample, optimize, symbios) approach for a two-thread SMT processor, and an average 19% (and up to 45%) reduction in job turnaround time for a four-thread SMT processor.

Energy savings in simultaneous multi-threaded processors through dynamic resizing of datapath resources

Nowadays, all the designers of systems from high-performance servers to battery-operated handheld devices aim for reliability, high-performance and longevity. Central within these aims the issue of processor power consumption is becoming increasingly important. In this study, we aim to adapt our already-proven method for single-threaded superscalar processors to simultaneous multi-threaded (SMT) processors for energy savings. The original method focused on resizing datapath resources according to the demands of running applications. To achieve this, the targeted resources are physically divided into multiple partitions, and turned on and off according to the needs of the applications. Since, the energy consumption of the turned-off datapath resources is quite low, as a result, it becomes possible to have great amount of energy savings within a processor. However, special care must be taken when there are multiple threads racing against each other to gain access to shared datapath resources. As a result, our proposed microarchitectural technique achieves 0.5% Instructions Per Cycle (IPC) and 3.2% Total number of instructions Per Cycle (TPC) improvement, while it turns off 45% of the Reorder Buffer (ROB), 59% of the Load-Store Queue (LSQ), 43% of the Issue Queue (IQ), 30% of the integer Physical Register Files (PRF) and, finally, 48% of the floating PRF, on the average across all simulated benchmarks. According to our estimates, the total processor power is reduced by 12%, on the average.

Exploring branch target buffer access filtering for low-energy and high-performance microarchitectures

Powerful branch predictors along with a large branch target buffer (BTB) are employed in superscalar and simultaneous multi-threading (SMT) processors for instruction-level parallelism and thread-level parallelism exploitation. However, the large BTB not only dominates the predictor energy consumption, but also becomes a major roadblock in achieving faster clock frequencies at deep sub-micron technologies. The authors propose here a filtering scheme to dramatically reduce the accesses to the BTB to achieve significantly reduced energy consumption in the BTB while maintaining the performance. For a simulated superscalar microprocessor, the experimental evaluation shows that the BTB access filtering (BAF) design achieves an 88.5% dynamic energy reduction with negligible performance loss. The authors also study the leakage behaviour and its control in the BAF design. The results show that by applying a drowsy strategy, very effective leakage control can be achieved. For the high-performance design, the BAF can also improve BTB's performance scalability at new technologies. For the simultaneous multi-threading environment, the authors evaluate the effectiveness of the BAF design and propose a banked BAF (BK-BAF) scheme to further reduce the energy consumption and performance overhead. The experimental results confirm that the BK-BAF scheme can be an energy/performance-effective design for next generation SMT processors.

Managing SMT resource usage through speculative instruction window weighting

Simultaneous multithreading processors dynamically share processor resources between multiple threads. In general, shared SMT resources may be managed explicitly, for instance, by dynamically setting queue occupation bounds for each thread as in the DCRA and Hill-Climbing policies. Alternatively, resources may be managed implicitly; that is, resource usage is controlled by placing the desired instruction mix in the resources. In this case, the main resource management tool is the instruction fetch policy which must predict the behavior of each thread (branch mispredictions, long-latency loads, etc.) as it fetches instructions. In this article, we present the use of Speculative Instruction Window Weighting (SIWW) to bridge the gap between implicit and explicit SMT fetch policies. SIWW estimates for each thread the amount of outstanding work in the processor pipeline. Fetch proceeds for the thread with the least amount of work left. SIWW policies are implicit as fetch proceeds for the thread with the least amount of work left. They are also explicit as maximum resource allocation can also be set. SIWW can use and combine virtually any of the indicators that were previously proposed for guiding the instruction fetch policy (number of in-flight instructions, number of low confidence branches, number of predicted cache misses, etc.). Therefore, SIWW is an approach to designing SMT fetch policies, rather than a particular fetch policy. Targeting fairness or throughput is often contradictory and a SMT scheduling policy often optimizes only one performance metric at the sacrifice of the other metric. Our simulations show that the SIWW fetch policy can achieve at the same time state-of-the-art throughput, state-of-the-art fairness and state-of-the-art harmonic performance mean.

Utilization-Based Resource Partitioning for Power-Performance Efficiency in SMT Processors

Simultaneous multithreading (SMT) increases processor throughput by allowing parallel execution of several threads. However, fully sharing processor resources may cause resource monopolization by a single thread or other misallocations, resulting in overall performance degradation. Static resource partitioning techniques have been suggested, but are not as effective as dynamic ones since program behavior does change over the course of its execution. In this paper, we propose an Adaptive Resource Partitioning Algorithm (ARPA) that dynamically assigns resources to threads according to changes in thread behavior. ARPA analyzes the resource usage efficiency of each thread in a given time period and assigns more resources to threads which can use them more efficiently. Its purpose is to improve the efficiency of resource utilization, thereby improving overall instruction throughput. Our simulation results on a set of 42 multiprogramming workloads show that ARPA outperforms the traditional fetch policy ICOUNT by 55.8 percent with regard to overall instruction throughput and achieves a 33.8 percent improvement over Static Partitioning. It also outperforms the current best dynamic resource allocation technique, Hill-climbing, by 5.7 percent. Considering fairness accorded to each thread, ARPA attains 43.6, 18.5, and 9.2 percent improvements over ICOUNT, Static Partitioning, and Hill-climbing, respectively, using a common fairness metric. We also explore the energy efficiency of dynamically controlling the number of powered-on reorder buffer entries for ARPA. Compared with ARPA, our energy-aware resource partitioning algorithm achieves 10.6 percent energy savings, while the performance loss is negligible.

Functional verification of the IBM POWER7 microprocessor and POWER7 multiprocessor systems

This paper describes the methods and techniques used to verify the POWER7® microprocessor and systems. A simple linear extension of the methodology used for POWER4®, POWER5®, and POWER6® was not possible given the aggressive design point and schedule of the POWER7 project. In addition to the sheer complexity of verifying an eight-core processor chip with scalability to 32 sockets, central challenges came from the four-way simultaneous multithreading processor core, a modular implementation structure with heavy use of asynchronous interfaces, aggressive memory subsystem design with numerous new reliability, availability, and serviceability (RAS) advances, and new power management and RAS mechanisms across the chip and the system. Key aspects of the successful verification project include a systematic application of IBM's random-constrained unit verification, unprecedented use of formal verification, thread-scaling support in core verification, and a consistent use of functional coverage across all verification disciplines. Functional coverage instrumentation, which is combined with the use of the newest IBM hardware simulation accelerator platform, enabled coverage-driven development of postsilicon exercisers in preparation of bring-up, a foundation for the desired systematic linkage of presilicon and postsilicon verification. RAS and power management verification also required new approaches, extending these disciplines to span all the way from the unit level to the end-to-end scenarios using the hardware accelerators.

Easing Instruction Queue Competition among Threads in RMT

As chip feature size decreases, processors are getting more and more sensitive to soft errors. To find cheaper reliability solutions has attracted the attention of many researches. SMT (Simultaneous Multithreading) processor permits multiple issues from different threads at the same time, which provides nature support for fault-tolerance by executing threads redundantly. Many RMT (Redundant Multithreading) architectures have been proposed. In those architectures, IQ (Instruction Queue) is a critical resource that affects the performance obviously. This paper proposed DDDI (Delay Dispatching Dependent Instructions) strategy which can use IQ more efficiently. In DDDI, instructions that dependent on load instructions that encounter cache miss can't be dispatched in to IQ until the load instructions get values from L2 cache or main memory. Experiments show that DDDI can avoid the threads that encounter cache miss blocking IQ resources, and not only IQ, but also the whole pipeline can be used more efficiently. Performance is boosted outstandingly.

On the Power Management of Simultaneous Multithreading Processors

Simultaneous multithreading (SMT) processors are widely used in high performance computing tasks. However, with the improved performance of the SMT architecture, the utilization of their functional units is significantly increased, straining the power budget of the processor. This increases not only the dynamic power consumption, but also the leakage power consumption due to the increased temperature. In this paper, a comparison of the static and dynamic sleep signal generation techniques for SMT processors is presented. This is conducted under various workloads to assess their effectiveness in leakage power management. Results show that the dynamic approach exhibits a threefold increase in leakage savings, compared with that of the static approach for certain functional units.

Probabilistic job symbiosis modeling for SMT processor scheduling

Symbiotic job scheduling boosts simultaneous multithreading (SMT) processor performance by co-scheduling jobs that have `compatible' demands on the processor's shared resources. Existing approaches however require a sampling phase, evaluate a limited number of possible co-schedules, use heuristics to gauge symbiosis, are rigid in their optimization target, and do not preserve system-level priorities/shares. This paper proposes probabilistic job symbiosis modeling, which predicts whether jobs will create positive or negative symbiosis when co-scheduled without requiring the co-schedule to be evaluated. The model, which uses per-thread cycle stacks computed through a previously proposed cycle accounting architecture, is simple enough to be used in system software. Probabilistic job symbiosis modeling provides six key innovations over prior work in symbiotic job scheduling: (i) it does not require a sampling phase, (ii) it readjusts the job co-schedule continuously, (iii) it evaluates a large number of possible co-schedules at very low overhead, (iv) it is not driven by heuristics, (v) it can optimize a performance target of interest (e.g., system throughput or job turnaround time), and (vi) it preserves system-level priorities/shares. These innovations make symbiotic job scheduling both practical and effective. Our experimental evaluation, which assumes a realistic scenario in which jobs come and go, reports an average 16% (and up to 35%) reduction in job turnaround time compared to the previously proposed SOS (sample, optimize, symbios) approach for a two-thread SMT processor, and an average 19% (and up to 45%) reduction in job turnaround time for a four-thread SMT processor.