Simultaneous Multithreading Architecture Research Articles

Memory Dependence Speculation for Simultaneous Multi-Threading Processors

Simultaneous Multi-Threading (SMT) processors provide improvement over the traditional out-of-order superscalar architecture by allowing instructions from several independent threads to execute out-of-order concurrently. Maintaining the accuracy of values read from and written to memory is a great bottleneck for processor performance, as loads must stall execution until all prior store addresses are known or risk reading invalid data. Prior research in this area has mainly focused on superscalar architecture, as such, it is only natural to extend memory dependence speculation techniques to an SMT architecture. In this paper, we allow for loads among threads to execute as soon as their addresses are resolved without checking for prior memory address conflicts. Stores also perform a check on all later loads to see if any read was too early due to an address match, if so, the processor state is recovered, and the load re-issued. This aggressive technique allows for the greatest potential instructions per clock cycle gains over predictive techniques as the pipeline is never stalled for loads. Our simulations show that an overall IPC gain up to 12% and 10% is possible for both 4-threaded and 8-threaded workloads respectively. Conversely, a maximum overall IPC loss of at least 2.3% and 2% for 4-threaded and 8-threaded workloads respectively was also observed.

Speculative inter-thread store-to-load forwarding in SMT architectures

Applications running on out-of-order cores have benefited for decades of store-to-load forwarding which accelerates communication of store values to loads of the same thread. Despite threads running on a simultaneous multithreading (SMT) core could also access the load queues (LQ) and store queues (SQ) / store buffers (SB) of other threads to allow inter-thread store-to-load forwarding, we have skipped exploiting it because if we allow communication of different SMT threads via their LQs and SQs/SBs, write atomicity may be violated with respect to the outside world beyond the acceptable model of read-own-write-early multiple-copy atomicity (rMCA).In our prior work, we leveraged this idea to propose inter-thread store-to-load forwarding (ITSLF). ITLSF accelerates synchronization and communication of threads running in a simultaneous multi-threading processor by allowing stores in the store-queue of a thread to forward data to loads of another thread running in the same core without violating rMCA.In this work, we extend the original ITSLF mechanism to allow inter-thread forwarding from speculative stores (Spec-ITSLF). Spec-ITSLF allows forwarding store values to other threads earlier, which further accelerates synchronization. Spec-ITSLF outperforms a baseline SMT core by 15%, which is 2% better on average (and up to 5% for the TATP workload) than the original ITSLF mechanism. More importantly, Spec-ITSLF is on par with the original ITSLF mechanism regarding storage overhead but does not need to keep track of the speculative state of stores, which was an important source of overhead and complexity in the original mechanism.

A simultaneous multithreading processor architecture with predictable timing behavior

Real-time embedded systems need software and hardware to be time-predictable to guarantee the correct behavior of the system. Precision Timed Machines are architectures designed for timing predictability and repeatability. They help to improve design time and the efficiency of real-time embedded systems by allowing to separately verify the timing properties of modules. This paper presents a Simultaneous Multithreading Precision Timed Machine named Hivek-RT that can execute hard real-time and conventional threads in parallel. It employs a repeatable thread-interleaved pipeline with an exposed memory hierarchy composed of scratchpads, caches, and a predictable SDRAM memory controller. The proposed architecture is well suited for real-time embedded systems as experimentation results show that the proposed architecture has improved throughput, presents low memory footprint and achieve a memory bandwidth of 90% of the theoretical value while providing deterministic time access to the memory hierarchy. This paper is an extended version of the paper presented on the 8th Brazilian Symposium on Computing Systems Engineering.

Assessing Parallel Thread Mapping Approaches on Shared Memory SMT Architectures

For better performance in multi-core architectures, when running parallel applications, thread mapping has become a relevant task. One way to explore this approach is to consider the memory hierarchy and the existing communication between threads. However, thread mapping is an NP-Hard problem and, therefore, heuristics have been studied to find solutions that are closer to optimal. In this context, the goal of this work is to evaluate the use of the BRD, Greedy and K-means heuristics for mapping threads in multi-core architectures with shared memory and simultaneous multithreading (SMT) support. All heuristics present performance improvements. The results show gains of up to 40.77% (with SMT) in execution time and 46.81% (with SMT) in power consumption when compared to the default Linux mapping strategy. Concerning all scenarios, SMT represents 72.22% of results with better execution time and 44.44% of results with better energy consumption.

Susan Eggers Receives Eckert-Mauchly Award for Outstanding Contributions to Computer Architecture

Susan Eggers, professor emerita at the University of Washingtons Paul G. Allen School of Computer Science & Engineering, has been honored with the 2018 Eckert-Mauchly Award in recognition of her outstanding contributions to the field of computer architecture. The award, which is sponsored by the ACM and IEEE Computer Society, is the computer architecture communitys most prestigious honor. Eggers was cited in particular for her work on simultaneous multithreaded processor architectures and multiprocessor memory sharing and coherency. Eggers is the first woman to receive the Eckert-Mauchly Award in its 39-year history.

A controlled fetching technique for effective management of shared resources in SMT processors

Simultaneous Multi-Threading (SMT) is a processor design technique that supports concurrent execution of instructions from multiple threads in every cycle by sharing the key datapath components. In the SMT architecture, the shared resources normally include the physical register file, Issue Queue (IQ), functional units, write buffer and the cache memory. Efficient utilization of the shared resources is critical to achieving high-performance gain. The physical rename register file is one of the most critical shared resources in the SMT architecture due to its being located at forefront of the pipeline stages. The inter-thread sharing of the physical registers reduces the number of registers required in the SMT processors than would have been needed in deploying multiple superscalar processors to achieve a similar throughput. However, due to the nature of sharing, an overwhelming occupancy of the physical register file by any slower threads can lead to a shortage of registers available for the other threads in the system and thus degrade the overall performance. In this paper, we propose an intelligent fetching algorithm for efficient management of the shared physical register file. Even though the primary focus of this paper is to manage the physical register file effectively, it indirectly controls the other shared resources downstream in the pipeline as well. The main goal of this paper is to propose a simple resource management scheme capable of achieving a considerable performance gain that neither incurs a substantial processing or hardware overhead for practical implementation nor requires modifications in the other pipeline stages. We demonstrate that temporarily suspending the slow threads from the system in the fetch stage can improve the overall system performance by a significant margin. An improvement of up to 63% and 68% is achieved when the proposed scheme is applied to the 4-threaded and the 8-threaded system respectively. The throughput of an 8-threaded system with 320 register file entries is significantly higher than the performance of default system with 416 register entries indicating a resource saving of 60%.

The IBM z13 multithreaded microprocessor

The IBM z13™ system is the latest generation of the IBM z Systems™ mainframes. The z13 microprocessor improves upon the IBM zEnterprise® EC12 (zEC12) processor with two vector execution units, higher instruction execution parallelism, and a simultaneous multithreaded (SMT) architecture that supports concurrent execution of two threads. These advances yield performance gains in legacy online transaction processing and business analytics workloads. This latest generation system features an eight-core processor chip, a robust cache hierarchy, and large multiprocessor system design optimized for enterprise database and transaction processing workloads. The microprocessor core features a wide super-scalar, out-of-order pipeline that can sustain an instruction fetch, decode, dispatch, and completion rate of six z/Architecture® instructions per cycle. The instruction execution path is predicted by multi-level branch direction and target prediction logic. Complex instructions are split into two or more simpler micro-operations. Instructions are issued out of program order from an instruction issue queue to multiple RISC (reduced instruction set computer) execution units. The super-scalar design can sustain an issue and execution rate of ten micro-operations per cycle: two load/store type instructions, four fixed point (integer) instructions, two floating point or vector instructions, and two branch instructions.

Reliability‐aware simultaneous multithreaded architecture using online architectural vulnerability factor estimation

Miniaturisation in modern microprocessors increases susceptibility to soft errors leading to reliability degradation. Recently simultaneous multithreaded (SMT) architecture is utilised to improve fault tolerance. Despite full coverage, redundant checking in such schemes causes significant performance and energy overheads. Fortunately, some of the soft errors can be masked at the architectural level and architectural vulnerability factor (AVF) of a structure represents the portion of soft errors which lead to a failure in the output of a program. In this study, the authors present an infrastructure for online monitoring of AVF of sensitive structures of an SMT processor. Based on estimated AVF, we have introduced partial thread redundancy (PTR) protection scheme for intervals whose AVF is greater than a predefined threshold and the estimated AVF is used for adaptation between reliability improvement or performance enhancement, especially when the processor executes more than one workload. We have utilised SPEC CPU2006 benchmarks for AVF estimation of some important hardware resources such as issue queue, reorder buffer, load/store queue and register file. Experimental results show that the mean absolute error of our AVF estimation method varies from 0.04 to 0.07 and combined online AVF estimation and PTR protection, leads to a reliability aware execution and lower performance overhead.

Effect of Thread Weight Readjustment Scheduler on Scheduling Criteria

In this paper, we propose a new approach to significantly optimize some scheduling criteria, context switches and turnaround times in this context, when running multithreaded processes concurrently. Current proportional sharing schedulers allocate CPU time based on the weights of running threads in the system and don’t take into consideration the greedy behavior of multithreaded processes (processes with more threads receive more aggregate CPU time from the scheduler relative to processes with fewer threads). In order to minimize turnaround times and context switches of running multithreaded processes in simultaneous multithreaded (SMT) architectures, we investigate the effect of adjusting the weights of running sibling threads (threads forked from the same process) using novel proportional sharing scheduler, Thread Weight Readjustment Scheduler (TWRS), a proportional share CPU scheduler designed for multithreaded processes, which aims to reduce undesirable events (e.g. context switches) and turnaround times. TWRS provides a practical solution for multitasking operating systems because it operates in concert with existing kernels. We have implemented TWRS in Linux 2.6.24-1, which represents the most prevalent scheduler design (i.e. Completely Fair Scheduler (CFS)). Our evaluation shows that our scheduler minimizes context switches and turnaround time.

A small and power efficient checkpoint core architecture for manycore processors

This article describes and evaluates a small, out-of-order, simultaneous multithreaded SMT core architecture suitable for power constrained microprocessors, such as manycore microprocessors for high performance computing. The architecture does not require a reorder buffer ROB or physical registers for register renaming and instruction retirement. Instead, it uses a large number of virtual register IDs for register renaming, and a logical register file with multiple contexts. The architecture improves total thread execution throughput using two register contexts to support SMT execution of parallel workloads. Moreover, the architecture improves instruction level parallelism ILP and execution performance when running single-thread applications. In addition to eliminating the reorder buffer and the physical renaming register file, the architecture minimises the logical register file hardware by using the two SMT register contexts and in-cell register file context fusion mechanism for recovering from branch mispredictions. We present results from Spec 2006 benchmarks running on a SimpleScalar performance simulator of our architecture. Our simulation measurements show 5% single-thread performance improvement and 9.6% 2-thread SMT performance improvement over a conventional SMT core architecture with reorder buffer.

A resource utilization based instruction fetch policy for SMT processors

Simultaneous Multithreading (SMT) architectures are proposed to better explore on-chip parallelism, which capture the essence of performance improvement in modern processors. SMT overcomes the limits in a single thread by fetching and executing from multiple of them in a shared fashion. The long-latency operations, however, still cause inefficiency in SMT processors. When instructions have to wait for data from lower-level memory hierarchy, the dependent instructions cannot proceed, hence continue occupying the shared resources on the chip for an extended number of clock cycles. This introduces undesired inter-thread interference in SMT processors, which further leads to negative impacts on overall system throughput and average thread performance. In practice, instruction fetch policies take the responsibility of assigning thread priority at the fetch stage, in an effort to better distribute the shared resources among threads in the same core to cope with the long-latency operations and other runtime behavior from the thread for better performance.In this paper we propose an instruction fetch policy RUCOUNT, which considers resource utilization of individual thread in the prioritization process. The proposed policy observes instructions in the front-end stages of the pipeline as well as low-level data misses to summarize the resource utilization for thread management. Higher priority is granted to the thread(s) with less utilized resources, such that overall resources are distributed more efficiently in SMT processors. As a result, it has two unique features compared to other policies: one is to observe the hardware resource comprehensively and the other is to monitor limited resource entries. Our experimental results demonstrate that RUCOUNT is 20% better than ICOUNT, 10% than Stall, 8% than DG and 3% than DWarn, in terms of averaged performance. Considering its hardware overhead is at the similar level as ICOUNT and DWarn, our proposed instruction fetch policy RUCOUNT is superior among the studied policies.

Revisiting LP-NUCA Energy Consumption

Cache working-set adaptation is key as embedded systems move to multiprocessor and Simultaneous Multithreaded Architectures (SMT) because interthread pollution harms system performance and battery life. Light-Power NUCA (LP-NUCA) is a working-set adaptive cache that depends on temporal-locality to save energy. This work identifies the sources of energy waste in LP-NUCAs: parallel access to the tag and data arrays of the tiles and low locality phases with useless block migration. To counteract both issues, we prove that switching to serial access reduces energy without harming performance and propose a machine learning Adaptive Drop Rate (ADR) controller that minimizes the amount of replacement and migration when locality is low. This work demonstrates that these techniques efficiently adapt the cache drop and access policies to save energy. They reduce LP-NUCA consumption 22.7% for 1SMT. With interthread cache contention in 2SMT, the savings rise to 29%. Versus a conventional organization, energy--delay improves 20.8% and 25% for 1- and 2SMT benchmarks, and, in 65% of the 2SMT mixes, gains are larger than 20%.

Performance assessment of multithreaded quicksort algorithm on simultaneous multithreaded architecture

Sorting huge amounts of datasets have become essential in many computer applications, such as search engines, database and web-based applications, in order to improve searching performance. Moreove...

Network Applications on Simultaneous Multithreading Processors

As network applications become increasingly sophisticated and Internet traffic is getting heavier, future network processors must continue processing computation-intensive network applications at line rates. Most programmable network processors on the market today, such as the Intel IXP2800, target relatively low performance (from 100 Mbps to 10 Gbps). However, low cost edge routers will find it hard to cope with the forthcoming sophistication of network applications to be processed at those speeds. Hence, new architectures should be designed for the programmable network processors of the future. The goal of this paper is to evaluate the applicability and efficiency of Simultaneous MultiThreaded (SMT) as the base architecture of a network processor. Indeed, the SMT model inherently allows the multiple parallel threads which must be dealt with in network processor applications. In this paper, we investigate the architectural implications of network applications on the SMT architecture. We demonstrate that, when executed as independent threads, applications chosen from different network layers show an improved Instructions Per Cycle (IPC) and cache behavior when compared with the situation where the program executed comes from a single network application. Finally, a new architectural solution to cope with packet dependency is proposed and evaluated.

On the Power Management of Simultaneous Multithreading Processors

Simultaneous multithreading (SMT) processors are widely used in high performance computing tasks. However, with the improved performance of the SMT architecture, the utilization of their functional units is significantly increased, straining the power budget of the processor. This increases not only the dynamic power consumption, but also the leakage power consumption due to the increased temperature. In this paper, a comparison of the static and dynamic sleep signal generation techniques for SMT processors is presented. This is conducted under various workloads to assess their effectiveness in leakage power management. Results show that the dynamic approach exhibits a threefold increase in leakage savings, compared with that of the static approach for certain functional units.

Architecture d'un processeur multiflot orienté temps-réel

Simultaneous multithreading (SMT) processors might be good candidates to fulfill the ever increasing performance needs of embedded applications. However, off-the-shelves SMT architectures do not fit the timing predictability requirements of hard real-time systems: to schedule critical threads so that they are guaranteed to meet their deadlines, it is necessary to estimate their Worst-Case Execution Times which is not possible when simultaneous threads might interfere. In this paper, we propose an SMT architecture designed to enforce isolation of hard real-time threads so that their worst-case execution time can be safely estimated. We report experimental results that show that this architecture still provides a high level of performance and we give an insight into how the thread isolation feature could be controlled by a real-time task scheduler.

Algorithm, software, and hardware optimizations for Delaunay mesh generation on simultaneous multithreaded architectures

This article focuses on the optimization of PCDM, a parallel, two-dimensional (2D) Delaunay mesh generation application, and its interaction with parallel architectures based on simultaneous multithreading (SMT) processors. We first present the step-by-step effect of a series of optimizations on performance. These optimizations improve the performance of PCDM by up to a factor of six. They target issues that very often limit the performance of scientific computing codes. We then evaluate the interaction of PCDM with a real SMT-based SMP system, using both high-level metrics, such as execution time, and low-level information from hardware performance counters.

Optimizing Instruction Scheduling through Combined In-Order and O-O-O Execution in SMT Processors

The resource sharing nature of Simultaneous Multithreading (SMT) processors and the presence of long latency instructions from concurrent threads make the instruction scheduling window (IW), which is a primary shared component among key pipeline structures in SMT, a performance bottleneck. Due to the tight constraints on its physical size, the IW faces more severe pressure to handle the instructions from various threads while attempting to avoid resource monopolization by some low-ILP threads. It is particularly challenging to optimize the efficiency and fairness in IW utilization to fulfill the affordable performance by SMT under the shadow of long latency instructions. Most of the existing optimization schemes in SMT processors rely on the fetch policy to control the instructions that are allowed to enter the pipeline, while little effort is put to control the long latency instructions that are already located in the IW. In this paper, we propose streamline buffers to handle the long latency instructions that have already entered the pipeline and clog the IW, while the controlling fetch policies take time to react. Each streamline buffer extracts from IW and holds a chain of instructions from a thread that are stalled by dependency on a long latency load. When the load value returns, the streamline buffer then serves these instructions directly to in-order execution, avoiding any instruction replay. This is done in supplement to the conventional IW that serves in parallel the other instructions for out-of-order (o-o-o) execution. Analysis of SPEC2000 integer and FP benchmarks reveals that instructions dependent on long latency loads, typically have their first source operand ready within 5 percent-15 percent of their total wait time in the IW. Our scheme is able to utilize this asymmetry in source operands' ready time to achieve a complexity effective design. As compared to the baseline SMT architecture, our design when working in conjunction with earlier proposed ICOUNT.2.8 fetch policy for 4-threads effectively reduces the IW full rate by 9.4 percent (11 percent for 2-thread), improves average IPC for MIXED workloads by 9.6 percent (8 percent for MEM workloads and 4.4 percent for CPU workloads), and fairness by 7.56 percent (7.24 percent for 2-thread). Similar enhancements are observed when run in conjunction with an RR.2.8 fetch policy. Further, our scheme when combined with DCRA improves the performance on the average by 21.7 percent, while DCRA improves by 16.3 percent when run alone.

Increasing data reuse of sparse algebra codes on simultaneous multithreading architectures

AbstractIn this paper the problem of the locality of sparse algebra codes on simultaneous multithreading (SMT) architectures is studied. In these kind of architectures many hardware structures are dynamically shared among the running threads. This puts a lot of stress on the memory hierarchy, and a poor locality, both inter‐thread and intra‐thread, may become a major bottleneck in the performance of a code. This behavior is even more pronounced when the code is irregular, which is the case of sparse matrix ones. Therefore, techniques that increase the locality of irregular codes on SMT architectures are important to achieve high performance. This paper proposes a data reordering technique specially tuned for these kind of architectures and codes. It is based on a locality model developed by the authors in previous works. The technique has been tested, first, using a simulator of a SMT architecture, and subsequently, on a real architecture as Intel's Hyper‐Threading. Important reductions in the number of cache misses have been achieved, even when the number of running threads grows. When applying the locality improvement technique, we also decrease the total execution time and improve the scalability of the code. Copyright © 2009 John Wiley & Sons, Ltd.

Exploiting selective instruction reuse and value prediction in a superscalar architecture

In our previously published research we discovered some very difficult to predict branches, called unbiased branches. Since the overall performance of modern processors is seriously affected by misprediction recovery, especially these difficult branches represent a source of important performance penalties. Our statistics show that about 28% of branches are dependent on critical Load instructions. Moreover, 5.61% of branches are unbiased and depend on critical Loads, too. In the same way, about 21% of branches depend on MUL/DIV instructions whereas 3.76% are unbiased and depend on MUL/DIV instructions. These dependences involve high-penalty mispredictions becoming serious performance obstacles and causing significant performance degradation in executing instructions from wrong paths. Therefore, the negative impact of (unbiased) branches over global performance should be seriously attenuated by anticipating the results of long-latency instructions, including critical Loads. On the other hand, hiding instructions' long latencies in a pipelined superscalar processor represents an important challenge itself. We developed a superscalar architecture that selectively anticipates the values produced by high-latency instructions. In this work we are focusing on multiply, division and loads with miss in L1 data cache, implementing a dynamic instruction reuse scheme for the MUL/DIV instructions and a simple last value predictor for the critical Load instructions. Our improved superscalar architecture achieves an average IPC speedup of 3.5% on the integer SPEC 2000 benchmarks, of 23.6% on the floating-point benchmarks, and an improvement in energy-delay product (EDP) of 6.2% and 34.5%, respectively. We also quantified the impact of our developed selective instruction reuse and value prediction techniques in a simultaneous multithreaded architecture (SMT) that implies per thread reuse buffers and load value prediction tables. Our simulation results showed that the best improvements on the SPEC integer applications have been obtained with 2 threads: an IPC speedup of 5.95% and an EDP gain of 10.44%. Although, on the SPEC floating-point programs, we obtained the highest improvements with the enhanced superscalar architecture, the SMT with 3 threads also provides an important IPC speedup of 16.51% and an EDP gain of 25.94%.