Simultaneous Multithreading Processors Research Articles

Memory Dependence Speculation for Simultaneous Multi-Threading Processors

Simultaneous Multi-Threading (SMT) processors provide improvement over the traditional out-of-order superscalar architecture by allowing instructions from several independent threads to execute out-of-order concurrently. Maintaining the accuracy of values read from and written to memory is a great bottleneck for processor performance, as loads must stall execution until all prior store addresses are known or risk reading invalid data. Prior research in this area has mainly focused on superscalar architecture, as such, it is only natural to extend memory dependence speculation techniques to an SMT architecture. In this paper, we allow for loads among threads to execute as soon as their addresses are resolved without checking for prior memory address conflicts. Stores also perform a check on all later loads to see if any read was too early due to an address match, if so, the processor state is recovered, and the load re-issued. This aggressive technique allows for the greatest potential instructions per clock cycle gains over predictive techniques as the pipeline is never stalled for loads. Our simulations show that an overall IPC gain up to 12% and 10% is possible for both 4-threaded and 8-threaded workloads respectively. Conversely, a maximum overall IPC loss of at least 2.3% and 2% for 4-threaded and 8-threaded workloads respectively was also observed.

Intelligent Thread-Specific Rename Register Allocation for Simultaneous Multi-Threading Processors Based on Cache Behavior

Intelligent Thread-Specific Rename Register Allocation for Simultaneous Multi-Threading Processors Based on Cache Behavior

Speculative inter-thread store-to-load forwarding in SMT architectures

Applications running on out-of-order cores have benefited for decades of store-to-load forwarding which accelerates communication of store values to loads of the same thread. Despite threads running on a simultaneous multithreading (SMT) core could also access the load queues (LQ) and store queues (SQ) / store buffers (SB) of other threads to allow inter-thread store-to-load forwarding, we have skipped exploiting it because if we allow communication of different SMT threads via their LQs and SQs/SBs, write atomicity may be violated with respect to the outside world beyond the acceptable model of read-own-write-early multiple-copy atomicity (rMCA).In our prior work, we leveraged this idea to propose inter-thread store-to-load forwarding (ITSLF). ITLSF accelerates synchronization and communication of threads running in a simultaneous multi-threading processor by allowing stores in the store-queue of a thread to forward data to loads of another thread running in the same core without violating rMCA.In this work, we extend the original ITSLF mechanism to allow inter-thread forwarding from speculative stores (Spec-ITSLF). Spec-ITSLF allows forwarding store values to other threads earlier, which further accelerates synchronization. Spec-ITSLF outperforms a baseline SMT core by 15%, which is 2% better on average (and up to 5% for the TATP workload) than the original ITSLF mechanism. More importantly, Spec-ITSLF is on par with the original ITSLF mechanism regarding storage overhead but does not need to keep track of the speculative state of stores, which was an important source of overhead and complexity in the original mechanism.

VMT: Virtualized Multi-Threading for Accelerating Graph Workloads on Commodity Processors

Modern-day graph workloads operate on huge graphs through pointer chasing which leads to high last-level cache (LLC) miss rates and limited memory-level parallelism (MLP). Simultaneous Multi-Threading (SMT) effectively hides the memory access latencies for multi-threaded graph workloads provided that sufficient threads are supported in hardware. Unfortunately, providing a sufficiently large number of physical threads incurs an unjustifiably high hardware cost for commodity SMT processors which typically implement only two physical hardware threads. Ideally, we would like to achieve aggressive-SMT performance when running graph workloads on modest commodity processors. In this paper, we propose Virtualized Multi-Threading (VMT), a low-overhead multi-threading paradigm for accelerating graph workloads on commodity processors. Unlike prior multi-threading paradigms, VMT virtualizes both the physical hardware threads and the architecture state: VMT maps a large number of logical software threads to a small number of physical hardware threads, while maintaining the architecture state of the logical threads in the processor's cache hierarchy. Implemented on top of a quad-core 2-way SMT processor, VMT achieves an average speedup of 1.74× for a set of representative graph workloads, while incurring minimal hardware cost (195 bytes per core to support up to 32 logical threads). VMT's low hardware cost paves the way for implementation in commodity processors.

Secure cache partitioning in simultaneous multi-threading processors for fairness

Bulut bilişimin yaygın kullanımı, sanallaştırma ve uzaktan çalıştırma nedeniyle sunucu kaynaklarının birden fazla bilgisayar tarafından paylaşılması, donanımı paylaşılan bir kaynak haline getirmiş,ve donanım güvenliği daha fazla önem kazanmaya başlamıştır. Meltdown ve Spectre saldırılarında olduğu gibi önbellek tabanlı yan kanal saldırıları yoluyla şifreleme anahtarları gibi gizli bilgileri çıkarmak mümkündür. Güvenli donanım çalışmaları içinde, güvenli önbellek mimarileri üzerine yapılan çalışmalar daha derin bir odak noktası haline geldi. Güvenli önbellek mimarileri kaçınılmaz bir performans kaybıyla birlikte gelir, çünkü her zaman saldırganlardan bilgi gizlemenin bir işleme ya da saklama maliyeti vardır. Performansın düşüşünü kullanıcılar arasında adil bir şekilde paylaştırmak, güvenlik çözümlerinde göz ardı edilmiştir. Bu çalışma ile birlikte, mevcut mimarilerin performans bozulmalarını analiz edip, genel çerçevenin adaletliliğini artıran yeni bir çözüm sunuyoruz. Önbellek tabanlı yan kanal saldırılarına karşı, bir işlemci içindeki rakip iş parçacıkları arasında adalete saygı duyan güvenli bir önbellek mekanizması öneriyoruz. FairSDP mimarisini 4 iş parçacıklı ve 8 iş parçacıklı işlemcilerde değerlendirdik. Sonuç olarak, 4 iş parçacıklı bir sistemde ortalama olarak SecDCP (Güvenli ve Dinamik Önbellek Bölümleme) mimarisine kıyasla yüzde 38,6 performans artışı elde ediyoruz. 8 iş parçacıklı bir sistemde taban çizgisine göre yüzde 8,7'ye kadar performans iyileştirmesi, statik bölümlemeye kıyasla yüzde 9,2 daha iyi performans ve ortalama olarak SecDCP'ye göre yüzde 14,1 performans artışı sağlayabildiğimizi gösteriyoruz. Ayrıca, güvenli olmayan bir dinamik önbellek bölümleme tekniğine kıyasla adalet metriği açısından hemen hemen aynı sonuçları elde ederiz.

Mitigating execution unit contention in parallel applications using instruction‐aware mapping

SummaryParallel applications running on simultaneous multithreading (SMT) processors naturally compete for execution units when their threads are mapped to the same core. This issue is further aggravated when such threads execute similar instructions that stress the same execution unit type, making their execution to behave very similarly as if the threads were running sequentially. This, in turn, will lead to performance degradation and underutilization of hardware resources. This work proposes a completely transparent framework (no modifications to the source code are necessary) that automatically maps threads of multiple parallel applications on SMT processors. The framework focuses on improving performance by mitigating the contention on execution units, considering each thread's instruction types, which are detected at runtime by our framework. Results show performance gains of 21% (geometric mean), compared to the native scheduler of the operating system.

Dynamic issue queue capping for simultaneous multithreaded processors

A simultaneous multithreaded (SMT) processor mixes multiple instruction streams in its superscalar out-of-order execution core for higher throughput. To achieve this, a superscalar processor is modified in such a way that someof its resources are duplicated and the rest is shared among multiple threads. The issue queue (IQ), which holds allwaiting instructions until they become ready and scheduled for execution, is among these shared resources. A baselineunmanaged IQ can give an unexpectedly low performance since a hungry thread can tie up most of the IQ entries.This type of scenario is also worse in terms of the fairness metric since some of the threads may experience starvation.Earlier studies propose both static and a limited type of dynamic capping of the IQ entries for regulating IQ traffic andproviding better SMT throughput and fairness. In this study, we propose an e?iciency-based dynamic capping (EDC)algorithm that calculates an efficiency metric for each thread and allocates the IQ entries for maximizing the throughputand the fairness metrics. EDC gives 3.6% better throughput and 3.9% better fairness results compared to the currentstate-of-the-art algorithms, on the average.

Hy-Sched: A Simple Hyperthreading-Aware Thread to Core Allocation Strategy

Simultaneous multithreading processors are dominating the High Computing Performance market. Among these processors, those supporting only two threads are being the most widely deployed in current systems, thus, only two threads compete at run-time for intra-core resources. The performance of these processors can be boosted by selecting symbiotic applications to be executed on the same core, which reduces the inter-application interference considerably. In this letter we propose Hy-Sched, an scheduling algorithm that exploits symbiosis to make pairs of applications to be launched on the same physical core. The proposed approach lies on the categories of the Top-Down Method for Performance Analysis. Different variants of the algorithm are explored. Experimental results show that Hy-Sched outperforms Linux on average by 15 percent in the studied workloads.

Hyperheuristics for explicit resource partitioning in simultaneous multithreadedprocessors

In simultaneous multithreaded (SMT) processors, various data path resources are concurrently shared by many threads. A few heuristic approaches that explicitly distribute those resources among threads with the goal of improved overall performance have already been proposed. A selection hyperheuristic is a high-level search methodology that mixes a predetermined set of heuristics in an iterative framework to utilize their strengths for solving a given problem instance. In this study, we propose a set of selection hyperheuristics for selecting and executing the heuristic with the best performance at a given stage. To the best of our knowledge, this is one of the first studies implementing a hyperheuristic algorithm on hardware. The results of our experimental study show that hyperheuristics are indeed capable of improving the performance of the studied workloads. Our best performing hyperheuristic achieves better throughput than both baseline heuristics in 5 out of 12 workloads and gives about 15% peak performance gain. The average performance gains over the well-known hill-climbing and adaptive resource partitioning heuristics are about 5% and 2%, respectively.

Thread Isolation to Improve Symbiotic Scheduling on SMT Multicore Processors

Resource sharing is a critical issue in simultaneous multithreading (SMT) processors as threads running simultaneously on an SMT core compete for shared resources. Symbiotic job scheduling, which co-schedules applications with complementary resource demands, is an effective solution to maximize hardware utilization and improve overall system performance. However, symbiotic job scheduling typically distributes threads evenly among cores, i.e., all cores get assigned the same number of threads, which we find to lead to sub-optimal performance. In this paper, we show that asymmetric schedules (i.e., schedules that assign a different number of threads to each SMT core) can significantly improve performance compared to symmetric schedules. To leverage this finding, we propose thread isolation , a technique that turns symmetric schedules into asymmetric ones yielding higher overall system performance. Thread isolation identifies SMT-adverse applications and schedules them in isolation on a dedicated core to mitigate their sharp performance degradation under SMT. Our experimental results on an IBM POWER8 processor show that thread isolation improves system throughput by up to 5.5 percent compared to a state-of-the-art symmetric symbiotic job scheduler.

A simultaneous multithreading processor architecture with predictable timing behavior

Real-time embedded systems need software and hardware to be time-predictable to guarantee the correct behavior of the system. Precision Timed Machines are architectures designed for timing predictability and repeatability. They help to improve design time and the efficiency of real-time embedded systems by allowing to separately verify the timing properties of modules. This paper presents a Simultaneous Multithreading Precision Timed Machine named Hivek-RT that can execute hard real-time and conventional threads in parallel. It employs a repeatable thread-interleaved pipeline with an exposed memory hierarchy composed of scratchpads, caches, and a predictable SDRAM memory controller. The proposed architecture is well suited for real-time embedded systems as experimentation results show that the proposed architecture has improved throughput, presents low memory footprint and achieve a memory bandwidth of 90% of the theoretical value while providing deterministic time access to the memory hierarchy. This paper is an extended version of the paper presented on the 8th Brazilian Symposium on Computing Systems Engineering.

Self-Spectre, Write-Execute and the Hidden State

Abstract The recent Meltdown and Spectre vulnerabilities have highlighted a very present and real threat in the on-chip memory cache units which can ultimately provide a hidden state, albeit only readable via memory timing instructions [Kocher, P.—Genkin, D.— Gruss, D.— Haas, W.—Hamburg, M.—Lipp, M.–Mangard, S.—Prescher, T.—Schwarz, M.—Yarom, Y.: Spectre attacks: Exploiting speculative execution, CoRR, abs/1801.01203, 2018]. Yet the exploits, although having some complexity and slowness, are demonstrably reliable on nearly all processors produced for the last two decades. Moving out from looking at this strictly as a means of reading protected memory, as the large microprocessor companies move to close this security vulnerability, an interesting question arises. Could the inherent design of the processor give the ability to hide arbitrary calculations in this speculative and parallel side channel? Without even using protected memory and exploiting the vulnerability, as has been the focus, there could very well be a whole class of techniques which exploit the side-channel. It could be done in a way which would be largely un-preventable behavior as the technology would start to become self-defeating or require a more complicated and expensive on-chip cache memory system to properly post-speculatively clean itself. And the ability to train the branch predictor to incorrectly speculatively behave is almost certain given hardware limitations, andthusprovidesexactly this pathway. A novel approach looks at just how much computation can be done speculatively with a result store via indirect reads and available through the memory cache. A multi-threaded approach can allow a multi-stage computation pipeline where each computation is passed to a read-out thread and then to the next computation thread [Swanson, S.—McDowell, L. K.—Swift, M. M.—Eggers, S. J.–Levy H. M.: An evaluation of speculative instruction execution on simultaneous multithreaded processors, ACM Trans. Comput. Syst. 21 (2003), 314–340]. Through channels like this, an application can surreptitiously make arbitrary calculations, or even leak data without any standard tracing tools being capable of monitoring the subtle changes. Like a variation of the famous physics Heisenberg uncertainty principle, even a tool capable of reading the cache states would not only be incredibly inefficient, but thereby tamper with and modify the state. Tools like in-circuit emulators, or specially designed cache emulators would be needed to unmask the speculative reads, and it is further difficult to visualize with a linear time-line. Specifically, the AES and RSA algorithms will be studied with respect to these ideas, looking at success rates for various calculation batches with speculative execution, while having a summary view to see the rather severe performance penalties for using such methods. Either approaches could provide for strong white-box cryptography when considering a binary, non-source code form. In terms of white-box methods, both could be significantly challenging to locate or deduce the inner workings of the code. Further, both methods can easily surreptitiously leak or hide data within shared memory in a seemingly innocuous manner.

Susan Eggers Receives Eckert-Mauchly Award for Outstanding Contributions to Computer Architecture

Susan Eggers, professor emerita at the University of Washingtons Paul G. Allen School of Computer Science & Engineering, has been honored with the 2018 Eckert-Mauchly Award in recognition of her outstanding contributions to the field of computer architecture. The award, which is sponsored by the ACM and IEEE Computer Society, is the computer architecture communitys most prestigious honor. Eggers was cited in particular for her work on simultaneous multithreaded processor architectures and multiprocessor memory sharing and coherency. Eggers is the first woman to receive the Eckert-Mauchly Award in its 39-year history.

System performance enhancement with thread suspension for simultaneous multi-threading processors

ABSTRACT Simultaneous Multi-Threading systems improve performance by allowing multiple independent threads to be executed concurrently with shared key resources. Unfair sharing of resources among multiple threads can easily clog the pipeline stages by slower threads and hamper the normal processing of faster threads. In this paper, we propose a thread-suspension scheme combined with a uniform register file capping technique to avoid imbalanced distribution of resources. When a shared resource congestion occurs on the pipeline stages, the thread with the lowest resource utilization efficiency among all active threads will be suspended so as to provide other threads more space to proceed for a higher throughput. Our experimental results show that our proposed technique improves the average system throughput in terms of Instruction Per Clock Cycle by up to 51.3%, with little compromise in execution fairness.

An integrated autonomous control mechanism to optimize sharing of multiple resources in Simultaneous Multi-Threading processors

Abstract Simultaneous Multi-Threading (SMT) processors allow concurrent execution of multiple independent threads with sharing of key datapath components to enhance system performance with more efficient resource utilization. However, due to the nature of SMT processing, imbalanced distribution of critical shared resources may easily clog some pipeline stages by slower threads and lead to unexpected system performance degradation. In this paper, an integrated autonomous control mechanism is proposed to optimize the use of two critical shared resources among multiple threads based on threads’ temporal behaviors in real time, by taking into account resource utilization snapshots at various stages of the pipeline. Our simulations based on M-Sim (Sharkey, 2005) [1] show that our proposed mechanism improves IPC (Instruction Per Clock Cycle) by a very significant margin (up to 100.4%) without sacrificing execution fairness among threads. Performance improvement from the proposed integrated approach on the two shared resources easily surpasses the aggregated benefits from two control mechanisms with each considering only one individual resource at a time.

A controlled fetching technique for effective management of shared resources in SMT processors

Simultaneous Multi-Threading (SMT) is a processor design technique that supports concurrent execution of instructions from multiple threads in every cycle by sharing the key datapath components. In the SMT architecture, the shared resources normally include the physical register file, Issue Queue (IQ), functional units, write buffer and the cache memory. Efficient utilization of the shared resources is critical to achieving high-performance gain. The physical rename register file is one of the most critical shared resources in the SMT architecture due to its being located at forefront of the pipeline stages. The inter-thread sharing of the physical registers reduces the number of registers required in the SMT processors than would have been needed in deploying multiple superscalar processors to achieve a similar throughput. However, due to the nature of sharing, an overwhelming occupancy of the physical register file by any slower threads can lead to a shortage of registers available for the other threads in the system and thus degrade the overall performance. In this paper, we propose an intelligent fetching algorithm for efficient management of the shared physical register file. Even though the primary focus of this paper is to manage the physical register file effectively, it indirectly controls the other shared resources downstream in the pipeline as well. The main goal of this paper is to propose a simple resource management scheme capable of achieving a considerable performance gain that neither incurs a substantial processing or hardware overhead for practical implementation nor requires modifications in the other pipeline stages. We demonstrate that temporarily suspending the slow threads from the system in the fetch stage can improve the overall system performance by a significant margin. An improvement of up to 63% and 68% is achieved when the proposed scheme is applied to the 4-threaded and the 8-threaded system respectively. The throughput of an 8-threaded system with 320 register file entries is significantly higher than the performance of default system with 416 register entries indicating a resource saving of 60%.

Real-time physical register file allocation with neural networks for simultaneous multi-threading processors

Simultaneous multi-threading (SMT) processors improve system performance by allowing concurrent execution of multiple independent threads with shared key resources. Physical register file, shared among the threads in real-time, is one of the most critical resources in deciding overall system performance. Disproportional distribution of registers among the threads may easily hamper normal processing of some threads. In this paper, we develop a machine learning algorithm to efficiently allocate registers among concurrent executing threads based on current resource utilisation circumstances. An offline training process is first employed to establish a well-trained neural network which is then applied to dynamically adjust the resource distribution in real-time. Our experiment results on M-sim, which is a multi-threaded micro-architectural simulation environment, show that our proposed technique significantly improves the average system throughput by up to 42% without sacrificing execution fairness among the threads.

Performance Enhancement with Speculative-Trace Capping at Different Pipeline Stages in Simultaneous Multi-Threading Processors

Performance Enhancement with Speculative-Trace Capping at Different Pipeline Stages in Simultaneous Multi-Threading Processors

Performance Enhancement with Speculative-Trace Capping at Different Pipeline Stages in Simultaneous Multi-Threading Processors

Performance Enhancement with Speculative-Trace Capping at Different Pipeline Stages in Simultaneous Multi-Threading Processors

Efficient resource sharing algorithm for physical register file in simultaneous multi-threading processors

Simultaneous Multi-Threading (SMT) processors increase performance by allowing concurrent execution of multiple independent threads with sharing of key datapath components and better utilization of the resources. An SMT processor usually maintains a shared register file to accommodate multiple threads for register renaming. By supporting inter-thread sharing of the physical registers, an SMT system processor can reduce the number of registers that would have been required in multiple superscalar processors while achieving a comparable throughput. However, congested shared resources due to slower threads can easily lead to inefficient usage of the resources and thus an undesirable performance outcome. In this paper, we propose an allocation algorithm at the architectural level for a better utilization of the shared register file. We show that, by limiting the number of the physical register entries each thread is allowed to occupy at any given time, the overall system throughput is enhanced by a substantial margin. An improvement in IPC of up to 44.6% and 32.7% is observed when the proposed technique is applied to a 4-threaded and a 6-threaded SMT system, respectively. Furthermore, a 4-threaded system with a physical register file of 160 entries can deliver a performance comparable to that of a default system with 256 entries, reflecting a resource saving of 37.5%.