Translation Lookaside Buffer Research Articles

Communication‐aware thread mapping using the translation lookaside buffer

SummaryThreads of parallel applications need to communicate in order to fulfill their tasks. The communication performance between the cores in modern multi‐core architectures differs because of the memory and interconnection hierarchies. In these architectures, it is important to map the threads of parallel applications by taking into account the communication between them, to improve their performance and energy consumption. In parallel applications based on shared memory, communication is implicit, which makes it difficult to detect the communication pattern between the threads.In this paper, we introduce a new lightweight mechanism to detect the communication pattern between threads of shared memory applications using the translation lookaside buffer. Our mechanism relies on hardware features, which make it transparent to the programmer and allow the detection to be performed by the operating system during the execution of the application. We also developed a heuristic mapping algorithm that uses the detected pattern to dynamically map the threads to cores. Experiments were performed with applications from the NAS‐OMP and PARSEC parallel benchmark suites in a simulated machine as well as a real machine. Results show that our mechanism can substantially improve parallel application performance, as well as processor and DRAM energy consumption. Copyright © 2015 John Wiley & Sons, Ltd.

Optimizing Memory Translation Emulation in Full System Emulators

The emulation speed of a full system emulator (FSE) determines its usefulness. This work quantitatively measures where time is spent in QEMU [Bellard 2005], an industrial-strength FSE. The analysis finds that memory emulation is one of the most heavily exercised emulator components. For workloads studied, 38.1% of the emulation time is spent in memory emulation on average, even though QEMU implements a software translation lookaside buffer (STLB) to accelerate dynamic address translation. Despite the amount of time spent in memory emulation, there has been no study on how to further improve its speed. This work analyzes where time is spent in memory emulation and studies the performance impact of a number of STLB optimizations. Although there are several performance optimization techniques for hardware TLBs, this work finds that the trade-offs with an STLB are quite different compared to those with hardware TLBs. As a result, not all hardware TLB performance optimization techniques are applicable to STLBs and vice versa. The evaluated STLB optimizations target STLB lookups, as well as refills, and result in an average emulator performance improvement of 24.4% over the baseline.

Fast Control Method of Software-Managed TLB for Reducing Zero-Copy Communication Overhead

SUMMARY Microkernel operating systems (OSes) use zero-copy communication to reduce the overhead of copying transfer data, because the communication between OS servers occurs frequently in the case of microkernel OSes. However, when a memory management unit manages the translation lookaside buffer (TLB) using software, TLB misses tend to increase the overhead of interprocess communication (IPC) between OS servers running on a microkernel OS. Thus, improving the control method of a software-managed TLB is important for microkernel OSes. This paper proposes a fast control method of software-managed TLB that manages page attachment in the area used for IPC by using TLB entries, instead of page tables. Consequently, TLB misses can be avoided in the area, and the performance of IPC improves. Thus, taking the SH-4 processor as an example of a processor having a software-managed TLB, this paper describes the design and the implementation of the proposed method for AnT oper

Software-Based Self-Test for Small Caches in Microprocessors

Nowadays, on-line testing is essential for modern microprocessors to detect latent defects that either escape manufacturing testing or appear during system operation. Small memories, such as L1 caches and translation lookaside buffers (TLBs) are not usually equipped with memory built-in self-test (MBIST) hardware. Software-based self-test (SBST) is a flexible and low-cost solution for on-line March test application and error detection in such small memories. Although, L1 caches and TLBs are small components, their reliable operation is crucial for the system performance due to the large penalties caused when L1 cache or TLB misses occur. In this paper, an SBST program development methodology is proposed for on-line testing of small cache memories in microprocessors. To overcome testability challenges that are due to the “hidden” or implicit operation of such memories, the proposed SBST methodology exploits: 1) existing special purpose instructions that modern instruction set architectures implement to access these cache arrays for debug-diagnostic (DD) purposes, termed hereafter direct cache access instructions and 2) performance monitoring and trap handling mechanisms. Besides, the proposed SBST methodology combines features that are crucial for on-line testing: a) compact test validation; b) simplified coding style; c) low invasiveness of the test program; and d) small memory footprint. The methodology is comprehensively demonstrated on the instruction and data L1 cache arrays and the instruction and data TLB arrays of OpenSPARC T1. Experimental results show that the exploitation of such DD instructions has a significant improvement of test time (up to 86% for instruction L1 cache, up to 87% for the data L1 cache, up to 37% for D-TLB, and up to 91% for I-TLB) when compared to SBST solutions that do not utilize these types of instructions.

Exploiting Existing Comparators for Fine-Grained Low-Cost Error Detection

Fault tolerance has become a fundamental concern in computer design, in addition to performance and power. Although several error detection schemes have been proposed to discover a faulty core in the system, these proposals could waste the whole core, including many error-free structures in it after error detection. Moreover, many fault-tolerant designs require additional hardware for data replication or for comparing the replicated data. In this study, we provide a low-cost, fine-grained error detection scheme by exploiting already existing comparators and data replications in the several pipeline stages such as issue queue, rename logic, and translation lookaside buffer. We reduce the vulnerability of the source register tags in IQ by 60%, the vulnerability of instruction TLB by 64%, the vulnerability of data TLB by 45%, and the vulnerability of the register tags of rename logic by 20%.

Automatic Charge Balancing Content Addressable Memory With Self-control Mechanism

Content addressable memory (CAM) is widely used in many applications, especially for those applications needing fast memory access. However, due to the parallel comparison feature and the frequent precharge/discharge of match-lines, the power consumption of CAM is considerable. In this paper, a fast and automatic charge balancing CAM architecture is proposed to control the voltage swing of match-lines so that the huge power consumption of CAM can be reduced. In the proposed design, the complementary property of the N-type CAM and the P-type CAM is used to balance the charge of match-lines. Especially, no specific sense amplifier is needed to detect the match status of match-lines. For a translation lookaside buffer example with 128 memory entries, the experimental result shows that 25.98% dynamic power and 3.30% leakage power can be reduced by using TSMC 90 nm CMOS process with 1.2 V supply voltage and only 2.23% circuit area is increased when compared with the traditional design.

Architectural support for address translation on GPUs

The proliferation of heterogeneous compute platforms, of which CPU/GPU is a prevalent example, necessitates a manageable programming model to ensure widespread adoption. A key component of this is a shared unified address space between the heterogeneous units to obtain the programmability benefits of virtual memory. To this end, we are the first to explore GPU Memory Management Units(MMUs) consisting of Translation Lookaside Buffers (TLBs) and page table walkers (PTWs) for address translation in unified heterogeneous systems. We show the performance challenges posed by GPU warp schedulers on TLBs accessed in parallel with L1 caches, which provide many well-known programmability benefits. In response, we propose modest TLB and PTW augmentations that recover most of the performance lost by introducing L1 parallel TLB access. We also show that a little TLB-awareness can make other GPU performance enhancements (e.g., cache-conscious warp scheduling and dynamic warp formation on branch divergence) feasible in the face of cache-parallel address translation, bringing overheads in the range deemed acceptable for CPUs (10-15\% of runtime). We presume this initial design leaves room for improvement but anticipate that our bigger insight, that a little TLB-awareness goes a long way in GPUs, will spur further work in this fruitful area.

Architectural support for address translation on GPUs

The proliferation of heterogeneous compute platforms, of which CPU/GPU is a prevalent example, necessitates a manageable programming model to ensure widespread adoption. A key component of this is a shared unified address space between the heterogeneous units to obtain the programmability benefits of virtual memory. To this end, we are the first to explore GPU Memory Management Units(MMUs) consisting of Translation Lookaside Buffers (TLBs) and page table walkers (PTWs) for address translation in unified heterogeneous systems. We show the performance challenges posed by GPU warp schedulers on TLBs accessed in parallel with L1 caches, which provide many well-known programmability benefits. In response, we propose modest TLB and PTW augmentations that recover most of the performance lost by introducing L1 parallel TLB access. We also show that a little TLB-awareness can make other GPU performance enhancements (e.g., cache-conscious warp scheduling and dynamic warp formation on branch divergence) feasible in the face of cache-parallel address translation, bringing overheads in the range deemed acceptable for CPUs (10-15\% of runtime). We presume this initial design leaves room for improvement but anticipate that our bigger insight, that a little TLB-awareness goes a long way in GPUs, will spur further work in this fruitful area.

Exploiting Early Tag Access for Reducing L1 Data Cache Energy in Embedded Processors

In this paper, we propose a new cache design technique, referred to as early tag access (ETA) cache, to improve the energy efficiency of data caches in embedded processors. The proposed technique performs ETAs to determine the destination ways of memory instructions before the actual cache accesses. It, thus, enables only the destination way to be accessed if a hit occurs during the ETA. The proposed ETA cache can be configured under two operation modes to exploit the tradeoffs between energy efficiency and performance. It is shown that our technique is very effective in reducing the number of ways accessed during cache accesses. This enables significant energy reduction with negligible performance overheads. Simulation results demonstrate that the proposed ETA cache achieves over 52.8% energy reduction on average in the L1 data cache and translation lookaside buffer. Compared with the existing cache design techniques, the ETA cache is more effective in energy reduction while maintaining better performance.

Exploiting Virtual Addressing for Increasing Reliability

A novel method to protect a system against errors resulting from soft errors occurring in the virtual address (VA) storing structures such as translation lookaside buffers (TLB), physical register file (PRF) and the program counter (PC) is proposed in this paper. The work is motivated by showing how soft errors impact the structures that store virtual page numbers (VPN). A solution is proposed by employing linear block encoding methods to be used as a virtual addressing scheme at link time. Using the encoding scheme to assign VPNs for VAs, it is shown that the system can tolerate soft errors using software with the help of the discussed decoding techniques applied to the page fault handler. The proposed solution can be used on all of the architectures using virtually indexed addressing. The main contribution of this paper is the decreasing of AVF for data TLB by 42.5%, instruction TLB by 40.3%, PC by 69.2% and PRF by 33.3%.

Reducing instruction fetch energy in multi-issue processors

The need to minimize power while maximizing performance has led to recent developments of powerful superscalar designs targeted at embedded and portable use. Instruction fetch is responsible for a significant fraction of microprocessor power and energy, and is therefore an attractive target for architectural power optimization. We present novel techniques that take advantage of guarantees so that the instruction translation lookaside buffer, branch target buffer, and branch prediction buffer can frequently be disabled, reducing their energy usage, while simultaneously reducing branch predictor contention. These techniques require no changes to the instruction set and can easily be integrated into most single- and multiple-issue processors.

Designing a practical data filter cache to improve both energy efficiency and performance

Conventional Data Filter Cache (DFC) designs improve processor energy efficiency, but degrade performance. Furthermore, the single-cycle line transfer suggested in prior studies adversely affects Level-1 Data Cache (L1 DC) area and energy efficiency. We propose a practical DFC that is accessed early in the pipeline and transfers a line over multiple cycles. Our DFC design improves performance and eliminates a substantial fraction of L1 DC accesses for loads, L1 DC tag checks on stores, and data translation lookaside buffer accesses for both loads and stores. Our evaluation shows that the proposed DFC can reduce the data access energy by 42.5% and improve execution time by 4.2%.

Hardware/software approaches for reducing the process variation impact on instruction fetches

As technology moves towards finer process geometries, it is becoming extremely difficult to control critical physical parameters such as channel length, gate oxide thickness, and dopant ion concentration. Variations in these parameters lead to dramatic variations in access latencies in Static Random Access Memory (SRAM) devices. This means that different lines of the same cache may have different access latencies. A simple solution to this problem is to adopt the worst-case latency paradigm. While this egalitarian cache management is simple, it may introduce significant performance overhead during instruction fetches when both address translation (instruction Translation Lookaside Buffer (TLB) access) and instruction cache access take place, making this solution infeasible for future high-performance processors. In this study, we first propose some hardware and software enhancements and then, based on those, investigate several techniques to mitigate the effect of process variation on the instruction fetch pipeline stage in modern processors. For address translation, we study an approach that performs the virtual-to-physical page translation once, then stores it in a special register, reusing it as long as the execution remains on the same instruction page. To handle varying access latencies across different instruction cache lines, we annotate the cache access latency of instructions within themselves to give the circuitry a hint about how long to wait for the next instruction to become available.

Area Efficient ROM-Embedded SRAM Cache

There are many important applications, such as math function evaluation, digital signal processing, and built-in self-test, whose implementations can be faster and simpler if we can have large on-chip “tables” stored as read-only memories (ROMs). We show that conventional de facto standard 6T and 8T static random access memory (SRAM) bit cells can embed ROM data without area overhead or performance degradation on the bit cells. Just by adding an extra wordline (WL) and connecting the WL to selected access transistor of the bit cell (based on whether a 0 or 1 is to be stored as ROM data in that location), the bit cell can work both in the SRAM mode and in the ROM mode. In the proposed ROM-embedded SRAM, during SRAM operations, ROM data is not available. To retrieve the ROM data, special write steps associated with proper via connections load ROM data into the SRAM array. The ROM data is read by conventional load instruction with unique virtual address space assigned to the data. This allows the ROM-embedded cache (R-cache) to bypass tag arrays and translation look-aside buffers, leading to fast ROM operations. We show example applications to illustrate how the R-cache can lead to low-cost logic testing and faster evaluation of mathematical functions.

Improving virtualization in the presence of software managed translation lookaside buffers

Virtualization has become an important technology that is used across many platforms, particularly servers, to increase utilization, multi-tenancy and security. Virtualization introduces additional overhead that often relates to memory management, interrupt handling and hypervisor mode switching. Among those, memory management and translation lookaside buffer (TLB) management have been shown to have a significant impact on the performance of systems. Two principal mechanisms for TLB management exist in today's systems, namely software and hardware managed TLBs. In this paper, we analyze and quantify the overhead of a pure software virtualization that is implemented over a software managed TLB. We then describe our design of hardware extensions to support virtualization in systems with software managed TLBs to remove the most dominant overheads. These extensions were implemented in the Power embedded A2 core, which is used in the PowerEN and in the Blue Gene/Q processors. They were used to implement a KVM port. We evaluate each of these hardware extensions to determine their overall contributions to performance and efficiency. Collectively these extensions demonstrate an average improvement of 232% over a pure software implementation.

TLB Improvements for Chip Multiprocessors

Translation Lookaside Buffers (TLBs) are critical to overall system performance. Much past research has addressed uniprocessor TLBs, lowering access times and miss rates. However, as Chip MultiProcessors (CMPs) become ubiquitous, TLB design and performance must be reevaluated. Our article begins by performing a thorough TLB performance evaluation of sequential and parallel benchmarks running on a real-world, modern CMP system using hardware performance counters. This analysis demonstrates the need for further improvement of TLB hit rates for both classes of application, and it also points out that the data TLB has a significantly higher miss rate than the instruction TLB in both cases. In response to the characterization data, we propose and evaluate both Inter-Core Cooperative (ICC) TLB prefetchers and Shared Last-Level (SLL) TLBs as alternatives to the commercial norm of private, per-core L2 TLBs. ICC prefetchers eliminate 19% to 90% of Data TLB (D-TLB) misses across parallel workloads while requiring only modest changes in hardware. SLL TLBs eliminate 7% to 79% of D-TLB misses for parallel workloads and 35% to 95% of D-TLB misses for multiprogrammed sequential workloads. This corresponds to 27% and 21% increases in hit rates as compared to private, per-core L2 TLBs, respectively, and is achieved this using even more modest hardware requirements. Because of their benefits for parallel applications, their applicability to sequential workloads, and their readily implementable hardware, SLL TLBs and ICC TLB prefetchers hold great promise for CMPs.

REDUCING MIGRATION-INDUCED MISSES IN AN OVER-SUBSCRIBED MULTIPROCESSOR SYSTEM

In a large multiprocessor server platform using multicore chips, the scheduler often migrates a thread or process, in order to achieve better load balancing or ensure fairness among competing scheduling entities. Each migration incurs a severe performance impact from the loss of cache and Translation Lookaside Buffer (TLB) footprints and subsequent higher cache misses and page walks. Such impact is likely to be more severe in virtualized environments, where high over-subscription of CPUs is very common for server consolidation workloads or virtual desktop infrastructure deployment, causing frequent migrations and context switches. We demonstrate the performance benefit of preserving a portion of L2 cache—in particular, MRU cache lines—and warming the destination L2 cache by prefetching those cache lines under different migration scenarios. We observed a 1.5-27% reduction in CPI (cycles per instruction) following a migration. We also study the effectiveness of preserving TLB entries over a context switch or migration.

PS-TLB

Traversing the page table during virtual to physical address translation causes pipeline stalls when misses occur in the translation-lookaside buffer (TLB). State-of-the-art translation proposals typically optimize a single aspect of translation performance (e.g., translation sharing, context switch performance, etc.) with potential trade-offs of additional hardware complexity, increased translation latency, or reduced scalability. In this article, we propose the partial sharing TLB (PS-TLB), a fast and scalable solution that reduces off-chip translation misses without sacrificing the timing-critical requirement of on-chip translation. We introduce the partial sharing buffer (PSB) which leverages application page sharing characteristics using minimal additional hardware resources. Compared to the leading TLB proposal that leverages sharing, PS-TLB provides a more than 45% improvement in translation latency with a 9% application speedup while using fewer storage resources. In addition, the page classification and PS-TLB architecture provide further optimizations including an over 30% reduction of interprocessor interrupts for coherence, and reduced context switch misses with fewer resources compared with existing methods.

Direct Compare of Information Coded With Error-Correcting Codes

There are situations in a computing system where incoming information needs to be compared with a piece of stored data to locate the matching entry, e.g., cache tag array lookup and translation look-aside buffer matching. If the stored data is protected with error-correcting codes (ECC) for reliability reason, the previous solution is to access the stored information, decode and correct if necessary before it is used to compare with the incoming data. The decoding and correcting step increases the total access time, which is often critical. In this paper, we propose a method to improve the compare latency for information encoded with ECC. We use the cache tag array look-up as an example, and results show that 30% gate count reduction and 12% latency reduction are achieved.

Leveraging Sharing in Second Level Translation-Lookaside Buffers for Chip Multiprocessors

Traversing page table during virtual to physical address translation causes significant pipeline stalls when misses occur in the translation-lookaside buffer (TLB). To mitigate this penalty, we propose a fast, scalable, multi-level TLB organization that leverages page sharing behaviors and performs efficient TLB entry placement. Our proposed partial sharing TLB (PSTLB) reduces TLB misses by around 60%. PSTLB also improves TLB performance by nearly 40% compared to traditional private TLBs and 17% over the state of the art scalable TLB proposal.