Translation Lookaside Buffer Research Articles

Reducing Instruction Translation Look-Aside Buffer Energy Through Compiler-Directed Resizing

Reducing Instruction Translation Look-Aside Buffer Energy Through Compiler-Directed Resizing

ReStore: Symptom-Based Soft Error Detection in Microprocessors

Device scaling and large-scale integration have led to growing concerns about soft errors in microprocessors. To date, in all but the most demanding applications, implementing parity and ECC for caches and other large, regular SRAM structures have been sufficient to stem the growing soft error tide. This will not be the case for long and questions remain as to the best way to detect and recover from soft errors in the remainder of the processor - in particular, the less structured execution core. In this work, we propose the ReStore architecture, which leverages existing performance enhancing checkpointing hardware to recover from soft error events in a low cost fashion. Error detection in the ReStore architecture is novel: symptoms that hint at the presence of soft errors trigger restoration of a previous checkpoint. Example symptoms include exceptions, control flow misspeculations, and cache or translation look-aside buffer misses. Compared to conventional soft error detection via full replication, the ReStore framework incurs little overhead, but sacrifices some amount of error coverage. These attributes make it an ideal means to provide very cost effective error coverage for processor applications that can tolerate a nonzero, but small, soft error failure rate. Our evaluation of an example ReStore implementation exhibits a 2times increase in MTBF (mean time between failures) over a standard pipeline with minimal hardware and performance overheads. The MTBF increases by 20times if ReStore is coupled with protection for certain particularly vulnerable pipeline structures

In-line interrupt handling and lock-up free translation lookaside buffers (TLBs)

The effects of the general-purpose precise interrupt mechanisms in use for the past few decades have received very little attention. When modern out-of-order processors handle interrupts precisely, they typically begin by flushing the pipeline to make the CPU available to execute handler instructions. In doing so, the CPU ends up flushing many instructions that have been brought in to the reorder buffer. In particular, these instructions may have reached a very deep stage in the pipeline - representing significant work that is wasted. In addition, an overhead of several cycles and wastage of energy (per exception detected) can be expected in refetching and reexecuting the instructions flushed. This paper concentrates on improving the performance of precisely handling software managed translation look-aside buffer (TLB) interrupts, one of the most frequently occurring interrupts. The paper presents a novel method of in-lining the interrupt handler within the reorder buffer. Since the first level interrupt-handlers of TLBs are usually small, they could potentially fit in the reorder buffer along with the user-level code already there. In doing so, the instructions that would otherwise be flushed from the pipe need not be refetched and reexecuted. Additionally, it allows for instructions independent of the exceptional instruction to continue to execute in parallel with the handler code. By in-lining the TLB interrupt handler, this provides lock-up free TLBs. This paper proposes the prepend and append schemes of in-lining the interrupt handler into the available reorder buffer space. The two schemes are implemented on a performance model of the Alpha 21264 processor built by Alpha designers at the Palo Alto Design Center (PADC), California. We compare the overhead and performance impact of handling TLB interrupts by the traditional scheme, the append in-lined scheme, and the prepend in-lined scheme. For small, medium, and large memory footprints, the overhead is quantified by comparing the number and pipeline state of instructions flushed, the energy savings, and the performance improvements. We find that lock-up free TLBs reduce the overhead of refetching and reexecuting the instructions flushed by 30-95 percent, reduce the execution time by 5-25 percent, and also reduce the energy wasted by 30-90 percent.

RESIZABLE TRANSLATION STORAGE BUFFERS

A translation lookaside buffer (TLB) is a high-speed associative cache of recently used virtual-to-physical address translations. The operating system can deal with a TLB miss through software trap handling. A possible technique for such software trap handling uses translation storage buffers (TSBs). A TSB is an operating system data structure that caches the most recent address translations. On a TLB miss, the TLB trap handler searches the TSB for the missing address translation. If the search generates a hit, the address mapping is added to the TLB. In current implementations, TSBs are organized in a variety of ways: a global TSB for all CPUs within the system, a per-processor TSB, or a per-process TSB. All of the aforementioned techniques have various limitations that will be addressed in this paper. In this work, we propose a new framework for TSBs generation and allocation. In the proposed approach, a policy of resizing and dynamically allocating TSBs for the different processes is used. This dynamic policy allows the system to adopt to different workloads while achieving a low TSB context invalidation overhead. In addition, with the ability to assign a separate TSB to each process, thrashing is practically eliminated. Implementation and experimental results of the proposed scheme are reported. Comparisons against existing implementations confirmed the expected performance enhancement.

A Memory System for Education

The memory system is one of the core topics in computer architecture and organization. An important problem in teaching this topic is how to help students connect their theoretical knowledge of memory system concepts with the practical problems facing the designer of various parts of a memory system. A common approach to tackling this problem is to organize practical exercises in the laboratory using a memory system simulator. The existing simulators mainly focus on subtle memory system issues, such as cache performance, latency, coherence and consistency models in multiprocessor systems and do not admit a view of design details and internal activities within various parts of a memory system. This paper presents an originally developed memory system for education and its web-based simulator. The memory system includes the virtual memory and translation look-aside buffer, the cache memory and the interleaved main memory. The simulator facilitates a web-based clock-by-clock interactive simulation of the memory system, its visual presentation at the register-transfer level and navigation through parts of the system. They can be used for exercises in the laboratory and self-learning from home.

Moving address translation closer to memory in distributed shared-memory multiprocessors

To support a global virtual memory space, an architecture must translate virtual addresses dynamically. In current processors, the translation is done in a TLB (translation lookaside buffer), before or in parallel with the first-level cache access. As processor technology improves at a rapid pace and the working sets of new applications grow insatiably, the latency and bandwidth demands on the TLB are difficult to meet, especially in multiprocessor systems, which run larger applications and are plagued by the TLB consistency problem. We describe and compare five options for virtual address translation in the context of distributed shared memory (DSM) multiprocessors, including CC-NUMAs (cache-coherent non-uniform memory access architectures) and COMAs (cache only memory access architectures). In CC-NUMAs, moving the TLB to shared memory is a bad idea because page placement, migration, and replication are all constrained by the virtual page address, which greatly affects processor node access locality. In the context of COMAs, the allocation of pages to processor nodes is not as critical because memory blocks can dynamically migrate and replicate freely among nodes. As the address translation is done deeper in the memory hierarchy, the frequency of translations drops because of the filtering effect. We also observe that the TLB is very effective when it is merged with the shared-memory, because of the sharing and prefetching effects and because there is no need to maintain TLB consistency. Even if the effectiveness of the TLB merged with the shared memory is very high, we also show that the TLB can be removed in a system with address translation done in memory because the frequency of translations is very low.

Operating system exploitation of the POWER5 system

The POWER5™ system incorporates several features designed to improve performance by eliminating bottlenecks and accelerating common functions used in operating systems. This paper discusses how two of the supported operating systems for POWER5—AIX® and Linux™—make use of these features to deliver improved system scalability and performance. In particular, the overheads for synchronizing translation-lookaside buffer (TLB) invalidations between processors, and for ensuring that the instruction cache is kept coherent by software, have been removed. The POWER5 simultaneous multithreading (SMT) implementation has features which allow operating systems to optimize the system for the kinds of applications being executed. We discuss how the operating systems approach the problems of scheduling tasks across the system, of determining when to switch processors between single-threaded (ST) and SMT mode, and of accounting accurately for CPU usage when in the SMT mode.

Automatic measurement of memory hierarchy parameters

The running time of many applications is dominated by the cost of memory operations. To optimize such applications for a given platform, it is necessary to have a detailed knowledge of the memory hierarchy parameters of that platform. In practice, this information is poorly documented if at all. Moreover, there is growing interest in self-tuning, autonomic software systems that can optimize themselves for different platforms; these systems must determine memory hierarchy parameters automatically without human intervention.One solution is to use micro-benchmarks to determine the parameters of the memory hierarchy. In this paper, we argue that existing micro-benchmarks are inadequate, and present novel micro-benchmarks for determining parameters of all levels of the memory hierarchy, including registers, all data caches and the translation look-aside buffer. We have implemented these micro-benchmarks in a tool called X-Ray that can be ported easily to new platforms. We present experimental results that show that X-Ray successfully determines memory hierarchy parameters on current platforms, and compare its accuracy with that of existing tools.

A low-power 2.5-GHz 90-nm level 1 cache and memory management unit

The design of a 90-nm virtually addressed cache subsystem with separate 32-kB instruction and data caches is described. The circuits and microarchitecture are illustrated, including architecture level trace data validating low-power features and provisions to support snooping while maintaining the latency and power of virtual addressing. Low-power memory management unit design including a translation lookaside buffer with process identifier mapping is also described. Level 1 caches with support for high bandwidth, single cycle 256 bit fill and evict, as well as features for low power are also described. The design approaches are validated through both simulation and experimental results.

Optimizing instruction TLB energy using software and hardware techniques

Power consumption and power density for the Translation Look-aside Buffer (TLB) are important considerations not only in its design, but can have a consequence on cache design as well. After pointing out the importance of instruction TLB (iTLB) power optimization, this article embarks on a new philosophy for reducing the number of accesses to this structure. The overall idea is to keep a translation currently being used in a register and avoid going to the iTLB as far as possible---until there is a page change. We propose four different approaches for achieving this, and experimentally demonstrate that one of these schemes that uses a combination of compiler and hardware enhancements can reduce iTLB dynamic power by over 85% in most cases.The proposed approaches can work with different instruction-cache (iL1) lookup mechanisms and achieve significant iTLB power savings without compromising on performance. Their importance grows with higher iL1 miss rates and larger page sizes. They can work very well with large iTLB structures that can possibly consume more power and take longer to lookup, without the iTLB getting into the common case. Further, we also experimentally demonstrate that they can provide performance savings for virtually indexed, virtually tagged iL1 caches, and can even make physically indexed, physically tagged iL1 caches a possible choice for implementation.

An address translation simulator

Virtual memory is a major topic in undergraduate operating systems courses. One aspect of virtual memory, address translation, is often covered in an abstract way. When examples are given, only a piece of the translation is done, using a small translation lookaside buffer or a small single-level page table. Since most students learn best by doing rather than watching, the topic is best understood by having students do realistic address translations. This is problematic since it involves lookup from several large tables of data which are difficult to fit on a piece of paper. The address translation simulator described here solves this problem by presenting the student with complete page tables in a way that allows simple navigation of these tables. The simulator can be used for both teaching and student evaluation.

Selective block buffering TLB system for embedded processors

The authors present a translation lookaside buffer (TLB) system with low power consumption for embedded processors. The proposed TLB is constructed as multiple banks, each with an associated block buffer and a corresponding comparator. Either the block buffer or the main bank is selectively accessed on the basis of two bits in the tag buffer. Dynamic power savings are achieved by reducing the number of entries accessed in parallel, as a result of using the tag buffer as a filtering mechanism. The performance overhead of the proposed TLB is negligible compared with other hierarchical TLB structures. For example, the two-cycle overhead of the proposed TLB is only ∼1%, as compared with 5% overhead for a filter (micro)-TLB and 14% overhead for a banked-TLB with block buffering. The authors show that the average hit ratios of the block buffers and the main banks of the proposed TLB are 94% and 6%, respectively. Dynamic power is reduced by ∼93% with respect to a fully associative TLB, 87% with respect to a filter-TLB and 60% relative to a banked-TLB with block buffering. Therefore, significant power savings are achieved with only a small performance degradation.

Concurrent support of multiple page sizes on a skewed associative TLB

Some architecture definitions (e.g., Alpha) allow the use of multiple virtual page sizes even for a single process. Unfortunately, on current set-associative TLBs (translation lookaside buffers), pages with different sizes cannot coexist together. Thus, processors supporting multiple page sizes implement fully associative TLBs. In this research note, we show how the skewed-associative TLB can accommodate the concurrent use of multiple page sizes within a single process. This allows us to envision either medium size L1 TLBs or very large L2 TLBs supporting multiple page sizes.

The IBM eServer z990 microprocessor

The IBM eServer™ z990 microprocessor implements many features designed to give excellent performance on both newer and traditional mainframe applications. These features include a new superscalar instruction execution pipeline, high-bandwidth caches, a huge secondary translation-lookaside buffer (TLB), and an onboard cryptographic coprocessor. The microprocessor maintains zSeries® leadership in RAS (reliability, availability, serviceability) capabilities that include state-of-the-art error detection and recovery.

Dynamic and selective low power data TLB system

We present a selective filter-bank translation lookaside buffer (TLB) system with low power consumption for embedded processors. The proposed TLB is constructed as multiple banks, with a small two-bank buffer, called as a filter-bank buffer, located above its associated bank. Either a filter-bank buffer or a main bank TLB can be selectively accessed based on two bits in the filter-bank buffer. Power savings are achieved by reducing the number of entries accessed at a time, by using filtering and bank mechanism. The overhead of the proposed TLB turns out to be negligible compared with other hierarchical structures. That is, the two-cycle overhead of the proposed TLB is only about 3%, as compared with the 6% overhead of filter (micro)-TLB and the 9% overhead of banked-filter TLB. We show that the average hit ratios of the filter-bank buffers and the main banks of the proposed TLB are 88 and 12%, respectively. Therefore, significant power savings can be achieved with just a small performance degradation. The Power×Delay product can be reduced by about 88% compared with a fully associative TLB, 67% with respect to filter-TLB, and 37% relative to banked-filter TLB.

A 2x load/store pipe for a low-power 1-GHz embedded processor

The load/store pipe for a low-power 1-GHz embedded processor is described. For area savings and logic complexity reduction, the load/store pipe is clocked at twice the frequency of the processor core. It can sustain two load or store operations per core clock cycle with zero load to use issue latency. The address generation unit for one of the two load/store pipes takes advantage of the common addressing mode in MIPS 64 ISA to generate the address within a core clock phase. Phase borrowing is employed in the translation lookaside buffer (TLB) design to enable a lookup process within a core clock phase. The data cache design enables the activation of a minimum number of data bank arrays for power savings. Small-swing differential buses are used for multiple address and data buses for improved signal transmission latency. The quadrature clocks used to derive the 2/spl times/ clock are generated with a novel 4-to-1 divider and distributed with matched paths, all to reduce the duty cycle variation of the 2/spl times/ clock phase. The design has been implemented in a 0.13-/spl mu/m CMOS process.

A dynamically tunable memory hierarchy

The widespread use of repeaters in long wires creates the possibility of dynamically sizing regular on-chip structures. We present a tunable cache and translation lookaside buffer (TLB) hierarchy that leverages repeater insertion to dynamically trade off size for speed and power consumption on a per-application phase basis using a novel configuration management algorithm. In comparison to a conventional design that is fixed at a single design point targeted to the average application, the dynamically tunable cache and TLB hierarchy can be tailored to the needs of each application phase. The configuration algorithm dynamically detects phase changes and selects a configuration based on the application's ability to tolerate different hit and miss latencies in order to improve the memory energy-delay product. We evaluate the performance and energy consumption of our approach and project the effects of technology scaling trends on our design.

Tiling, block data layout, and memory hierarchy performance

Recently, several experimental studies have been conducted on block data layout in conjunction with tiling as a data transformation technique to improve cache performance. In this paper, we analyze cache and translation look-aside buffer (TLB) performance of such alternate layouts (including block data layout and Morton layout) when used in conjunction with tiling. We derive a tight lower bound on TLB performance for standard matrix access patterns, and show that block data layout and Morton layout achieve this bound. To improve cache performance, block data layout is used in concert with tiling. Based on the cache and TLB performance analysis, we propose a data block size selection algorithm that finds a tight range for optimal block size. To validate our analysis, we conducted simulations and experiments using tiled matrix multiplication, LU decomposition, and Cholesky factorization. For matrix multiplication, simulation results using UltraSparc II parameters show that tiling and block data layout with a block size given by our block size selection algorithm, reduces up to 93 percent of TLB misses compared with other techniques. The total miss cost is reduced considerably. Experiments on several platforms show that tiling with block data layout achieves up to 50 percent performance improvement over other techniques that use conventional layouts. Morton layout is also analyzed and compared with block data layout. Experimental results show that matrix multiplication using block data layout is up to 15 percent faster than that using Morton data layout.

A computer architecture with access control and cache option tags on individual instruction operands

The access control on data is usually on per memory page basis and is implemented in the Translation Lookaside Buffer (TLB) via page tables managed by memory management of operating systems. In addition, to optimize the memory reference performance, it is desirable to specify if a page should be encached or not, so that unnecessary and undesirable data encaching will be avoided. It is also desirable to be able to specify if maintaining data coherency in a multiprocessing system is required. Such maintenance demands a lot of checking, which creates performance bottlenecks and system complexity, but only a small percentage of data requires absolute data coherency. In all these cases, pages are not logical entities for such attributes or characteristic assignments. Better choices are the operands, which can map directly to the variables in a program.In this paper, we are proposing an architecture to support the access control, the maintenance requirement of data coherency and optional encaching on data as system attributes on individual operands.

Practical, transparent operating system support for superpages

Most general-purpose processors provide support for memory pages of large sizes, called superpages. Superpages enable each entry in the translation lookaside buffer (TLB) to map a large physical memory region into a virtual address space. This dramatically increases TLB coverage, reduces TLB misses, and promises performance improvements for many applications. However, supporting superpages poses several challenges to the operating system, in terms of superpage allocation and promotion tradeoffs, fragmentation control, etc. We analyze these issues, and propose the design of an effective superpage management system. We implement it in FreeBSD on the Alpha CPU, and evaluate it on real workloads and benchmarks. We obtain substantial performance benefits, often exceeding 30%; these benefits are sustained even under stressful workload scenarios.