Set-associative Cache Research Articles

Effective padding of multidimensional arrays to avoid cache conflict misses

Caches are used to significantly improve performance. Even with high degrees of set associativity, the number of accessed data elements mapping to the same set in a cache can easily exceed the degree of associativity. This can cause conflict misses and lower performance, even if the working set is much smaller than cache capacity. Array padding (increasing the size of array dimensions) is a well-known optimization technique that can reduce conflict misses. In this paper, we develop the first algorithms for optimal padding of arrays aimed at a set-associative cache for arbitrary tile sizes. In addition, we develop the first solution to padding for nested tiles and multi-level caches. Experimental results with multiple benchmarks demonstrate a significant performance improvement from padding.

A Power Saving Set Associative Cache Model

Background/Objectives: Placing conflict cache lines in alternate set is proposed for set associative caches in literature. The conflicting lines are placed in one alternate set for each set. Methods/Statistical Analysis: This paper proposes architecture to place conflicting lines in set with maximum vacant ways in set associative caches. The model introduces one register per cache way with number of bits equal to number of cache sets. A sequential circuit using the register enables the cache ways of any set. The unoccupied ways are disabled. The proposed model is simulated with SPEC2K benchmarks. The power consumption is calculated using Quartus 2 tool and verilog code. Findings: An average improvement in power consumption of 68% for benchmarks with partial filled cache ways is observed with average memory access degradation of about 4.86%. A performance degradation in average memory access time of about 34% compared with direct mapped cache system and 3% compared with random placement algorithm is observed for the proposed system. The proposed model is for power consumption as is validated from the findings. The proposed model can be adapted for set associative caches to improve power consumption. Applications/Improvements: A model with optimal operational cache sets with methods to decrease power dissipation of sequential circuits, low power techniques for digital circuits can be used to improve the power consumption

Cache Controller for 4-way Set-Associative Cache Memory

This paper presents design of a cache controller for 4-way set associative cache memory and analyzing the performance in terms of cache hit verses miss rates. An FSM based cache controller has been designed for a 4-way set-associative cache memory of 1K byte with block size of 16 bytes. Main memory of 4K byte has been considered. The synthesis has been performed using Xilinx Synthesis Tool (XST) with Virtex-6 FPGA device XC6VLX240T. ISim simulator is used for functional verification of the designed code.The maximum output required time i.e. hold-time after clock is 0.777ns and minimum input time before clock arrival i.e. setup-time for designed module is 1.66ns. The maximum clock frequency is 257.202MHz.The design has also been synthesized in Cadence RTL Compiler tool. Finally, ASIC implementation of the designed cache controller has been done in Cadence Encounter Digital Implementation tool and the GDSII file has been generated.The designed cache controller consumes 5.53mW of total power.

V-SET CACHE: AN EFFICIENT ADAPTIVE SHARED CACHE FOR MULTI-CORE PROCESSORS

Development in VLSI design allows multi- to many-cores to be integrated on a single microprocessor chip. This increase in the core count per chip makes it more critical to design an efficient memory sub-system especially the shared last level cache (LLC). The efficient utilization of the LLC is a dominant factor to achieve the best microprocessor throughput. Conventional set-associative cache cannot cope with the new access pattern of the cache blocks in the multi-core processors. In this paper, the authors propose a new design for LLC in multi-core processor. The proposed v-set cache design allows an adaptive and dynamic utilization of the cache blocks. Unlike lately proposed design such as v-way caches, v-set cache design limits the serial access of cache blocks. In our paper, we thoroughly study the proposed design including area and power consumption as well as the performance and throughput. On eight-core microprocessor, the proposed v-set cache design can achieve a maximum speedup of 25% and 12% and an average speedup of 16% and 6% compared to conventional n-way and v-way cache designs, respectively. The area overhead of v-set does not exceed 7% compared to n-way cache.

Energy efficient data access techniques

Energy has become a first class design constraint for all types of processors. Data accesses contribute to processor energy usage and can account for up to 25% of the total energy used in embedded processors. Using a set-associative level-one data cache (L1 DC) organization is particularly energy inefficient as load operations access all L1 DC tag and data arrays in parallel to reduce access latency, but the data can reside in at most one way. Techniques that reduce L1 DC energy usage at the expense of degrading performance, such as filter caches, have not been adopted. In this presentation I will describe various techniques we have developed to reduce the energy usage for L1 DC accesses without adversely affecting performance. These techniques include avoiding unnecessary loads from L1 DC data arrays and a practical data filter cache design that not only significantly reduces data access energy usage, but also avoids the traditional execution time penalty associated with data filter caches.

IMPROVING peLIFO CACHE REPLACEMENT POLICY: HARDWARE REDUCTION AND THREAD-AWARE EXTENSION

Studying blocks behavior during their lifetime in cache can provide useful information to reduce the miss rate and therefore improve processor performance. According to this rationale, the peLIFO replacement algorithm [M. Chaudhuri, Proc. Micro'09, New York, 12–16 December, 2009, pp. 401–412], which learns dynamically the number of cache ways required to satisfy short-term reuses preserving the remaining ways for long-term reuses, has been recently proposed. In this paper, we propose several changes to the original peLIFO policy in order to reduce the implementation complexity involved, and we extend the algorithm to a shared-cache environment considering dynamic information about threads behavior to improve cache efficiency. Experimental results confirm that our simplification techniques reduce the required hardware with a negligible performance penalty, while the best of our thread-aware extension proposals reduces average CPI by 8.7% and 15.2% on average compared to the original peLIFO and LRU respectively for a set of 43 multi-programmed workloads on an 8 MB 16-way set associative shared L2 cache.

Hybrid Approach of Filtering Unnecessary Way Accesses for Set-Associative Caches

Power has been a big issue in processor design for several years.Conventional popular approaches for addressing this issue like DVFS(Dynamic Voltage Frequency Scaling) now hit the law of diminishing returns.As multi/many-core processors becoming the main stream processors,caches account for more and more CPU die area and power,this paper presents using filtering unnecessary way accesses to reduce dynamic power consumption of caches shared by instruction and data.The methods include using Invalid Filter,which could eliminate accesses to cache ways contained invalid blocks,and I/D Filter,which could eliminate accesses to cache ways contained instruction/data access type mismatch blocks,and Tag-2 Filter,which could eliminate accesses to cache ways contained tag lowest 2 bits mismatch blocks.Since the methods reducing the activities happened in cache architecture,dynamical CPU power could be significantly decreased.In the paper,we also propose combining the above methods together,which is called Invalid+I/D+Tag-2 Filter,in an attempt to achieve better power saving results.We have verified the effectiveness and complementariness of the three proposed methods through analysis and experiments.Also,our evaluations show that,we could obtain 19.6%~47.8%(which is on average 34.3%) improvement on a 64KB-4way set-associative cache and 19.6%~55.2%(which is on average 39.2%) improvement on a 128KB-8way set-associative cache comparing to Invalid+I/D Filter,and 16.1%~27.7%(which is on average 16.6%) improvement on a 64KB-4way set-associative cache and 6.9%~44.4%(which is on average 25.0%) improvement on a 128KB-8way set-associative cache comparing to Invalid+Tag-2 Filter,respectively.

Image-Optimized Rolling Cache: Reducing the Miss Penalty for Memory-Intensive Vision Algorithms

In real-time memory-intensive image processing and vision applications, increasing image resolution requires the use of external SDR/DDR memories. However, the arbitrary pixel access patterns used in most algorithms reduce their memory throughput as a result of increasing access latency. Efficient cache design is paramount in real-time memory-intensive applications. Its effectiveness depends on the spatial and temporal locality of data access. In image processing, the spatial locality denotes the neighboring pixels, located horizontally and vertically in 2-D. However, the conventional caches used in general processors cannot define the vertical locality. We propose a rolling cache optimized for image format and algorithms, a method to reduce the miss penalty by moving the cache horizontally and vertically, and a parallel processing architecture with interpolation, multilevel and multiple caches. To support our idea, we compare it with other types of caches and show that the average memory access time and the memory bandwidth are decreased by 28% and 74%, respectively, for a 2048 × 2048 image. Its performance is greater than that of the 16-way set associative cache, but the tag memory size is a bit larger than that of the direct-mapped cache. Using two different applications, we show that the proposed architecture is applicable to a number of algorithms if data access follows an arbitrary curve or block-wise pattern, which is the usual case with image processing and vision algorithms. If an application is based on local data access in resource-limited systems, it is possible to achieve high performance with lower operational frequency using the proposed architecture.

TurboLock: increasing associativity of lock table in transactional memory

Transactional memory (TM) is emerging as a promising paradigm to simplify parallel programming for chip multiprocessors. Most of software transactional memory (STM) systems exploit a lock table to synchronize transactional accesses to the shared memory locations. Memory addresses map to entries of the lock table through a hash function to detect conflicts in the event of simultaneous accesses to the shared memory locations. In the current implementation of the lock table, if two distinct addresses map to the same entry of the table, they are treated as a conflict even though there is no true conflict between the two addresses. This is called false conflict. In the event of a false conflict, transactions are aborted conservatively which reduces concurrency level in programs. In this paper, we study false conflicts in STMs and propose TurboLock to reduce frequency of false conflicts. Inspired by set associative caches, TurboLock increases associativity of the lock table to reduce likelihood of aliasing-induced conflicts. While TurboLock is effective in reducing the false conflicts, it may degrade performance due to overhead of the associative lock table in software. To improve performance of TurboLock, we propose HW-TurboLock which exploits hardware to reduce overhead of the associative lock table. We have evaluated our optimization techniques using Gem5 full-system simulator and compared the performance of the new implementation with the baseline scheme. Our simulation results reveal that HW-TurboLock is highly effective and significantly improves performance of TM systems.

Virtually split cache

First-level caches are usually split for both instructions and data instead of unifying them in a single cache. Although that approach eases the pipeline design and provides a simple way to independently treat data and instructions, its global hit rate is usually smaller than that of a unified cache. Furthermore, unified lower-level caches usually behave and process memory requests disregarding whether they are data or instruction requests. In this article, we propose a new technique aimed to balance the amount of space devoted to instructions and data for optimizing set-associative caches: the Virtually Split Cache or VSC. Our technique combines the sharing of resources from unified approaches with the bandwidth and parallelism that split configurations provide, thus reducing power consumption while not degrading performance. Our design dynamically adjusts cache resources devoted to instructions and data depending on their particular demand. Two VSC designs are proposed in order to track the instructions and data requirements. The Shadow Tag VSC (ST-VSC) is based on shadow tags that store the last evicted line related to data and instructions in order to determine how well the cache would work with one more way per set devoted to each kind. The Global Selector VSC (GS-VSC) uses a saturation counter that is updated every time a cache miss occurs either under an instruction or data request applying a duel-like mechanism. Experiments with a variable and a fixed latency VSC show that ST-VSC and GS-VSC reduce on average the cache hierarchy power consumption by 29% and 24%, respectively, with respect to a standard baseline. As for performance, while the fixed latency designs virtually match the split baseline in a single-core system, a variable latency ST-VSC and GS-VSC increase the average IPC by 2.5% and 2%, respectively. In multicore systems, even the slower fixed latency ST-VSC and GS-VSC designs improve the baseline IPC by 3.1% and 2.5%, respectively, in a four-core system thanks to the reduction in the bandwidth demanded from the lower cache levels. This is in contrast with many techniques that trade performance degradation for power consumption reduction. VSC particularly benefits embedded processors with a single level of cache, where up to an average 9.2% IPC improvement is achieved. Interestingly, we also find that partitioning the LLC for instructions and data can improve performance around 2%.

Different access mechanisms for set-associative cache architecture for reduced power consumption

In recent years, power consumption has become one of the most critical design concern. Due to lower miss rate, set-associative caches are usually employed to improve embedded system's performance. However, a conventional set-associative cache implementation is not power efficient. Thus, in order to achieve low-power consumption, novel set-associative cache architectures have been proposed in which efficient accessing of tag bits and the data bits takes place in two phases unlike the conventional set-associative cache architectures. The behavioural modelling of the cache architectures was done using Verilog Hardware Description Language (HDL). The synthesis results illustrate that the proposed parallel and sequential set-associative cache architectures showed significant power reduction of 34.92% and 35.36% on an average as compared to conventional parallel and conventional sequential set-associative cache architectures, respectively.

An Energy Efficient 32-nm 20-MB Shared On-Die L3 Cache for Intel® Xeon® Processor E5 Family

An energy efficient on-die 20-way set associative L3 cache of size 20 MB for the Intel® Xeon® processor E5 family is presented. It is manufactured in the Intel's 32-nm second generation of high-K dielectric metal gate process with 9-copper metal layers. The L3 cache design uses 0.2119 um 2 cell for the high density big array and 0.2725 um 2cell for the high performance smaller arrays. The power efficiency was achieved by employing advanced power saving schemes and effective Vccmin design techniques. The proposed L3 cache topology seamlessly supports a high density modular and energy efficient designs. The effective and rich redundancy design improves both yield and low voltage operations. The L3 cache achieves more than 20-40% energy efficiency when compared to previous generations and demonstrates wide operating ranges from 1.2 GHz at below 0.7 V to greater than 4.0 GHz at above 1.0 V.

An 86 mW 98GOPS ANN-Searching Processor for Full-HD 30 fps Video Object Recognition With Zeroless Locality-Sensitive Hashing

Approximate nearest neighbor (ANN) searching is an essential task in object recognition. The ANN-searching stage, however, is the main bottleneck in the object recognition process due to increasing database size and massive dimensions of keypoint descriptors. In this paper, a high throughput ANN-searching processor is proposed for high-resolution (full-HD) and real-time (30 fps) video object recognition. The proposed ANN-searching processor adopts an interframe cache architecture as a hardware-oriented approach and a zeroless locality-sensitive-hashing (zeroless-LSH) algorithm as a software-oriented approach to reduce the external memory bandwidth required in nearest neighbor searching. A four-way set associative on-chip cache has a dedicated architecture to exploit data correlation at the frame-level. Zeroless-LSH minimizes data transactions from external memory at the vector-level. The proposed ANN-searching processor is fabricated as part of an object recognition SoC using a 0.13 μm 6 metal CMOS technology. It achieves 62 720 vectors/s throughput and 1140 GOPS/W power efficiency, which are 1.45 and 1.37 times higher than the state-of-the-art, respectively, enabling real-time object recognition for full-HD 30 fps video streams.

Reducing cache and TLB power by exploiting memory region and privilege level semantics

The L1 cache in today’s high-performance processors accesses all ways of a selected set in parallel. This constitutes a major source of energy inefficiency: at most one of the N fetched blocks can be useful in an N-way set-associative cache. The other N-1 cachelines will all be tag mismatches and subsequently discarded. We propose to eliminate unnecessary associative fetches by exploiting certain software semantics in cache design, thus reducing dynamic power consumption. Specifically, we use memory region information to eliminate unnecessary fetches in the data cache, and ring level information to optimize fetches in the instruction cache. We present a design that is performance-neutral, transparent to applications, and incurs a space overhead of mere 0.41% of the L1 cache. We show significantly reduced cache lookups with benchmarks including SPEC CPU, SPECjbb, SPECjAppServer, PARSEC, and Apache. For example, for SPEC CPU 2006, the proposed mechanism helps to reduce cache block fetches from the data and instruction caches by an average of 29% and 53% respectively, resulting in power savings of 17% and 35% in the caches, compared to the aggressively clock-gated baselines.

An Energy Efficient Set Associative Cache Algorithm

associative caches have fixed ways. Entire cache is enabled during cache operation. This paper proposes cache architecture mapping cache line to fixed cache way of mapped set. The address is mapped to set as in conventional set associative cache. The tag value of the mapped line is divided into blocks of size of number of cache ways. The average of maximum and minimum frequency of this division is the mapped way. The proposed model is simulated with SPEC2K benchmarks. The average memory access time degradation of 3.8% is seen over traditional set associative cache. The energy saving of 49% is observed in proposed model.

4Ghz 고성능 CPU 위한 캐시 메모리 시스템

본 논문에서는 4Ghz의 빠른 클럭 속도의 CPU에 적합한 고성능 L1 캐시 메모리 구조를 제안한다. 제안된 캐시 메모리는 빠른 접근 시간을 위한 직접사상 캐시와 시간적 지역성을 고려한 2-way 연관사상 버퍼 그리고 버퍼 선택 테이블로 구성된다. 빠른 접근 시간을 보장하는 직접사상 캐시는 가장 최근 접근한 데이터를 저장하게 된다. 만약에 직접사상 캐쉬로부터 추출되는 데이터가 다시 참조되어질 높은 확률을 가지는 데이터이면 그 데이터들은 2-웨이 연관사상 버퍼로 선택적으로 저장되어 진다. 그리고 고성능과 저전력의 효과를 높이기 위하여 2-웨이 연관사상 버퍼중 하나의 웨이만 선택적으로 먼저 접근되어지며, 이러한 동작은 버퍼 선택 테이블에 의해 선택된다. 시뮬레이션 결과에 따르면, 에너지 소비와 평균 메모리 접근 시간을 고려한 에너지<TEX>$^*$</TEX>지연시간에서 두배 이상의 크기를 가지는 직접사상 캐시, 4-웨이 연관사상 캐시 그리고 희생 캐시에 비해 각각 45%, 70% 그리고 75%의 성능향상을 이루었다. TIn this paper, we propose a high performance L1 cache structure on the high clock CPU of 4GHz. The proposed cache memory consists of three parts, i.e., a direct-mapped cache to support fast access time, a two-way set associative buffer to exploit temporal locality, and a buffer-select table. The most recently accessed data is stored in the direct-mapped cache. If a data has a high probability of a repeated reference, when the data is replaced from the direct-mapped cache, the data is selectively stored into the two-way set associative buffer. For the high performance and low power consumption, we propose an one way among two ways set associative buffer is selectively accessed based on the buffer-select table(BST). According to simulation results, Energy <TEX>$^*$</TEX> Delay product can improve about 45%, 70% and 75% compared with a direct mapped cache, a four-way set associative cache, and a victim cache with two times more space respectively.

An Algorithm for Variable Cache Ways

This paper proposed an algorithm for varying cache ways in set associative cache. The ways of the mapped cache set are extended to half of the ways of adjacent sets in this model. A set is thus viewed as growing in both ways. If a way is free in this extended model, the line is placed in the way. An owner vector indicates the set owner ship of the way. If all the ways are full, the least recently used way of the mapped set is replaced. The proposed model adopts tag cache model proposed in literature. The proposed model is simulated with SPEC2000 benchmarks. An improvement of 5% in average memory access time over traditional w-way set associative model and 3% over 2w-way set associative cache model for chosen parameters is observed. The energy consumed in the proposed model is comparable to the traditional set associative cache of same size.

Run-Time Reconfiguration of Expandable Cache for Embedded Systems

Expandable cache proposed by Bournoutian and Orailoglu is very efficient in reducing miss rate and energy consumption with small area overhead. However, the original expandable cache with only one expansion scheme may lead to thrashing problems. In this work, based on the structure of expandable cache, we will introduce a new cache design which has many expansion schemes to fit different run-time program behaviors. The expansion scheme of our proposed cache is dynamically changed by executing configuration instructions which are inserted at compile time. The experimental results of SPEC CPU2000 have shown that our proposed cache design effectively improves the miss rate by 14.74% as compared with the original expandable cache. In terms of energy improvement ratio, our method is 5.62% higher than that of expandable cache when the baseline is set as the energy consumption of 2-way set-associative cache.

Design, Performance, and Energy Consumption of eDRAM/SRAM Macrocells for L1 Data Caches

SRAM and DRAM have been the predominant technologies used to implement memory cells in computer systems, each one having its advantages and shortcomings. SRAM cells are faster and require no refresh since reads are not destructive. In contrast, DRAM cells provide higher density and minimal leakage energy since there are no paths within the cell from Vdd to ground. Recently, DRAM cells have been embedded in logic-based technology (eDRAM), thus overcoming the speed limit of typical DRAM cells. In this paper, we propose a hybrid n-bit macrocell that implements one SRAM cell and n-1 eDRAM cells. This cell is aimed at being used in an n-way set-associative first-level data cache. Architectural mechanisms (e.g., special writeback policies) have been devised to completely avoid refresh logic. Performance, energy, and area have been analyzed in detail. Experimental results show that using typical eDRAM capacitors, and compared to a conventional cache, a 4-way set-associative hybrid cache reduces both energy consumption and area up to 54 and 29 percent, respectively, while having negligible impact on performance (less than 2 percent).

Performance Gains and Drawbacks using Set Associative Cache

Abstract This paper explains why there are performance drawbacks for dense matrix multiplication algorithm using multilevel associative cache memory. We give an overview of cache memory organization and theoretical analysis why performance drawback appears in dense matrix multiplication algorithm. We also provide a method to avoid situations where this performance drawback is significant and how to achieve maximum performance. The analyzed problem appears in storage of matrix columns and inefficient usage of cache where the matrix will always map onto a small group of same cache sets and initiate a significant number of cache misses. In this case it looks like the processor is using only a small group of cache sets instead of complete number of sets in associative memory where maximum performance can be achieved.