Set-associative Cache Research Articles

Design and Optimization of 4-way set Associative Mapped Cache Controller

Abstract: In the realm of modern computer systems, the 4-way set associative mapped cache controller emerges as a cornerstone, revolutionizing memory access efficiency. This exploration delves into its core principles, revealing its pivotal role in synchronizing rapid CPUs with slower main memory. By orchestrating seamless data exchange and employing intelligent replacement policies, this controller optimizes performance. Embarking on practical realization, a non-pipelined processor materializes using Xilinx Vivado and Verilog HDL, propelling frequent memory read/write requests for the 4-way set associative mapped cache. The quest for efficiency fuels refinements, culminating in an optimized cache controller design. Rigorously validated within the Xilinx Vivado environment, the architecture demonstrates tangible success with quantified outcomes. The design framework encompasses a 4K byte primary memory, complemented by a 1K byte 4-way set associative cache. This setting scrutinizes the optimized cache controller's efficacy. The dedicated test module, housing a suite of instructions, underscores its performance. Remarkably, the evaluation showcases 19 cache hits and 6 cache misses, revealing the potency of the optimized design in minimizing cache misses, particularly in call and jump instructions, an essential stride towards enhanced memory efficiency.

An FPGA-Based Performance Analysis of Hardware Caching Techniques for Blockchain Key-Value Database

The speedy advancement in wireless communication technologies provides considerable development to enable smart cities with applications such as Intelligent Transport Systems (ITS) and the Internet of Medical Things (IoMT). Blockchain is an emerging technology that provides a secure and distributed data storage mechanism useful for smart city applications. The full nodes in the Blockchain contain a record of all the transactions and data blocks of the Blockchain users. As the number of full nodes is less and the number of Blockchain users is high, there is a huge load on the full nodes for accessing and verifying the data by the Blockchain users. Efficient hardware caching techniques are needed to decrease the data access delay. In this paper, we implement different caching techniques on the Field-Programmable Gate Array (FPGA) Network Interface Card (NIC) and analyze their performance for the key-value store caching in the Blockchain. We design the 2-way and 4-way caching techniques on Block Random-Access Memory (BRAM) and compare them with the conventional direct-mapped caching technique in terms of cache hits and cache misses. The improvements in the hit ratio of the 2-way set-associative cache technique with respect to the direct-mapped cache technique for 10 K, 25 K, and 50 K addresses are 0.8%, 0.77%, and 1.67%, respectively. On the other hand, for the same sets of addresses, the hit rate improvement of the 4-way set-associative cache technique with respect to the direct-mapped cache technique is 0.92%, 2.01%, and 2.4%, respectively. The improvements in hit rate for large data sets show that 2-way and 4-way set-associative cache techniques perform better than the direct-mapped cache technique for caching systems.

ASA: A ccelerating S parse A ccumulation in Column-wise SpGEMM

Sparse linear algebra is an important kernel in many different applications. Among various sparse general matrix-matrix multiplication (SpGEMM) algorithms, Gustavson’s column-wise SpGEMM has good locality when reading input matrix and can be easily parallelized by distributing the computation of different columns of an output matrix to different processors. However, the sparse accumulation (SPA) step in column-wise SpGEMM, which merges partial sums from each of the multiplications by the row indices, is still a performance bottleneck. The state-of-the-art software implementation uses a hash table for partial sum search in the SPA, which makes SPA the largest contributor to the execution time of SpGEMM. There are three reasons that cause the SPA to become the bottleneck: (1) hash probing requires data-dependent branches that are difficult for a branch predictor to predict correctly; (2) the accumulation of partial sum is dependent on the results of the hash probing, which makes it difficult to hide the hash probing latency; and (3) hash collision requires time-consuming linear search and optimizations to reduce these collisions require an accurate estimation of the number of non-zeros in each column of the output matrix. This work proposes ASA architecture to accelerate the SPA. ASA overcomes the challenges of SPA by (1) executing the partial sum search and accumulate with a single instruction through ISA extension to eliminate data-dependent branches in hash probing, (2) using a dedicated on-chip cache to perform the search and accumulation in a pipelined fashion, (3) relying on the parallel search capability of a set-associative cache to reduce search latency, and (4) delaying the merging of overflowed entries. As a result, ASA achieves an average of 2.25× and 5.05× speedup as compared to the state-of-the-art software implementation of a Markov clustering application and its SpGEMM kernel, respectively. As compared to a state-of-the-art hashing accelerator design, ASA achieves an average of 1.95× speedup in the SpGEMM kernel.

Kangaroo: Theory and Practice of Caching Billions of Tiny Objects on Flash

Many social-media and IoT services have very large working sets consisting of billions of tiny (≈100 B) objects. Large, flash-based caches are important to serving these working sets at acceptable monetary cost. However, caching tiny objects on flash is challenging for two reasons: (i) SSDs can read/write data only in multi-KB “pages” that are much larger than a single object, stressing the limited number of times flash can be written; and (ii) very few bits per cached object can be kept in DRAM without losing flash’s cost advantage. Unfortunately, existing flash-cache designs fall short of addressing these challenges: write-optimized designs require too much DRAM, and DRAM-optimized designs require too many flash writes. We present Kangaroo , a new flash-cache design that optimizes both DRAM usage and flash writes to maximize cache performance while minimizing cost. Kangaroo combines a large, set-associative cache with a small, log-structured cache. The set-associative cache requires minimal DRAM, while the log-structured cache minimizes Kangaroo’s flash writes. Experiments using traces from Meta and Twitter show that Kangaroo achieves DRAM usage close to the best prior DRAM-optimized design, flash writes close to the best prior write-optimized design, and miss ratios better than both. Kangaroo’s design is Pareto-optimal across a range of allowed write rates, DRAM sizes, and flash sizes, reducing misses by 29% over the state of the art. These results are corroborated by analytical models presented herein and with a test deployment of Kangaroo in a production flash cache at Meta.

Enabling In-Memory Computations in Non-Volatile SRAM Designs

The rapid growth in development of neural networks has necessitated the requirement of large capacity on-chip SRAM’s for Machine Learning accelerators. This has resulted in SRAM’s occupying significant portion of the die area. Furthermore, due to increased short channel effects in advanced CMOS technology nodes, the Vt of the transistors are increased to reduce the leakage power effectively. Vt increase results in direct increase in V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MIN</sub> (minimum operating voltage) of the device. The conventional 6T SRAM with the use of RRAM(R) to store the bitcell storage node values and PTM(S) as a selector device (6T-2R-2S) can help in decoupling V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MIN</sub> and Vt requirement with minimum area overhead. Functionalities of 6T-2R-2S bitcell are investigated to present a 2T-2R-2S mode and SRAM-RRAM hybrid mode of operation, further utilized in performing Compute in Memory(CIM). The above functionalities can be presented in a 8T2R bitcell, that makes use of transistor in place of PTM. 2T-2R-2S/2T-2R mode is a fully non-differential mode of operation, leveraging only the NVM portion of the bitcell. Furthermore, SRAM-RRAM hybrid mode is proposed making use of the SRAM read port transistors to perform read operation of the data stored on the RRAM, during standby mode. The architecture study of set-associative cache made of 6T-2R-2S array is also presented. The 2T-2R-2S/2T-2R mode coupled with SRAM-only mode can be efficiently used to perform CIM for dot product and XNOR computation with co-locating the weights and activations stored onto the same bitcell. System level study highlighting energy efficiency and speedup along with the proposed CIM architecture’s analysis on CIFAR-10 dataset is presented. Design sensitivity analysis with respect to PTM and RRAM parameters is discussed for both the bitcells.

Dual-Port Content Addressable Memory for Cache Memory Applications

Multicore systems oftentimes use multiple levels of cache to bridge the gap between processor and memory speed. This paper presents a new design of a dedicated pipeline cache memory for multicore processors called dual port content addressable memory (DPCAM). In addition, it proposes a new replacement algorithm based on hardware which is called a near-far access replacement algorithm (NFRA) to reduce the cost overhead of the cache controller and improve the cache access latency. The experimental results indicated that the latency for write and read operations are significantly less in comparison with a set-associative cache memory. Moreover, it was shown that a latency of a read operation is nearly constant regardless of the size of DPCAM. However, an estimation of the power dissipation showed that DPCAM consumes about 7% greater than a set-associative cache memory of the same size. These results encourage for embedding DPCAM within the multicore processors as a small shared cache memory.

Optimization of data allocation in hierarchical memory for blocked shortest paths algorithms

Thispaper is devoted to the reduction of data transfer between the main memory and direct mapped cache for blocked shortest paths algorithms (BSPA), which represent data by a D[M×M] matrix of blocks. For large graphs, the cache size S = δ×M2, δ &lt; 1 is smaller than the matrix size. The cache assigns a group of main memory blocks to a single cache block. BSPA performs multiple recalculations of a block over one or two other blocks and may access up to three blocks simultaneously. If the blocks are assigned to the same cache block, conflicts occur among the blocks, which imply active transfer of data between memory levels. The distribution of blocks on groups and the block conflict count strongly depends on the allocation and ordering of the matrix blocks in main memory. To solve the problem of optimal block allocation, the paper introduces a block conflict weighted graph and recognizes two cases of block mapping: non-conflict and minimum-conflict. In first case, it formulates an equitable color-class-size constrained coloring problem on the conflict graph and solves it by developing deterministic and random algorithms. In second case, the paper formulates a problem of weighted defective color-count constrained coloring of the conflict graph and solves it by developing a random algorithm. Experimental results show that the equitable random algorithm provides an upper bound of the cache size that is very close to the lower bound estimated over the size of a complete subgraph, and show that a non-conflict matrix allocation is possible at δ = 0.5 for M = 4 and at δ = 0.1 for M = 20. For a low cache size, the weighted defective algorithm gives the number of remaining conflicts that is up to 8.8 times less than the original BSPA gives. The proposed model and algorithms are applicable to set-associative cache as well.

ECC-United Cache: Maximizing Efficiency of Error Detection/Correction Codes in Associative Cache Memories

Error Detection/Correction Codes (EDCs/ECCs) are the most conventional approaches to protect on-chip caches against radiation-induced soft errors. The overhead of EDCs/ECCs is a major concern and is of decisive importance when a higher protection capability is required to tolerate multiple adjacent bit errors (burst errors). This article proposes the ECC-United Cache (EUC) architecture to improve the efficiency of EDCs/ECCs in set-associative L1 caches. EUC architecture extends the data protection granularity from a single word to multiple words by exploiting the parallel cache lines access, which is inherently available in the cache. As compared with the conventional architecture, EUC can be configured to provide: 1) the same protection capability with a significantly lower overhead, 2) a significantly higher protection capability with the same number of check bits, or 3) a trade-off between the former two features. Simulation results show that, when configured to minimize the overhead, EUC reduces the number of check bits by 69 and 75 percent in data-cache and instruction-cache, respectively. When configured to maximize the protection capability, EUC provides fourfold higher burst error detection/correction capability. Moreover, EUC is orthogonal to previous protection schemes and they can be redesigned based on the EUC architecture to further improve their efficiency.

Small-LRU: A Hardware Efficient Hybrid Replacement Policy

Replacement policy plays a major role in improving the performance of the modern highly associative cache memo-ries. As the demand of data intensive application is increasing it is highly required that the size of the Last Level Cache (LLC) must be increased. Increasing the size of the LLC also increases the associativity of the cache. Modern LLCs are divided into multiple banks where each bank is a set-associative cache. The replacement policy implemented on such highly associative banks consume significant hardware (storage and area) overhead. Also the Least Recently Used (LRU) based replacement policy has an issue of dead blocks. A block in the cache is called dead, if the block is not used in the future before its eviction from the cache. In LRU policy, a dead block can not be remove early until it become LRU-block. So, we have proposed a replacement technique which is capable of removing dead block early with reduced hardware cost between 77% to 91% in comparison to baseline techniques. In this policy random replacement is used for 70% ways and LRU is applied for rest of the ways. The early eviction of dead blocks also improves the performance of the system by 5%.

Parloom: A New Low-Power Set-Associative Instruction Cache Architecture Utilizing Enhanced Counting Bloom Filter and Partial Tags

The cache system dissipates a significant amount of energy compared to the other memory components. This will be intensified if a cache is designed with a set-associative structure to improve the system performance because the parallel accesses to the entries of a set for tag comparisons lead to even more energy consumption. In this paper, a novel method is proposed as a combination of a counting Bloom filter and partial tags to mitigate the energy consumption of set-associative caches. This new hybrid method noticeably decreases the cache energy consumption especially in highly-associative instruction caches. In fact, it uses an enhanced counting Bloom filter to predict cache misses with a high accuracy as well as partial tags to decrease the overall cache size. This way, unnecessary tag comparisons can be prevented and therefore, the cache energy consumption is considerably reduced. Based on the simulation results, the proposed method provides the energy reduction from 22% to 31% for 4-way–32-way set-associative L1 caches bigger than 16[Formula: see text]kB running the MiBench programs. The improvements are attained with a negligible system performance degradation compared to the traditional cache system.

Segmented Tag Cache: A Novel Cache Organization for Reducing Dynamic Read Energy

A set-associative cache organization is widely used to achieve a high hit-rate in modern caches, leading to a considerable enhancement in the system performance. In a conventional set-associative cache, tag and data arrays are simultaneously accessed to achieve fast access, but doing so causes a large amount of energy consumption. Previous attempts, such as way prediction and sequential tag access for energy reduction, are still associated with substantial energy consumption due to the tag access. This paper presents a new cache organization termed Segmented Tag Cache (STC), which reduces the amount of energy consumed during the tag access. The proposed organization initially implements a new tag organization that supports partial tag access. More specifically, the tag array is segmented into two parts, and partial tag access only reads low-order part of the tag. A cache access scheme suitable for the proposed tag organization then are developed. Under the scheme, the delay of the tag organization is hidden by the data access delay, avoiding an increase in the overall cache access time to be maintained. Simulation results show that the STC reduces the energy-delay product by approximately 58 percent compared to the conventional cache organizations with a negligible performance penalty.

A Varying Processor Cache Sets Architecture

Any processor cache has three parameters capacity, line size and associativity. Usually all three are fixed at design time. Algorithms to have variable cache sets are proposed in literature. This paper proposes a method to have variable cache sets logically. The cache comes with fixed sets. The cache is visualized to have logically any number of sets greater than or equal to one. An algorithm for line placement/replacement is proposed in this paper for this model. The proposed model is simulated with SPEC2K benchmarks using Simplescalar Toolkit for two level inclusive set associative cache system. A power saving of 8.4% for L1 cache size 512x4, 17.58% for 1024x4 and 31.3% for 2048x4 is observed compared with traditional set associative cache of same size. A power saving of 7.53% compared with model proposed in literature for L1 size 512x4, 7.64% for 1024x4 and 7.645% for 2048x4 is observed. The L2 cache size is fixed at 2048x8. The average memory access time (AMAT) is found to degrade compared with conventional set associative cache by 19.63% for L1 size of 512x4, 24.68% for 1024x4 and 2048x4. (Abstract)

FOS: a low-power cache organization for multicores

The cache hierarchy of current multicore processors typically consists of one or two levels of private caches per core and a large shared last-level cache. This approach incurs area and energy wasting due to oversizing the private cache space, data replication through the inclusive cache levels, as well as the use of highly set-associative caches. In this paper, we claim that although this is the commonly adopted approach, it presents important design issues that can be addressed by a more energy efficient organization. This work proposes Flat On-chip Storage (FOS), a novel cache organization that, aimed at addressing energy and area on low-power processors, resolves the mentioned issues. For this purpose, FOS combines L2 and L3 cache levels into a single one, organized as a flat space, and composed of a pool of private small cache slices. These slices are initially powered off to save energy, and they are powered on and assigned to cores provided that the system performance is expected to improve. To provide fast and uniform access from the private L1 caches to the FOS’s cache slices, multiple architectural challenges are overcome, which entails the design of a custom optical network-on-chip. Experimental results show that FOS achieves significant energy savings on both static and dynamic energy over conventional cache organizations with the same storage capacity. FOS static energy savings are as much as 60% over an electrically connected shared cache; these savings grow up to 75% compared to optically connected baselines. Moreover, despite deactivating part of the cache space, FOS achieves similar performance values as those achieved by conventional approaches.

Trade-off between Hit Rate and Hit Latency for Optimizing DRAM Cache

Due to the large storage capacity, high bandwidth and low latency, 3D DRAM is proposed to be the last level cache, referred to as DRAM cache. The hit rate and hit latency are two conflicting optimization goals for DRAM cache. To address this issue, we design a new DRAM organization that trades the lower hit rate for shorter hit latency by way-locator cache and novel cache set layout. We have designed a novel DRAM cache organization to simultaneously achieve a good hit rate and shorter latency, referred to as SODA-cache. The SODA-cache adapts 2-way set associate cache motivated by the observation that 2-way set associative cache provides the most hit rate improvement from the direct-mapped cache to highly associative cache. The proposed way-locator cache and a novel set layout effectively reduce the cache-hit latency. We use SPEC 2006 CPU benchmark to evaluate our design and two stat-of-art DRAM cache designs. Experimental results show that SODA-cache can improve hit rate by 8.1 percent compared with Alloy-cache and reduce average access latency by 23.1, 13.2 and 8.6 percent compared with LH-cache, Alloy-cache and ATCache respectively on average. Accordingly, SODA-cache outperforms over LH-cache, Alloy-cache and ATCache by on average 17, 12.8 and 8.4 percent respectively in term of weighted speedup.

Analytical modeling of cache behavior for affine programs

Optimizing compilers implement program transformation strategies aimed at reducing data movement to or from main memory by exploiting the data-cache hierarchy. However, instead of attempting to minimize the number of cache misses, very approximate cost models are used, due to the lack of precise compile-time models for misses for hierarchical caches. The current state of practice for cache miss analysis is based on accurate simulation. However, simulation requires time proportional to the dataset/problem size, as well as the number of distinct cache configurations of interest to be evaluated. This paper takes a fundamentally different approach, by focusing on polyhedral programs with static control flow. Instead of relying on costly simulation, a closed-form solution for modeling of misses in a set associative cache hierarchy is developed. This solution can enable program transformation choice at compile time to optimize cache misses. A tool implementing the approach has been developed and used for validation of the framework.

Reducing the second-level cache conflict misses using a set folding technique

The cache memory has a direct effect on the performance of a computer system. Instructions and data are fetched from a fast cache instead of a slow memory to save hundreds of cycles. Reducing the cache miss ratio will definitely improve the execution time of an application. In this work, we propose cache memory designs that reduce the number of conflict misses significantly. The proposed designs reduce the conflict misses in the last level multi-way set associative cache. Each set is divided into a group of subsets: the first is referred to as the exclusive subset, and the rest are the shared subsets. The exclusive is configured as a traditional cache where each block is mapped to the set whose index matches the block index. In addition to their standard cache indexing role, the shared subsets are configured to host blocks with different indices. A memory block can be mapped to one subset from the exclusive type or one of multiple subsets from the shared type. Since the proposed technique is based on combining multiple sets of the shared part to form a larger set, that is shared between memory blocks with different indices, we have chosen the name “set folding.” The decision as to where to map a memory block depends on the number of misses encountered at each of the potential target sets. To evaluate the proposed design based on the overall hit rate, twenty-three benchmarks from SPEC CPU 2006 were simulated using the SuperESCalar simulator. The proposed designs require a few extra storage bits which adds a small overhead on the hardware complexity in comparison with the conventional cache. However, the proposed designs achieve lower miss rates for most of the benchmarks.

Performance linked dynamic cache tuning: A static energy reduction approach in tiled CMPs

Advancement in semiconductor technology increases power density in recent Chip Multi-Processors (CMPs) which significantly increases the leakage energy consumptions of on-chip Last Level Caches (LLCs). Performance linked dynamic tuning in LLC size is a promising option for reducing the cache leakage.This paper reduces static power consumption by dynamically shutting down or turning on cache banks based upon system performance and cache bank usage statistics. Shutting down of a cache bank remaps its future requests to another active bank, called as target bank. The proposed method is evaluated on three different implementation policies, viz (1) The system can decide to shutdown or turn-on some cache banks periodically throughout the process execution. (2) The system allows to shutdown banks initially and once the bank restarting initiates, no more shutdown is permitted further. (3) This policy resizes cache like first policy with some predefined time slices, in which cache cannot be resized.For a 4MB 4 way set associative L2 cache, experimental analysis shows 66% reduction in static energy with 29% gain in Energy Delay Product (EDP) for first strategy; for the second policy, static power is reduced by 59% with 27% savings in EDP. Finally, last policy saves 65% in static power and 30% in EDP with minimal performance penalty.

On the assessment of probabilistic WCET estimates reliability for arbitrary programs

Measurement-Based Probabilistic Timing Analysis (MBPTA) has been shown to be an industrially viable method to estimate the Worst-Case Execution Time (WCET) of real-time programs running on processors including several high-performance features. MBPTA requires hardware/software support so that program’s execution time, and so its WCET, has a probabilistic behaviour and can be modelled with probabilistic and statistic methods. MBPTA also requires that those events with high impact on execution time are properly captured in the (R) runs made at analysis time. Thus, a representativeness argument is needed to provide evidence that those events have been captured.This paper addresses the MBPTA representativeness problems caused by set-associative caches and presents a novel representativeness validation method (ReVS) for cache placement. Building on cache simulation, ReVS explores the probability and impact (miss count) of those cache placements that can occur during operation. ReVS determines the number of runs R′, which can be higher than R, such that those cache placements with the highest impact are effectively observed in the analysis runs, and hence, MBPTA can be reliably applied to estimate the WCET.

ANALYSIS OF ASSOCIATIVITY AND CONFLICT MISSES

Cache memory is playing a huge role in determining the performance when solving scientific problems. Most of these problems include a high number of repetition of complex or simple calculations on various data elements stored as arrays in sequential order in the memory. When executing the algorithms, these elements are brought to cache and then used by the processor. This process is usually followed by conflict and capacity misses in the cache, and the performance is degraded by a cache placement function, replacement policy or capacity constraints. In this paper we analyze the algorithms and performance impact of set associative caches when a large array is referenced in sequentially ordered memory. We map the problem of cache use into an IT related mathematical model to analyze the performance and give a scientific explanation for performance drops due to associativity and conflict cache misses in caches.

Newcache: Secure Cache Architecture Thwarting Cache Side-Channel Attacks

Newcache is a secure cache that can thwart cache side-channel attacks to prevent the leakage of secret information. All caches today are susceptible to cache side-channel attacks, despite software isolation of memory pages in virtual address spaces or virtual machines. These cache attacks can leak secret encryption keys or private identity keys, nullifying any protection provided by strong cryptography. Newcache uses a novel dynamic, randomized memory-to-cache mapping to thwart contention-based side-channel attacks, rather than the static mapping used by conventional set-associative caches. In this article, the authors present an improved design of Newcache, in terms of security, circuit design and simplicity. They show Newcache's security against a suite of cache side-channel attacks. They evaluate Newcache's system performance for cloud computing, smartphone, and SPEC benchmarks and find that Newcache performs as well as conventional set-associative caches, and sometimes better. They also designed a VLSI test chip with a 32-Kbyte Newcache and a 32-Kbyte, eight-way, set-associative cache and verified that the access latency, power, and area of the two caches are comparable. These results show that Newcache can be used as L1 data and instruction caches to improve security without impacting performance.