Cache Access Research Articles

Macro Data Load: An Efficient Mechanism for Enhancing Loaded Data Reuse

This paper presents a study on macro data load, a novel mechanism to increase the amount of loaded data reuse within a processor. A macro data load brings into the processor a maximum-width data the cache port allows. In a 64-bit processor, for example, a byte load will bring a full 64-bit data from cache and save it in an internal hardware structure, while using for itself only the specified byte out of the 64-bit data. The saved data can be opportunistically reused by later loads internally, reducing relatively more expensive cache accesses. We present a comprehensive availability study using a generalized memory data reuse table (MDRT) to quantify available memory data reuse opportunities in a set of benchmark programs drawn from the SPEC2k and MiBench suites, and to demonstrate the efficacy of the proposed scheme. The macro data load mechanism is shown to open up significantly more loaded data reuse opportunities than previous schemes with no support for spatial locality. We observe 15.1 percent (SPEC2k integer), 20.9 percent (SPEC2k floating-point), and 45.8 percent (MiBench) more load-to-load forwarding instances when a 256-entry MDRT is used. We also describe a modified load store queue design as a possible implementation of the proposed concept. Our quantitative study using a realistic processor model shows that 21.3 percent, 14.8 percent, and 23.6 percent of L1 cache accesses in the SPEC2k integer, floating-point, and MiBench programs can be eliminated, resulting in a related energy reduction of 11.4 percent, 9.0 percent, and 14.3 percent on average, respectively.

Online Capacity Identification of Multitier Websites Using Hardware Performance Counters

Understanding server capacity is crucial to system capacity planning, configuration, and QoS-aware resource management. Conventional stress testing approaches measure server capacity offline in terms of application-level performance metrics like response time and throughput. They are limited in measurement accuracy and timeliness. In a multitier website, resource bottleneck often shifts between tiers as client access pattern changes. This makes the problem of online capacity measurement even more challenge. This paper presents an online measurement approach based on low-level hardware performance metrics such as instructions execution rate and cache access behavior. Such metrics together define a system internal running state. The measurement approach uses machine learning techniques to infer application-level performance at each tier from a set of selected hardware performance counters. A coordinated predictor is induced over individual tier-wide models to make global system performance prediction and identify the bottleneck when the system becomes overloaded. Experiments were conducted on a two-tier Tomcat/MySQL-configured website using TPC-W benchmarks. Experimental results demonstrated that this approach was able to achieve an overload prediction accuracy of higher than 90 percent for a priori known input traffic mix and over 85 percent accuracy even for traffic causing frequent bottleneck shifting. It costs less than 0.5 percent runtime overhead for data collection and no more than 50 ms for each online decision making.

An Adaptive Various-Width Data Cache for Low Power Design

Modern microprocessors employ caches to bridge the great speed variance between a main memory and a central processing unit, but these caches consume a larger and larger proportion of the total power consumption. In fact, many values in a processor rarely need the full-bit dynamic range supported by a cache. The narrow-width value occupies a large portion of the cache access and storage. In view of these observations, this paper proposes an Adaptive Various-width Data Cache (AVDC) to reduce the power consumption in a cache, which exploits the popularity of narrow-width value stored in the cache. In AVDC, the data storage unit consists of three sub-arrays to store data of different widths. When high sub-arrays are not used, they are closed to save its dynamic and static power consumption through the modified high-bit SRAM cell. The main advantages of AVDC are: 1) Both the dynamic and static power consumption can be reduced. 2) Low power consumption is achieved by the modification of the data storage unit with less hardware modification. 3) We exploit the redundancy of narrow-width values instead of compressed values, thus cache access latency does not increase. Experimental results using SPEC 2000 benchmarks show that our proposed AVDC can reduce the power consumption, by 34.83% for dynamic power saving and by 42.87% for static power saving on average, compared with a cache without AVDC.

An Evaluation of an OS-Based Coherence Scheme for Tiled CMPs

The interconnect mechanisms (shared bus or crossbar) used in current chip-multiprocessors (CMPs) are expected to become a bottleneck that prevents these architectures from scaling to a larger number of cores. Tiled CMPs offer better scalability by integrating relatively simple cores with a lightweight point-to-point interconnect. However, such interconnects make snooping impractical and, thus, require alternative solutions to cache coherence. In this article, we investigate a novel, cost-effective mechanism to support shared-memory parallel applications that forgoes hardware maintained cache coherence. This mechanism is based on the key ideas that mapping of lines to physical caches is done at the page level with OS support and that hardware supports remote cache accesses. We extend our previous work by investigating in detail the impact of system design parameters and extending the system to support multi-level cache hierarchies. Results show that the choice of implementation of multi-level cache hierarchies can have a significant impact on performance.

C-AMTE: A location mechanism for flexible cache management in chip multiprocessors

This paper describes Constrained Associative-Mapping-of-Tracking-Entries (C-AMTE), a scalable mechanism to facilitate flexible and efficient distributed cache management in large-scale chip multiprocessors (CMPs). C-AMTE enables fast locating of cache blocks in CMP cache schemes that employ one-to-one or one-to-many associative mappings. C-AMTE stores in per-core data structures tracking entries to avoid on-chip interconnect traffic outburst or long distance directory lookups. Simulation results using a full system simulator demonstrate that C-AMTE achieves improvement in cache access latency by up to 34.4%, close to that of a perfect location strategy.

Energy- and Performance-Efficient Communication Framework for Embedded MPSoCs through Application-Driven Release Consistency

We present a framework for performance-, bandwidth-, and energy-efficient intercore communication in embedded MultiProcessor Systems-on-a-Chip (MPSoC). The methodology seamlessly integrates compiler, operating system, and hardware support to achieve a low-cost communication between synchronized producers and consumers. The technique is especially beneficial for data-streaming applications exploiting pipeline parallelism with computational phases mapped to separate cores. Code transformations utilizing a simple ISA support ensure that producer writes are propagated to consumers with a single interconnect transaction per cache block just prior to the producer exiting its synchronization region. Furthermore, in order to completely eliminate misses to shared data caused by interference with private data and also to minimize the cache energy, we integrate to the proposed framework a cache way partitioning policy based on a simple cache configurability support, which isolates the shared buffers from other cache traffic. This mechanism results in significant power savings since only a subset of the cache ways needs to be looked up for each cache access. The end result of the proposed framework is a single communication transaction per shared cache block between a producer and a consumer with no coherence misses on the consumer caches. Our experiments demonstrate significant reductions in interconnect traffic, cache misses, and energy for a set of multiprocessor benchmarks.

A hybrid cache placement scheme for multi-hop wireless service network

In this paper, a hybrid cache placement scheme for multihop wireless service networks is proposed. In this scheme, hot nodes in data transferring path are mined up by means of routing navigation graph, and whole network is covered with network clustering scheme. A hot node has been chosen for cache placement in each cluster, and the nodes within a cluster access cache data with no more than two hops. The cache placement scheme reduces data access latency and workload of the server node. It also reduces the average length of data transferring, which means that fewer nodes are involved. The network system energy consumption decreased as involved relay nodes reduced. The performance analysis shows that the scheme achieves significant system performance improvement in network environment, with a large number of nodes.

Design and Analysis of On-Chip Networks for Large-Scale Cache Systems

Switched networks have been adopted in on-chip communication for their scalability and efficient resource sharing. However, using a general network for a specific domain may result in unnecessary high cost and low performance when the interconnects are not optimized for the domain. Designing an optimal network for the specific domain is challenging because in-depth knowledge of interconnects and the application domain is required. Recently proposed Nonuniform Cache Architectures (NUCAs) use wormhole-routed 2D mesh networks in L2 caches. We observe that in NUCAs, network resources are underutilized with the considerable area cost (41 percent of cache) and the network delay is significantly large (63 percent of cache access time). Motivated by our observations, we investigate both router architecture and network topology for communication behaviors in large-scale cache systems. We present Fast-LRU replacement, where cache replacement overlaps with data request delivery. Next, we propose a deadlock-free XYX routing algorithm in a mesh network and present a new halo network topology to reduce the required links. Finally, we introduce a single-cycle multicast router that needs small modification of the unicast router design. Simulation results show that our design improves the average IPC by 38 percent over the mesh design with Multicast Promotion replacement and uses 12 percent of the interconnection area of the mesh network.

Aggressive drowsy cache cells

An aggressive drowsy cache block management, where the cache block is forced into drowsy mode all the time except during write and read operations, is proposed. The word line (WL) is used to enable the normal supply voltage (V DD_high) to the cache line only when it is accessed for read or write whereas the drowsy supply voltage (V DD_low) is enabled to the cache cell otherwise. The proposed block management neither needs extra cycles nor extra control signals to wake the drowsy cache cell, thereby reducing the performance penalty associated with traditional drowsy caches. In fact, the proposed aggressive drowsy mode can reduce the total power consumption of the traditional drowsy mode by 13% or even more, depending on the cache access rate, access frequency and the CMOS technology used.

Low-power L2 cache design for multi-core processors

A low-power set-associative L2 cache design for a multi-core processor is proposed. Since this way-predicting L2 cache (WP-L2) predicts a destination way and accesses only the predicted way, it consumes less energy than a conventional set-associative L2 cache. Exploiting access patterns of an L2 cache, WP-L2 is based on two prediction logics; a look-ahead buffer (LAB) predicts the next sequential cache block and a way-affinity table (WAT) records the way number of the previous L2 cache access. Combining the logics, WP-L2 predicts correct ways for about 83% of L2 cache accesses and reduces about 22% of access latency and 44% of energy consumption compared to the conventional eight-way set-associative L2 cache.

On reducing load/store latencies of cache accesses

Effective address calculations for load and store instructions need to compete for ALU with other instructions and hence extra latencies might be incurred to data cache accesses. Fast address generation is an approach proposed to reduce cache access latencies. This paper presents a fast address generator that can eliminate most of the effective address computations by storing computed effective addresses of previous load/store instructions in a dummy register file. Experimental results show that this fast address generator can reduce effective address computations of load and store instructions by about 74% on average for SPECint2000 benchmarks and cut the execution times by 8.5%. Furthermore, when multiple dummy register files are deployed, this fast address generator eliminates over 90% of effective address computations of load and store instructions and improves the average execution times by 9.3%.

A Novel instruction stream buffer for VLIW architectures

The instruction compression mechanism used to solve the drawbacks of traditional very long instruction word (VLIW) architectures often leads to poor code density in the instruction cache, which causes the irregular lengths of long instructions to cross the different cache line. These split long instructions cannot be fetched simultaneously, which creates a bottleneck for VLIW architectures. This paper proposes a buffing mechanism which can slide the split long instruction as a continuous form to offer better efficiency in instruction fetching. This approach helps maintain the behaviors of the software pipeline technology, which schedules iterative instructions to enhance the performance of streaming processing for VLIW architectures. In the proposed mechanism, the instruction stream buffer stores the repeat block completely and suspends as far as possible the cache access to reduce access time. The advantages of repeatedly issuing instructions in the instruction buffer and preventing split long instructions, can substantially improve the performance in fetching instructions. Simulation results show that the mechanism is efficient at the instruction level for the basic DSP/IMG library by improving performance by 35% on average.

Process-Variation-Aware Adaptive Cache Architecture and Management

Fabricating circuits that employ ever-smaller transistors leads to dramatic variations in critical process parameters. This in turn results in large variations in execution/access latencies of different hardware components. This situation is even more severe for memory components due to minimum-sized transistors used in their design. Current design methodologies that are tuned for the worst case scenarios are becoming increasingly pessimistic from the performance angle, and thus, may not be a viable option at all for future designs. This paper makes two contributions targeting on-chip data caches. First, it presents an adaptive cache management policy based on nonuniform cache access. Second, it proposes a latency compensation approach that employs several circuit-level techniques to change the access latency of select cache lines based on the criticalities of the load instructions that access them. Our experiments reveal that both these techniques can recover significant amount of the lost performance due to worst case designs.

Reactive NUCA

Increases in on-chip communication delay and the large working sets of server and scientific workloads complicate the design of the on-chip last-level cache for multicore processors. The large working sets favor a shared cache design that maximizes the aggregate cache capacity and minimizes off-chip memory requests. At the same time, the growing on-chip communication delay favors core-private caches that replicate data to minimize delays on global wires. Recent hybrid proposals offer lower average latency than conventional designs, but they address the placement requirements of only a subset of the data accessed by the application, require complex lookup and coherence mechanisms that increase latency, or fail to scale to high core counts. In this work, we observe that the cache access patterns of a range of server and scientific workloads can be classified into distinct classes, where each class is amenable to different block placement policies. Based on this observation, we propose Reactive NUCA (R-NUCA), a distributed cache design which reacts to the class of each cache access and places blocks at the appropriate location in the cache. R-NUCA cooperates with the operating system to support intelligent placement, migration, and replication without the overhead of an explicit coherence mechanism for the on-chip last-level cache. In a range of server, scientific, and multiprogrammed workloads, R-NUCA matches the performance of the best cache design for each workload, improving performance by 14% on average over competing designs and by 32% at best, while achieving performance within 5% of an ideal cache design.

Memory mapped ECC

This paper presents a novel technique, Memory Mapped ECC, which reduces the cost of providing error correction for SRAM caches. It is important to limit such overheads as processor resources become constrained and error propensity increases. The continuing decrease in SRAM cell size and the growing capacity of caches increases the likelihood of errors in SRAM arrays. To address this, redundant information can be used to correct a value after an error occurs. Information redundancy is typically provided through error-correcting codes (ECC), which append bits to every SRAM row and increase the array's area and energy consumption. We make three observations regarding error protection and utilize them in our architecture: (1) much of the data in a cache is replicated throughout the hierarchy and is inherently redundant; (2) error-detection is necessary for every cache access and is cheaper than error correction, which is very infrequent; (3) redundant information for correction need not be stored in high-cost SRAM. Our unique architecture only dedicates SRAM for error detection while the ECC bits are stored within the memory hierarchy as data. We associate a physical memory address with each cache line for ECC storage and rely on locality to minimize the impact. The cache is dynamically and transparently partitioned between data and ECC with the fraction of ECC growing with the number of dirty cache lines. We show that this has little impact on both performance (1.3% average and < 4%) and memory traffic (3%) across a range of memory-intensive applications.

A High Density and Low Power Cache Based on Novel SRAM Cell

Based on the observation that dynamic occurrence of zeros in the cache access stream and cacheresident memory values of ordinary programs exhibit a strong bias towards zero, this paper presents a novel CMOS five-transistor SRAM cell (5T SRAM cell) for very high density and low power cache applications. This cell retains its data with leakage current and positive feedback without refresh cycle. Novel 5T SRAM cell uses one word-line and one bit-line and extra read-line control. The new cell size is 17% smaller than a conventional six-transistor SRAM cell using same design rules with no performance degradation. Simulation and analytical results show purposed cell has correct operation during read/write and also the average dynamic energy consumption of new cell is 30% smaller than a six-transistor SRAM cell.

Impact of on-chip network parameters on NUCA cache performances

Non-uniform cache architectures (NUCAs) are a novel design paradigm for large last-level on-chip caches, which have been introduced to deliver low access latencies in wire-delay-dominated environments. Their structure is partitioned into sub-banks and the resulting access latency is a function of the physical position of the requested data. Typically, NUCA caches employ a switched network, made up of links and routers with buffered queues, to connect the different sub-banks and the cache controller, and the characteristics of the network elements may affect the performance of the entire system. This work analyses how different parameters for the network routers, namely cut-through latency and buffering capacity, affect the overall performance of NUCA-based systems for the single processor case, assuming a reference NUCA organisation proposed in literature. The entire analysis is performed utilising a cycle-accurate execution-driven simulator of the entire system and real workloads. The results indicate that the sensitivity of the system to the cut-through latency is very high, thus limiting the effectiveness of the NUCA solution, and that modest buffering capacity is sufficient to achieve a good performance level. As a consequence, in this work we propose an alternative clustered NUCA organisation that limits the average number of hops experienced by cache accesses. This organisation is better performing and scales better as the cut-through latency increases, thus simplifying the implementation of routers, and it is also more effective than another latency reduction solution proposed in literature (hybrid network).

A low power and high density cache memory based on novel SRAM cell

Based on the observation that dynamic occurrence of zeros in the cache access stream and cache-resident memory values of ordinary programs exhibit a strong bias towards zero, this paper presents a novel CMOS four-transistor (4T) SRAM cell for very high density and low power cache applications. This cell retains its data with leakage current and positive feedback without refresh cycle. The new cell size is 20% smaller than a conventional six-transistor cell using same design rules and delay access of a cache based on new 4T SRAM cell is 32% smaller than a cache based on 6T SRAM cell. Also the dynamic and static power consumption of new cell is 40% and 20% smaller than 6T SRAM cell, respectively.

Scalability study of cache access mechanisms in multiple-cell wireless networks

Strong-consistent cache access algorithms have been proposed and studied for wireless networks. In this paper, we carry out a comprehensive study to compare four strong-consistent cache access algorithms including the invalidation report (IR), poll-each-read (PER), call-back (CB), and Lease schemes in multiple-cell wireless/mobile networks. The purpose of this study is to examine the scalability of the IR schemes. To the best of our knowledge, this is the first such study. We focus on the scalability issue of these four fundamental strong-consistent schemes in terms of network transmission costs regarding network size, database size, subscription ratio, and network traffic, through extensive computer simulations. Our results show that: (1) the IR schemes do not perform well in a multiple-cell wireless/mobile networks with a large update rate, database size, and IR window size, and with a small subscription ratio; (2) the IR schemes do scale up well with the IR period and the access rate; however, while good performance can be obtained when the IR period is large, this implies a large access latency; (3) the PER, the CB and the Lease schemes perform well with a large update rate, small subscription ratio, and large database size.

Design of a low‐power way‐predicting cache using valid‐bit pre‐decision strategy

Way‐predicting cache is one of the set‐associative caches that can effectively reduce power consumption, which only speculatively selects an MRU (most recently used) way before starting a normal cache access. Focusing on the way‐predicting cache using sub‐block placement, we propose a new cache scheme that uses valid bits from data memory to decide the disabled tag‐subarrays and data‐subarrays. By using valid‐bit pre‐decision, the proposed scheme has a significant improvement in average energy saving over the conventional way‐predicting cache, especially for large associativity and small sub‐block size. Moreover, because those original second accessed ways are first accessed during the first cycle when the valid bit of the MRU way does not exist, the proposed cache also can reduce the average access time.