Cache Configurations Research Articles

Online cache modeling for commodity multicore processors

Modern chip-level multiprocessors (CMPs) contain multiple processor cores sharing a common last-level cache, memory interconnects, and other hardware resources. Workloads running on separate cores compete for these resources, often resulting in highlyvariable performance. It is generally desirable to co-schedule workloads that have minimal resource contention, in order to improve both performance and fairness. Unfortunately, commodity processors expose only limited information about the state of shared resources such as caches to the software responsible for scheduling workloads that execute concurrently. To make informed resourcemanagement decisions, it is important to obtain accurate measurements of per-workload cache occupancies and their impact on performance, often summarized by utility functions such as miss-ratio curves (MRCs) In this paper, we first introduce an efficient online technique for estimating the cache occupancy of individual software threads using only commonly-available hardware performance counters. We derive an analytical model as the basis of our occupancy estimation, and extend it for improved accuracy on modern cache configurations, considering the impact of set-associativity, line replacement policy, and memory locality effects. We demonstrate the effectiveness of occupancy estimation with a series of CMP simulations in which SPEC benchmarks execute concurrently on multiple cores. Leveraging our occupancy estimation technique, we also introduce a lightweight approach for online MRC construction, and demonstrate its effectiveness using a prototype implementation in the VMware ESX Server hypervisor. We present a series of experiments involving SPEC benchmarks, comparing the MRCs we construct online with MRCs generated offline in which various cache sizes are enforced via static page coloring

A Combined Optimization Method for Tuning Two-Level Memory Hierarchy Considering Energy Consumption

Tuning cache hierarchies in platforms for embedded systems can significantly reduce energy consumption. In this paper we combined two optimization methods for tuning both instruction and data cache configurations in a two-level memory hierarchy, where both levels have separate instruction and data caches. This kind of hierarchy allows us to evaluate instruction and data caches branches separately, although previous approaches have applied the same method for both branches of the hierarchy. This work evaluates several methods intended for two-level hierarchies, and the results showed that when we combine different methods for each branch of the hierarchy, results can be improved. Experiments based on simulations were performed for 12 applications from the Mibench suite benchmark and the combined method achieved better efficiency in 60% of the evaluated cases compared with existing heuristics. The proposed solution is only 11% less economic in terms of energy consumption than optimal values and required, on average, 42 simulations to conclude optimization mechanism, representing only 9% of the design space.

High performance cache replacement using re-reference interval prediction (RRIP)

Practical cache replacement policies attempt to emulate optimal replacement by predicting the re-reference interval of a cache block. The commonly used LRU replacement policy always predicts a near-immediate re-reference interval on cache hits and misses. Applications that exhibit a distant re-reference interval perform badly under LRU. Such applications usually have a working-set larger than the cache or have frequent bursts of references to non-temporal data (called scans). To improve the performance of such workloads, this paper proposes cache replacement using Re-reference Interval Prediction (RRIP). We propose Static RRIP (SRRIP) that is scan-resistant and Dynamic RRIP (DRRIP) that is both scan-resistant and thrash-resistant. Both RRIP policies require only 2-bits per cache block and easily integrate into existing LRU approximations found in modern processors. Our evaluations using PC games, multimedia, server and SPEC CPU2006 workloads on a single-core processor with a 2MB last-level cache (LLC) show that both SRRIP and DRRIP outperform LRU replacement on the throughput metric by an average of 4% and 10% respectively. Our evaluations with over 1000 multi-programmed workloads on a 4-core CMP with an 8MB shared LLC show that SRRIP and DRRIP outperform LRU replacement on the throughput metric by an average of 7% and 9% respectively. We also show that RRIP outperforms LFU, the state-of the art scan-resistant replacement algorithm to-date. For the cache configurations under study, RRIP requires 2X less hardware than LRU and 2.5X less hardware than LFU.

Cache partitioning for energy-efficient and interference-free embedded multitasking

We propose a technique that leverages configurable data caches to address the problem of energy inefficiency and intertask interference in multitasking embedded systems. Data caches are often necessary to provide the required memory bandwidth. However, caches introduce two important problems for embedded systems. Caches contribute to a significant amount of power as they typically occupy a large part of the chip and are accessed frequently. In nanometer technologies, such large structures contribute significantly to the total leakage power as well. Additionally, cache outcomes in multitasking environments are notoriously difficult to predict, if not impossible, thus resulting in poor real-time guarantees. We study the effect of multiprogramming workloads on the data cache in a preemptive multitasking environment, and propose a technique which leverages configurable cache architectures to not only eliminate intertask cache interference, but also to significantly reduce both dynamic and leakage power. By mapping tasks to different cache partitions, interference is completely eliminated. Dynamic and leakage power are significantly reduced as only a subset of the cache is active at any moment. We introduce a profile-based, off-line algorithm, which identifies a beneficial cache partitioning. The OS configures the data cache during context-switch by activating the corresponding partition. Our experiments on a large set of multitasking benchmarks demonstrate that our technique not only efficiently eliminates intertask interference, but also significantly reduces both dynamic and leakage power.

Dynamic Context Compression for Low-Power Coarse-Grained Reconfigurable Architecture

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Most of the coarse-grained reconfigurable architectures (CGRAs) are composed of reconfigurable ALU arrays and configuration cache (or context memory) to achieve high performance and flexibility. Specially, configuration cache is the main component in CGRA that provides distinct feature for dynamic reconfiguration in every cycle. However, frequent memory-read operations for dynamic reconfiguration cause much power consumption. Thus, reducing power in configuration cache has become critical for CGRA to be more competitive and reliable for its use in embedded systems. In this paper, we propose dynamically compressible context architecture for power saving in configuration cache. This power-efficient design of context architecture works without degrading the performance and flexibility of CGRA. Experimental results show that the proposed approach saves up to 39.72% power in configuration cache with negligible area overhead (2.16%). </para>

Real-time reconfigurable cache for low-power embedded systems

Modern embedded systems execute a small set of applications or even a single one repeatedly. Specialising cache configurations to a particular application is well-known to have great benefits on performance and power. However, the fact that the behaviour of an application varies from phase to phase has been shown in recent years. Tuning cache configuration to fit a target application in different phases gives a further improvement in power consumption. This work presents a mechanism which determines the optimal configurations in different phases during an execution process. By applying corresponding cache configuration for each time interval of an execution process on L1 instruction cache, this work shows that on average 91.6% energy saving is obtained by comparing with average energy consumption of all four-way set-associative caches in search space. On average 5.29% power reduction is achieved by comparing with energy consumption of benchmarks with their respective global optimal cache configurations.

Replica victim caching to improve cache reliability against transient errors

Future cache memories must deal with soft errors to ensure data reliability. This paper investigates the addition of a small cache – replica victim cache, to accommodate the replicas that are frequently evicted from the L1 data cache due to in-cache replication (ICR) for further enhancing the data reliability against soft errors. Our experimental results reveal that a replica victim cache with only four entries can increase the reliability of the L1 data cache by 21.7% more than ICR without impacting performance significantly. Also, the area overhead is less than 10% for most L1 data cache configurations. We believe the proposed scheme provides an interesting cache design option, especially for the applications that are operated in highly noisy environments or demand very high data integrity.

AUGMENTED FIFO CACHE REPLACEMENT POLICIES FOR LOW-POWER EMBEDDED PROCESSORS

This paper explores a family of augmented FIFO replacement policies for highly setassociative caches that are common in low-power embedded processors. In such processors, the implementation cost and complexity of the replacement policy is as important as the cache hit rate. By exploiting the cache hit way information between two replacements, the proposed replacement schemes reduce cache misses by 1% to 18% on average depending on the cache configuration, compared with the conventional FIFO policy. The proposed schemes come at a small implementation cost of additional state bits and control logic. The reduction in cache misses directly translates into data access energy savings of 1% to 15% on average, depending on the cache configuration. Our work suggests that there is room for improving the popular FIFO policy at a small cost.

Partially Protected Caches to Reduce Failures Due to Soft Errors in Multimedia Applications

With advances in process technology, soft errors are becoming an increasingly critical design concern. Owing to their large area, high density, and low operating voltages, caches are worst hit by soft errors. Based on the observation that in multimedia applications, not all data require the same amount of protection from soft errors, we propose a partially protected cache (PPC) architecture, in which there are two caches, one protected and the other unprotected at the same level of memory hierarchy. We demonstrate that as compared to the existing unprotected cache architectures, PPC architectures can provide 47 times reduction in failure rate, at only 1% runtime and 3% power overheads. In addition, the failure rate reduction obtained by PPCs is very sensitive to the PPC cache configuration. Therefore, this observation provides an opportunity for further improvement of the solution by correctly parameterizing the PPC configurations. Consequently, we develop design space exploration (DSE) strategies to discover the best PPC configuration. Our DSE technique can reduce the exploration time by more than six times as compared to an exhaustive approach.

Low Power Reconfiguration Technique for Coarse-Grained Reconfigurable Architecture

Coarse-grained reconfigurable architectures (CGRAs) require many processing elements (PEs) and a configuration memory unit (configuration cache) for reconfiguration of its PE array. Although this structure is meant for high performance and flexibility, it consumes significant power. Specially, power consumption by configuration cache is explicit overhead compared to other types of intellectual property (IP) cores. Reducing power is very crucial for CGRA to be more competitive and reliable processing core in embedded systems. In this paper, we propose a reusable context pipelining (RCP) architecture to reduce power-overhead caused by reconfiguration. It shows that the power reduction can be achieved by using the characteristics of loop pipelining, which is a multiple instruction stream, multiple data stream (MIMD)-style execution model. RCP efficiently reduces power consumption in configuration cache without performance degradation. Experimental results show that the proposed approach saves much power even with reduced configuration cache size. Power reduction ratio in the configuration cache and the entire architecture are up to 86.33% and 37.19%, respectively, compared to the base architecture.

FlexDCP

Current multicore architectures offer high throughput by increasing hardware resource utilization. As the number of cores in a multicore system increases, providing Quality of Service (QoS) to applications in addition to throughput is becoming an important problem. In this work, we present FlexDCP, a framework that allows the Operating System (OS) to guarantee a QoS for each application running in a chip multiprocessor. FlexDCP directly estimates the performance of applications for different cache configurations instead of using indirect measures of performance like the number of misses. This information allows the OS to convert QoS requirements into resource assignments. Consequently, it offers more flexibility to the OS as it can optimize different QoS metrics like per-application performance or global performance metrics such as fairness, weighted speed up or throughput. Our results show that FlexDCP is able to force applications in a workload to run at a certain percentage of their maximum performance in 94% of the cases considered, being on average 1:48% under the objective for remaining cases. When optimizing a global QoS metric like fairness, FlexDCP consistently outperforms traditional eviction policies like LRU, pseudo LRU and previous dynamic cache partitioning proposals for two-, four- and eightcore configurations. In an eight-core architecture FlexDCP obtains a fairness improvement of 10:1% over Fair, the best policy in the literature optimizing fairness.

Fast Configurable-Cache Tuning With a Unified Second-Level Cache

Tuning a configurable cache subsystem to an application can greatly reduce memory hierarchy energy consumption. Previous tuning methods use a level one configurable cache only, or a second level with separate instruction and data configurable caches. We instead use a commercially-common unified second level cache, a seemingly minor difference that actually expands the configuration space from 500 to about 20 000. We develop additive way tuning for tuning a cache subsystem with this large space, yielding 61% energy savings and 9% performance improvements over a nonconfigurable cache, greatly outperforming an extension of a previous method.

An L1 Cache Design Space Exploration System for Embedded Applications

In an embedded system where a single application or a class of applications is repeatedly executed on a processor, its cache configuration can be customized such that an optimal one is achieved. We can have an optimal cache configuration which minimizes overall memory access time by varying the three cache parameters: the number of sets, a line size, and an associativity. In this paper, we first propose two cache simulation algorithms: CRCB1 and CRCB2, based on Cache Inclusion Property. They realize exact cache simulation but decrease the number of cache hit/miss judgments dramatically. We further propose three more cache design space exploration algorithms: CRMF1, CRMF2, and CRMF3, based on our experimental observations. They can find an almost optimal cache configuration from the viewpoint of access time. By using our approach, the number of cache hit/miss judgments required for optimizing cache configurations is reduced to 1/10-1/50 compared to conventional approaches. As a result, our proposed approach totally runs an average of 3.2 times faster and a maximum of 5.3 times faster compared to the fastest approach proposed so far. Our proposed cache simulation approach achieves the world fastest cache design space exploration when optimizing total memory access time.

고성능 DSP에서 동영상 인코더의 최적화 구현을 위한 캐쉬 및 내부 메모리 성능 분석

고성능 DSP는 보통 캐쉬와 내부 메모리를 지원한다. 이러한 고성능 DSP에 멀티미디어 스트림 응용을 최적화하여 구현하고자 하는 경우에는, DSP 가 지원하는 캐쉬와 내부 메모리를 효율적으로 잘 활용하여야 한다. 본 논문에서는 2단계 레벨 캐쉬 구조 및 내부 메모리 구성을 지원하는 고성능 DSP인 TMS320C6000 시리즈에 대해 동영상 인코더와 같은 멀티미디어 스트림 처리 응용을 최적으로 구현하기 위해서 필요한 캐쉬 성능 분석, 내부 메모리 구성 및 배치에 따른 성능 분석과 개선 방안에 대해 연구하였다. 분석 및 실험 결과, L2 메모리의 경우, 이중 집합연관 캐쉬로 구성하고, 남은 메모리는 내부 메모리로 구성하는 것이 수행 시간 성능 개선에 효과적임을 확인하였다. 또한, L1P 캐쉬의 경우는 자주 호출되고 시간이 많이 소요되는 루틴들을 연속적으로 내부 메모리에 배치하는 것이 L1P 캐쉬의 히트 율을 개선하며, L1D 캐쉬의 경우는 사용하는 데이터의 크기를 조절하므로 써 쉽게 히트 율을 개선할 수 있다는 것을 밝혔다. 본 논문의 연구 결과는 고성능 DSP 에 멀티미디어 스트림 처리 응용을 최적화로 구현하는데 도움을 줄 것으로 기대한다. High Performance DSP usually supports cache and internal memory. For an optimal implementation of a multimedia stream application on such a high performance DSP, one needs to utilize the cache and internal memory efficiently. In this paper, we investigate performance analysis of cache, and internal memory configuration and placement necessary to achieve an optimal implementation of multimedia stream applications like motion picture encoder on high performance DSP, TMS320C6000 series, and propose strategies to improve performance for cache and internal memory placement. From the results of analysis and experiments, it is verified that 2-way L2 cache configuration with the remaining memory configured as internal memory shows relatively good performance. Also, it is shown that L1P cache hit rate is enhanced when frequently called routines and routines having caller-callee relationships with them are continuously placed in the internal memory and that L1D cache hit rate is enhanced by the simple change of the data size. The results in the paper are expected to contribute to the optimal implementation of multimedia stream applications on high performance DSPs.

Temperature-Aware Configurable Cache to Reduce Energy in Embedded Systems

Energy consumption is a major concern in embedded computing systems. Several studies have shown that cache memories account for 40% or more of the total energy consumed in these systems. Active power used to be the primary contributor to total power dissipation of CMOS designs, but with the technology scaling, the share of leakage in total power consumption of digital systems continues to grow. Moreover, temperature is another factor that exponentially increases the leakage current. In this paper, we show the effect of temperature on the optimal (minimum-energy-consuming) cache configuration for low energy embedded systems. Our results show that for a given application and technology, the optimal cache size moves toward smaller caches at higher temperatures, due to the larger leakage. Consequently, a Temperature-Aware Configurable Cache (TACC) is an effective way to save energy in finer technologies when the embedded system is used in different temperatures. Our results show that using a TACC, up to 61% energy can be saved for instruction cache and 77% for data cache compared to a configurable cache that has been configured for only the corner-case temperature (100°C). Furthermore, the TACC also enhances the performance by up to 28% for the instruction cache and up to 17% for the data cache.

Investigating cache energy and latency break-even points in high performance processors

In this work we study how cache complexity impacts energy and performance in high performance processors. Moreover, we estimate cache energy budget for two high performance processors. We calculate energy and latency break-even points for realistic and ideal cache organizations for different applications. We show that design efforts made to reduce cache miss rate are only justifiable from the energy and performance point of view only if the associated latency and energy overhead remain below the calculated break-even points. Furthermore, we show that, for the processors and applications studied here, the instruction cache has a lower latency break-even point compared to the data cache. However, investing energy in the data cache is likely to result in better energy efficiency compared to the instruction cache. We also study alternative cache configurations for different processors and investigate if such alternatives would improve energy efficiency.

Evaluating instruction cache vulnerability to transient errors

Recent research shows that microprocessors are increasingly susceptible to transient errors. In order to protect microprocessors cost-effectively, the first step is to accurately understand the impact of transient errors on the system reliability. While many research efforts have been focused on studying the vulnerability of data caches and other on-chip hardware components, instruction caches have received less attention. However, instructions are read every cycle, any undetected or uncorrected soft errors in instructions can lead to erroneous computation, wrong control flow or system crash. This paper studies the instruction cache vulnerability by considering both the raw SRAM rate and the cache vulnerability factor. Based on the concept of cache vulnerability factor, we also investigate the impact of different cache configuration parameters on the reliability of instruction caches. We find that on average 67.5% of instruction cache soft errors can be masked by the I-cache itself without impacting other system components. While quantifying the instruction cache vulnerability itself does not solve the reliability problem of instruction cache against transient errors, we believe this work can provide useful insights for designers to develop cost-effective solutions to protect I-caches and to optimally balance the reliability of instruction caches with other system goals, such as cost, performance and energy.

Miss Rate Prediction Across Program Inputs and Cache Configurations

Improving cache performance requires understanding cache behavior. However, measuring cache performance for one or two data input sets provides little insight into how cache behavior varies across all data input sets and all cache configurations. This paper uses locality analysis to generate a parameterized model of program cache behavior. Given a cache size and associativity, this model predicts the miss rate for arbitrary data input set sizes. This model also identifies critical data input sizes where cache behavior exhibits marked changes. Experiments show this technique is within 2 percent of the hit rate for set associative caches on a set of floating-point and integer programs using array and pointer-based data structures. Building on the new model, this paper presents an interactive visualization tool that uses a three-dimensional plot to show miss rate changes across program data sizes and cache sizes and its use in evaluating compiler transformations. Other uses of this visualization tool include assisting machine and benchmark-set design. The tool can be accessed on the Web at http://www.cs.rochester.edu/research/locality

Energy-Efficient MAC-Layer Error Recovery for Mobile Multimedia Applications in 3GPP2 BCMCS

Combining broadcast and mobile phone technologies, 3GPP2 has introduced the Broadcast and Multicast Services (BCMCS) architecture to deliver multimedia content over cdma2000 1xEV-DO wireless networks. In designing a mobile device to support multimedia broadcast services, it is important to reduce delay and energy consumption, while maintaining a tolerable level of data loss. We analyse the energy consumed by a mobile device receiving broadcast services, focusing on error recovery using the Reed-Solomon (RS) MAC-layer coding scheme. Our model is based on the energy consumed by each computational component of the RS decoder, which we determined by running the decoding process on a realistic ARM7TDMI testbed with experimentally justified cache sizes, and characterizing the energy consumption accurately with the SNU Energy Explorer. By varying the radio channel conditions, RS coding scheme and other system parameters, we determined energy-efficient cache configurations and operating ranges for each RS encoding scheme, corresponding to given levels of service quality in a video application. We have also found that significant energy can be saved by selecting a size of error control block that is appropriate to the target video quality and the channel conditions at the mobile

Comprehensive multivariate extrapolation modeling of multiprocessor cache miss rates

Cache miss rates are an important subset of system model inputs. Cache miss rate models are used for broad design space exploration in which many cache configurations cannot be simulated directly due to limitations of trace collection setups or available resources. Often it is not practical to simulate large caches. Large processor counts and consequent potentially high degree of cache sharing are frequently not reproducible on small existing systems. In this article, we present an approach to building multivariate regression models for predicting cache miss rates beyond the range of collectible data. The extrapolation model attempts to accurately estimate the high-level trend of the existing data, which can be extended in a natural way. We extend previous work by its applicability to multiple miss rate components and its ability to model a wide range of cache parameters, including size, line size, associativity and sharing. The stability of extrapolation is recognized to be a crucial requirement. The proposed extrapolation model is shown to be stable to small data perturbations that may be introduced during data collection.We show the effectiveness of the technique by applying it to two commercial workloads. The wide design space contains configurations that are much larger than those for which miss rate data were available. The fitted data match the simulation data very well. The various curves show how a miss rate model is useful for not only estimating the performance of specific configurations, but also for providing insight into miss rate trends.