Cache Partitioning Research Articles

Cache isolation for virtualization of mixed general-purpose and real-time systems

Shared cache has been widely used to lessen the communication overhead between multi-core processors and off-chip main memories. In a virtualization execution environment, real-time tasks supported by real-time operating systems (RTOS) need to share the last level cache with other general purpose tasks (supported by general purpose operating systems (GPOS) or execute on bare metal). Since current hypervisors are not aware of the attributes of tasks in the virtual machines, real-time tasks cannot be adequately supported by the hypervisor. Therefore, real-time task will suffer from many cache misses, and the response of a virtual machine may be poor due to cache pollution, especially if real-time tasks compete for cache resources with heavy general-purpose tasks. To address this issue, we propose two cache optimization management policies. One is cache partitioning (CAP) - which isolates the cache statically for real-time tasks and general-purpose tasks using page coloring. Hence the predictability of cache usage is greatly increased. CAP reduces both cache pollution, and the amount of cache space that real-time tasks use, even if other cache is not used by general purpose tasks. We also study allocating different proportions of the cache for real-time tasks and find that, a proper cache ratio is optimal to guarantee response time. Due to this, another dynamic cache optimization policy is proposed, called page table prefetching (PTP). PTP employs a daemon-like thread to maintain the cache, so as to make the contents used by real-time tasks be in cache. PTP is implemented by utilizing a dedicated core assigned to general-purpose tasks to do prefectching of a page table into shared physical cache for a soft real-time task. As the page table is referenced frequently by tasks, PTP gives a considerable advantage because fewer TLB misses occur. The results from the Cyclitest and CPUspec2006 benchmarks show that both CAP and PTP can provide better response for real-time tasks. CAP has benefits due to its simplicity, but PTP has advantages in flexibility and high cache utilization due to its dynamic cached maintenance. In addition, the experimental results show that PTP improves real-time performance more but sacrifices some GP performance.

A hardware evaluation of cache partitioning to improve utilization and energy-efficiency while preserving responsiveness

Computing workloads often contain a mix of interactive, latency-sensitive foreground applications and recurring background computations. To guarantee responsiveness, interactive and batch applications are often run on disjoint sets of resources, but this incurs additional energy, power, and capital costs. In this paper, we evaluate the potential of hardware cache partitioning mechanisms and policies to improve efficiency by allowing background applications to run simultaneously with interactive foreground applications, while avoiding degradation in interactive responsiveness. We evaluate these tradeoffs using commercial x86 multicore hardware that supports cache partitioning, and find that real hardware measurements with full applications provide different observations than past simulation-based evaluations. Co-scheduling applications without LLC partitioning leads to a 10% energy improvement and average throughput improvement of 54% compared to running tasks separately, but can result in foreground performance degradation of up to 34% with an average of 6%. With optimal static LLC partitioning, the average energy improvement increases to 12% and the average throughput improvement to 60%, while the worst case slowdown is reduced noticeably to 7% with an average slowdown of only 2%. We also evaluate a practical low-overhead dynamic algorithm to control partition sizes, and are able to realize the potential performance guarantees of the optimal static approach, while increasing background throughput by an additional 19%.

Multi-level Cache Prediction and Partitioning Mechanism in CMP

Multi-level Cache Prediction and Partitioning Mechanism in CMP

Improving Cache Partitioning Algorithms for Pseudo-LRU Policies

Improving Cache Partitioning Algorithms for Pseudo-LRU Policies

A Capacity-Aware Thread Scheduling Method Combined with Cache Partitioning to Reduce Inter-Thread Cache Conflicts

A Capacity-Aware Thread Scheduling Method Combined with Cache Partitioning to Reduce Inter-Thread Cache Conflicts

An effective cache scheduling scheme for improving the performance in multi-threaded processors

Abstract Although in a multi-threaded processor, the processor may execute more than one process simultaneously to maximize the overall throughput of the system, the executing processes may compete with each other in using shared caches of the processor. This can seriously affect the average performance of the processes as the probability of cache hit for each process could be lowered. In this paper, we propose a new algorithm called the sharable cache partitioning algorithm (ShaParti), for scheduling the processor caches amongst co-running processes. In ShaParti, each executing process has its own cache and a priority scheme is designed for them to share the caches belonging to other executing processes. The performance goals of ShaParti are to improve the cache hit rates of the processes and at the same time the cache miss rates of other concurrent processes will not be lowered compared with the case in which each process has its own cache. Extensive experiments have been performed to illustrate the effectiveness of ShaParti in improving the performance in accessing shared processor caches.

Dynamic cache partitioning based on hot page migration

Static cache partitioning can reduce inter-application cache interference and improve the composite performance of a cache-polluted application and a cache-sensitive application when they run on cores that share the last level cache in the same multi-core processor. In a virtualized system, since different applications might run on different virtual machines (VMs) in different time, it is inapplicable to partition the cache statically in advance. This paper proposes a dynamic cache partitioning scheme that makes use of hot page detection and page migration to improve the composite performance of co-hosted virtual machines dynamically according to prior knowledge of cache-sensitive applications. Experimental results show that the overhead of our page migration scheme is low, while in most cases, the composite performance is an improvement over free composition.

Multi-dimension and Multi-level Associated Cache Partitioning Mechanism in CMP

Multi-dimension and Multi-level Associated Cache Partitioning Mechanism in CMP

Scalable and Efficient Fine-Grained Cache Partitioning with Vantage

The Vantage cache-partitioning technique enables configurability and quality-of-service guarantees in large-scale chip multiprocessors with shared caches. Caches can have hundreds of partitions with sizes specified at cache line granularity, while maintaining high associativity and strict isolation among partitions.

Writeback-aware partitioning and replacement for last-level caches in phase change main memory systems

Phase-Change Memory (PCM) has emerged as a promising low-power main memory candidate to replace DRAM. The main problems of PCM are that writes are much slower and more power hungry than reads, write bandwidth is much lower than read bandwidth, and limited write endurance. Adding an extra layer of cache, which is logically the last-level cache (LLC), can mitigate the drawbacks of PCM. However, writebacks from the LLC might (a) overwhelm the limited PCM write bandwidth and stall the application, (b) shorten lifetime, and (c) increase energy consumption. Cache partitioning and replacement schemes are important to achieve high throughput for multi-core systems. However, we noted that no existing partitioning and replacement policy takes into account the writeback information. This paper proposes two writeback-aware schemes to manage the LLC for PCM main memory systems. Writeback-aware Cache Partitioning (WCP) is a runtime mechanism that partitions a shared LLC among multiple applications. Unlike past partitioning schemes, our scheme considers the reduction in cache misses as well as writebacks. Write Queue Balancing (WQB) replacement policy manages the cache partition of each application intelligently so that the writebacks are distributed evenly among PCM write queues. In this way, applications rarely stall due to unbalanced PCM write traffic among write queues. Our evaluation shows that WCP and WQB result in, on average, 21% improvement in throughput, 49% reduction in PCM writes, and 14% reduction in energy over a state-of-the-art cache partitioning scheme.

The gradient-based cache partitioning algorithm

This paper addresses the problem of partitioning a cache between multiple concurrent threads and in the presence of hardware prefetching. Cache replacement designed to preserve temporal locality (e.g., LRU) will allocate cache resources proportional to the miss-rate of each competing thread irrespective of whether the cache space will be utilized [Qureshi and Patt 2006]. This is clearly suboptimal as applications vary dramatically in their use of recently accessed data. We address this problem by partitioning a shared cache such that a global goodness metric is optimized. This paper introduces the Gradient-based Cache Partitioning Algorithm (GPA), whose variants optimize either hitrate, total instructions per cycle (IPC) or a weighted IPC metric designed to enforce Quality of Service (QoS) [Iyer 2004]. In the context of QoS, GPA enables us to obtain the maximum throughput of low-priority threads, while ensuring high performance on high-priority threads. The GPA mechanism is robust, low-cost, integrates easily with existing cache designs and improves the throughput of an in-order 8-core system sharing an 8MB L3 cache by ∼14%.

An analytical model for transport layer caching in wireless sensor networks

Reliable transport protocols have traditionally been designed to perform end-to-end error control transparently to the intermediate nodes (e.g., TCP). However, the resource constraints featured by Wireless Sensor Network (WSN) require a different paradigm where intermediate nodes are able to cache packets, retransmitting them on-demand in order to avoid incurring on costly end-to-end retransmissions. This paper presents an analytical model of end-to-end delivery cost for WSN reliable transport with intermediate caching. The model calculates the cost as the total number of physical layer transmissions using a probabilistic formulation that has been validated through network simulation. Although the model is based on a specific transport protocol (DTSN), the addressed mechanisms are more generic, allowing it to be easily adapted to other WSN transport protocols that also feature intermediate caching. Numerical results confirm the improved efficiency introduced by a transport layer with intermediate caching in comparison with end-to-end approaches that are based exclusively on MAC layer reliability. Different cache partitioning policies were tested, and it is shown that cache partitioning policies should take into account the network conditions experienced by concurrent flows, namely the status of the radio links and the flow lengths.

Dynamically Adaptive I-Cache Partitioning for Energy-Efficient Embedded Multitasking

The ever increasing importance of battery-powered devices coupled with high performance requirements and shrinking process geometries have further exacerbated the problem of energy efficiency in modern embedded systems. The cache memories are a major contributor to the system power consumption, and as such have been a primary target for energy reduction techniques. Recent advances in configurable cache architectures have enabled an entirely new set of approaches for application-driven energy- and cost-efficient cache resource utilization. We propose a run-time and adaptive instruction cache partitioning methodology, which leverages configurable cache architectures to achieve an energy- and performance-conscious adaptive mapping of instruction cache resources to tasks in dynamic multi-task workloads sharing a processor core trough preemptive multitasking. Sizable leakage and dynamic power reductions are achieved with only a negligible and system-controlled performance impact. The methodology assumes no prior information regarding the dynamics and the structure of the workload. As the proposed dynamic cache partitioning alleviates the adverse effects of cache interference, performance is maintained very close to the baseline case, while achieving 50%-80% reductions in dynamic and leakage power for the on-chip instruction cache memory.

Vantage

Cache partitioning has a wide range of uses in CMPs, from guaranteeing quality of service and controlled sharing to security-related techniques. However, existing cache partitioning schemes (such as way-partitioning) are limited to coarse-grain allocations, can only support few partitions, and reduce cache associativity, hurting performance. Hence, these techniques can only be applied to CMPs with 2-4 cores, but fail to scale to tens of cores. We present Vantage, a novel cache partitioning technique that overcomes the limitations of existing schemes: caches can have tens of partitions with sizes specified at cache line granularity, while maintaining high associativity and strong isolation among partitions. Vantage leverages cache arrays with good hashing and associativity, which enable soft-pinning a large portion of cache lines. It enforces capacity allocations by controlling the replacement process. Unlike prior schemes, Vantage provides strict isolation guarantees by partitioning most (e.g. 90%) of the cache instead of all of it. Vantage is derived from analytical models, which allow us to provide strong guarantees and bounds on associativity and sizing independent of the number of partitions and their behaviors. It is simple to implement, requiring around 1.5% state overhead and simple changes to the cache controller. We evaluate Vantage using extensive simulations. On a 32-core system, using 350 multiprogrammed workloads and one partition per core, partitioning the last-level cache with conventional techniques degrades throughput for 71% of the workloads versus an unpartitioned cache (by 7% average, 25% maximum degradation), even when using 64-way caches. In contrast, Vantage improves throughput for 98% of the workloads, by 8% on average (up to 20%), using a 4-way cache .

Joint task assignment and cache partitioning with cache locking for WCET minimization on MPSoC

Cache locking technique is often utilized to guarantee a tighter prediction of Worst-Case Execution Time (WCET) which is one of the most important performance metrics for embedded systems. However, in Multi-Processor Systems-on-Chip (MPSoC) systems with multi-tasks, Level 2 (L2) cache is often shared among different tasks and cores, which leads to extended unpredictability of cache. Task assignment has inherent relevancy for cache behavior, while cache behavior also affects the efficiency of task assignment. Task assignment and cache behavior have dramatic influences on the overall WCET of MPSoC. This paper proposes joint task assignment and cache partitioning techniques to minimize the overall WCET for MPSoC systems. Cache locking is applied to each task to guarantee a precise WCET. We prove that the joint problem is NP-hard and propose several efficient algorithms. Experimental results show that the proposed algorithms can consistently reduce the overall WCET compared to previous techniques.

CaPaS: an optimal security-aware cache replacement algorithm for cluster storage systems

In this paper, we introduce a novel dynamic cache management approach, i.e., a cache partitioning system (CaPaS), to optimise security levels in contemporary cluster storage systems for data-intensive applications with acceptable disk response times. CaPaS utilises an adaptive security control mechanism to adjust for varying workload conditions initiated by client-issued disk requests as well as security requirements and contains a cache partitioning scheme, a response-time estimator, and an adaptive security quality controller. The CaPaS algorithm solves a non-linear optimisation problem to increase the quality of security of disk requests in cluster storage systems while ensuring the disk requests to be completed within their desired response times. The efficiency of the proposed algorithm has been tested with the newly defined performance indicators on a cluster storage system including CaPaS, eight cryptographic and seven integrity services. Numerical simulation results demonstrate that the CaPaS algorithm improves the overall system performance enormously over two traditional baseline cache management algorithms.

Adaptive Block Pinning Based: Dynamic Cache Partitioning for Multi-core Architectures

Adaptive Block Pinning Based: Dynamic Cache Partitioning for Multi-core Architectures

Elastic cooperative caching

Next generation tiled microarchitectures are going to be limited by off-chip misses and by on-chip network usage. Furthermore, these platforms will run an heterogeneous mix of applications with very different memory needs, leading to significant optimization opportunities. Existing adaptive memory hierarchies use either centralized structures that limit the scalability or software based resource allocation that increases programming complexity. We propose Elastic Cooperative Caching, a dynamic and scalable memory hierarchy that adapts automatically and autonomously to application behavior for each node. Our configuration uses elastic shared/private caches with fully autonomous and distributed repartitioning units for better scalability. Furthermore, we have extended our elastic configuration with an Adaptive Spilling mechanism to use the shared cache space only when it can produce a performance improvement. Elastic caches allow both the creation of big local private caches for threads with high reuse of private data and the creation of big shared spaces from unused caches. Local data allocation in private regions allows to reduce network usage and efficient cache partitioning allows to reduce off-chip misses. The proposed scheme outperforms previous proposals by a minimum of 12% (on average across the benchmarks) and reduces the number of offchip misses by 16%. Plus, the dynamic and autonomous management of cache resources avoids the reallocation of cache blocks without reuse which results in an increase in energy efficiency of 24%.

Dynamic Partition of Shared Cache for Multi-Thread Application in Multi-Core System

In a chip-multiprocessor with a shared cache structure , the competing accesses from different applications degrade the system performance.The accesses degrade the performance and result in non-predicting executing time. Cache partitioning techniques can exclusively partition the shared cache among multiple competing applications. In this paper, the authors design the framework of Process priority-based Multithread Cache Partitioning(PP-MCP),a dynamic shared cache partitioning mechanism to improve the performance of multi-threaded multi-programmed workloads. The framework includes a miss rate monitor , called Application-oriented Miss Rate Monitor (AMRM) , which dynamically collect s miss rate information of multiple multi-threaded applications on different cache partitions , and process priority-based weighted cache partitioning algorithm ,which extends traditional miss rate oriented cache partition algorithms.The algorithm allocates Cache in sequence of the value of the process priority and it ensures that the highest priority process will get enough Cache space; and the applications with more threads tend to get more shared cache in order to improve t he overall system performance. Experiments show that PP-MCP has better IPC throughput and weighted speedup. Specifically , for multi-threaded multi-programmed scientific computing workloads , PP-MCP-1 improves throughput by up to 20% and on average 10 % over PP-MCP-0.

The Review of Cache Partitioning in Multi-Core Processor

With the development of microelectronics technology, Chip Multi-Processor (CMP) or multi-core design has become a mainstream choice for major microprocessor vendors. But in a chip-multiprocessor with a shared cache structure , the competing accesses from different applications degrade the system performance , resulting in non-optimal performance and non-predicting executing time. Cache partitioning techniques can exclusively partition the shared cache among multiple competing applications. In this paper, we first introduce the problems caused by Cache pollution in multicore processor structure; then present the different methods of Cache partitioning in multicore processor structure¬ －－categorizing them based on the different metrics. And finally, we discuss some possible directions for future research in the area.