Cache Performance Research Articles

Cache Memory: An Analysis on Replacement Algorithms and Optimization Techniques

Caching strategies can improve the overall performance of a system by allowing the fast processor and slow memory to at a same pace. One important factor in caching is the replacement policy. Advancement in technology results in evolution of a huge number of techniques and algorithms implemented to improve cache performance. In this paper, analysis is done on different cache optimization techniques as well as replacement algorithms. Furthermore this paper presents a comprehensive statistical comparison of cache optimization techniques.To the best of our knowledge there is no numerical measure which can tell us the rating of specific cache optimization technique. We tried to come up with such a numerical figure. By statistical comparison we find out which technique is more consistent among all others. For said purpose we calculated mean and CV (Coefficient of Variation). CV tells us about which technique is more consistent. Comparative analysis of different techniques shows that victim cache has more consistent technique among all.

캐시 이미지의 동적 관리 방법을 이용한 명령어 캐시 성능 개선

프로세스 구동시마다 캐시 이미지를 메모리로부터 버스트 로딩하여 초기 캐시 실패를 줄이는 방법은 프로그램의 시작으로부터 초기화 부분의 지연을 줄이고 에너지 소모를 줄이는 데 효과적이다. 하지만 로딩에 사용하기 위한 적절한 캐시 이미지는 컴파일러와 인스톨러 등 소프트웨어적인 접근 방법을 이용하여 적절한 캐시 이미지를 생성하는 과정이 필요하며, 동적인 수행 특성을 보이는 프로세스의 경우 비효율적이다. 본 논문은 이러한 손실에 주목하여, 하드웨어를 부가하여 캐시 이미지를 동적으로 생성하고 관리하는 방법을 제안하고자 한다. 시뮬레이션 결과에 따르면 제안한 방법을 사용할 경우 프로그램의 캐시 필요량에 따른 적절한 이미지 크기를 유지할 수 있어 기존의 캐시 이미지 로딩 기법을 더욱 효율적으로 개선할 수 있었다.

Efficient cache resource aggregation using adaptive multi-level exclusive caching policies

Multi-level buffer cache hierarchies are now commonly seen in most client/server cluster configurations, especially in today’s big data application deployment. However, multi-level caching policies deployed so far typically use independent cache replacement algorithms in each level, which has two major drawbacks: (1) File blocks may be redundantly cached on multiple levels, reducing the actual aggregate cache usable size; (2) Less accurate replacement decisions at lower level caches due to weakened locality. Inefficient cache resource usage may result in noticeable performance degradation for big data applications.To address these problems, we propose new adaptive multi-level exclusive caching policies that can dynamically adjust replacement and placement decisions in response to changing access patterns. (1) First, to capture locality information in multi-level cache hierarchies, we propose a Reuse Distance based Adaptive Replacement Caching (ReDARC) algorithm that adopts reuse distance as the means of locality measure and adaptively balances between the Small Reuse Distance (SRD) set and Large Reuse Distance (LRD) set. (2) Second, to achieve exclusive caching and make global caching decisions, we propose an Adaptive Level-Aware Caching Algorithm (ALACA) that works collaboratively with ReDARC. The ALACA algorithm uses an adaptive probabilistic PUSH technique that allows lower caches to push blocks to higher caches and appropriately decide blocks’ caching locations with the ReDARC algorithm. In this way, we achieve multi-level exclusive caching with significant cache performance improvement. Our trace-driven simulation experiments show that the policies we proposed achieve a reduction of the client average response time of 8 percent to 56 percent over other multi-level cache schemes.

A performance study of the time-varying cache behavior: a study on APEX, Mantevo, NAS, and PARSEC

Cache has long been used to minimize the latency of main memory accesses by storing frequently used data near the processor. Processor performance depends on the underlying cache performance. Therefore, significant research has been done to identify the most crucial metrics of cache performance. Although the majority of research focuses on measuring cache hit rates and data movement as the primary cache performance metrics, cache utilization is significantly important. We investigate the application’s locality using cache utilization metrics. Furthermore, we present cache utilization and traditional cache performance metrics as the program progresses providing detailed insights into the dynamic application behavior on parallel applications from four benchmark suites running on multiple cores. We explore cache utilization for APEX, Mantevo, NAS, and PARSEC, mostly scientific benchmark suites. Our results indicate that 40% of the data bytes in a cache line are accessed at least once before line eviction. Also, on average a byte is accessed two times before the cache line is evicted for these applications. Moreover, we present runtime cache utilization, as well as, conventional performance metrics that illustrate a holistic understanding of cache behavior. To facilitate this research, we build a memory simulator incorporated into the Structural Simulation Toolkit (Rodrigues et al. in SIGMETRICS Perform Eval Rev 38(4):37–42, 2011). Our results suggest that variable cache line size can result in better performance and can also conserve power.

A Centrality-measures based Caching Scheme for Content-centric Networking (CCN)

Content-centric networking (CCN) is gradually becoming an alternative approach to the conventional Internet architecture through the distribution of enlightening information (named as content) on the Internet. It is evaluated that the better performance can be achieved by caching is done on a subset of content routers instead of all the routers in the content delivery path. The subset of a content router must be selected in a manner such that maximum cache performance can be achieved. Motivated by this, we propose a Centrality-measures based algorithm (CMBA) for selection of an appropriate content router for caching of the contents. The Centrality-measures are based on the question: ”Who are the most important or central content router in the network for the caching of contents?”. We found that our novel CMBA could improve content cache performance along the content delivery path by using only a subset of available content routers. Our results recommend that our proposed work consistently achieves better caching gain across the multiple network topologies.

Pannier

Classic caching algorithms leverage recency, access count, and/or other properties of cached blocks at per-block granularity. However, for media such as flash which have performance and wear penalties for small overwrites, implementing cache policies at a larger granularity is beneficial. Recent research has focused on buffering small blocks and writing in large granularities, sometimes called containers, but it has not explored the ramifications and best strategies for caching compound blocks consisting of logically distinct, but physically co-located, blocks. Containers may have highly diverse blocks, with mixtures of frequently accessed, infrequently accessed, and invalidated blocks. We propose and evaluate Pannier, a flash cache layer that provides high performance while extending flash lifespan. Pannier uses three main techniques: (1) leveraging block access counts to manage cache containers, (2) incorporating block liveness as a property to improve flash cache space efficiency, and (3) designing a multi-step feedback controller to ensure a flash cache reaches its desired lifespan while maintaining performance. Our evaluation shows that Pannier improves flash cache performance and extends lifespan beyond previous per-block and container-aware caching policies. More fundamentally, our investigation highlights the importance of creating new policies for caching compound blocks in flash.

Fog Caching and a Trace-Based Analysis of its Offload Effect

Many years of research on Content Delivery Networks (CDNs) offers a number of effective methods for caching of content replicas or forwarding requests. However, recently CDNs have aggressively started migrating to clouds. Clouds present a new kind of distribution environment as each location can support multiple caching options varying in the level of persistence of stored content. A subclass of clouds located at network edge is referred to as fog clouds. Fog clouds help by allowing CDNs to offload popular content to network edge, closer to end users. However, due to the fact that fog clouds are extremely heterogeneous and vary wildly in network and caching performance, traditional caching technology is no longer applicable. This paper proposes a multi-level caching technology specific to fog clouds. To deal with the heterogeneity problem and, at the same time, avoid centralized control, this paper proposes a function that allows CDN services to discover local caching facilities dynamically, at runtime. Using a combination of synthetic models and real measurement dataset, this paper analyzes efficiency of offload both at the local level of individual fog locations and at the global level of the entire CDN infrastructure. Local analysis shows that the new method can reduce inter-cloud traffic by between 16 and 18 times while retaining less than 30% of total content in a local cache. Global analysis further shows that, based on existing measurement datasets, centralized optimization is preferred to distributed coordination among services.

Cache simulation for measuring cache performance suitable for sound rendering

Cache simulation for measuring cache performance suitable for sound rendering

Analysis of time-dependent dielectric breakdown induced aging of SRAM cache with different configurations

Time dependent dielectric breakdown degrades the reliability of SRAM cache. A novel methodology to estimate SRAM cache reliability and performance is presented. The performance and reliability characteristics are obtained from activity extraction and Monte Carlo simulations, considering device dimensions, process variations, the stress probability, and the thermal distribution. Based on the reliability-performance estimation methodology, caches with various settings on associativity, cache line size, and cache size are analysed and compared. Experiments show that there exists a contradiction between performance and reliability for different cache configurations. Understanding the variation of performance and reliability can provide SRAM designers with insight on reliability-performance trade-offs for cache system design.

A Study on Performance and Power Efficiency of Dense Non-Volatile Caches in Multi-Core Systems

This paper presents a novel cache design based on Multi-Level Cell Spin-Transfer Torque RAM (MLC STT-RAM).Our design exploits the asymmetric nature of the MLC STT-RAM to build cache lines featuring heterogeneous performances, that is, half of the cache lines are read-friendly,while the other half are write-friendly--this asymmetry in read/write latencies are then used by a migration policy in order to overcome the high latency of the baseline MLC cache. Furthermore, in order to enhance the device lifetime, we propose to dynamically deactivate ways of a set in underutilized sets to convert MLC to Single-Level Cell (SLC)mode.Our experiments show that our design gives an average improvement of 12% in system performance and 26% in last-level cache(L3) access energy for various workloads.

A cache-friendly sampling strategy for texture-based volume rendering on GPU

The texture-based volume rendering is a memory-intensive algorithm. Its performance relies heavily on the performance of the texture cache. However, most existing texture-based volume rendering methods blindly map computational resources to texture memory and result in incoherent memory access patterns, causing low cache hit rates in certain cases. The distance between samples taken by threads of an atomic scheduling unit (e.g. a warp of 32 threads in CUDA) of the GPU is a crucial factor that affects the texture cache performance. Based on this fact, we present a new sampling strategy, called Warp Marching, for the ray-casting algorithm of texture-based volume rendering. The effects of different sample organizations and different thread-pixel mappings in the ray-casting algorithm are thoroughly analyzed. Also, a pipeline manner color blending approach is introduced and the power of warp-level GPU operations is leveraged to improve the efficiency of parallel executions on the GPU. In addition, the rendering performance of the Warp Marching is view-independent, and it outperforms existing empty space skipping techniques in scenarios that need to render large dynamic volumes in a low resolution image. Through a series of micro-benchmarking and real-life data experiments, we rigorously analyze our sampling strategies and demonstrate significant performance enhancements over existing sampling methods.

Feedback cache mechanism for dynamically reconfigurable VLIW processors

Very Long Instruction Word (VLIW) architectures are commonly used in application-specific domains due to their parallelism and low-power characteristics. Recently, parameterization of such architectures allows for runtime adaptation of the issue-width to match the inherent Instruction Level Parallelism (ILP) of an application. One implementation of such an approach is that the event of the issue-width switching dynamically triggers the reconfiguration of the data cache at runtime. In this paper, the relationship between cache resizing and issue-width is well investigated. We have observed that the requirement of the cache does not always correlate with the issuewidth of the VLIW processor. To further coordinate the cache resizing with the changing issue-width, we present a novel feedback mechanism to block the low yields of cache resizing when the issue-width changes. In this manner, our feedback cache mechanism has a coordinated effort with the issue-width changes, which leads to a noticeable improvement of the cache performance. The experiments show that there is 10% energy savings as well as a 2.3% cache misses decline on average achieved, compared with the cache without the feedback mechanism. Therefore, the feedback mechanism is proven to have the capability to ensure more benefits are achieved from the dynamic and frequent reconfiguration.

Understanding Performance of Edge Content Caching for Mobile Video Streaming

Today’s Internet has witnessed an increase in the popularity of mobile video streaming, which is expected to exceed 3/4 of the global mobile data traffic by 2019. To satisfy the considerable amount of mobile video requests, video service providers have been pushing their content delivery infrastructure to edge networks—from regional content delivery network (CDN) servers to peer CDN servers (e.g., smartrouters in users’ homes)—to cache content and serve users with storage and network resources nearby. Among the edge network content caching paradigms, Wi-Fi access point caching and cellular base station caching have become two mainstream solutions. Thus, understanding the effectiveness and performance of these solutions for large-scale mobile video delivery is important. However, the characteristics and request patterns of mobile video streaming are unclear in practical wireless network. In this paper, we use real-world data sets containing 50 million trace items of nearly 2 million users viewing more than 0.3 million unique videos using mobile devices in a metropolis in China over two weeks, not only to understand the request patterns and user behaviors in mobile video streaming, but also to evaluate the effectiveness of Wi-Fi and cellular-based edge content caching solutions. To understand the performance of edge content caching for mobile video streaming, we first present temporal and spatial video request patterns, and we analyze their impacts on caching performance using frequency-domain and entropy analysis approaches. We then study the behaviors of mobile video users, including their mobility and geographical migration behaviors, which determine the request patterns. Using trace-driven experiments, we compare strategies for edge content caching, including least recently used (LRU) and least frequently used (LFU), in terms of supporting mobile video requests. We reveal that content, location, and mobility factors all affect edge content caching performance. Moreover, we design an efficient caching strategy based on the measurement insights and experimentally evaluate its performance. The results show that our design significantly improves the cache hit rate by up to 30% compared with LRU/LFU.

Custom Multicache Architectures for Heap Manipulating Programs

Memory-intensive implementations often require access to an external, off-chip memory which can substantially slow down an field-programmable gate array accelerator due to memory bandwidth limitations. Buffering frequently reused data on chip is a common approach to address this problem and the optimization of the cache architecture introduces yet another complex design space. This paper presents a high-level synthesis (HLS) design aid that automatically generates parallel multicache systems which are tailored to the specific requirements of the application. Our program analysis identifies nonoverlapping memory regions, supported by private caches, and regions which are shared by parallel units after parallelization, which are supported by coherent caches and synchronization primitives. It also decides whether the parallelization is legal with respect to data dependencies. The novelty of this paper is the focus on programs using dynamically allocated, pointer-based data structures which, while common in software engineering, remain difficult to analyze and are beyond the scope of the overwhelming majority of HLS techniques to date. Second, we devise a high-level cache performance estimation to find a heterogeneous configuration of cache sizes that maximizes the performance of the multicache system subject to an on-chip memory resource constraint. We demonstrate our technique with three case studies of applications using dynamic data structures and use Xilinx Vivado HLS as an exemplary HLS tool. We show up to $\boldsymbol {15}\boldsymbol {\times }$ speed-up after parallelization of the HLS implementations and the insertion of the application-specific distributed hybrid multicache architecture.

Realisation of Cache Optimisation using New Technique

An impressive design of cache memory is an important view in computer architecture to boost or increase the system performance. In this task we study the response of reducing the cache performance to the cache address on the performance experimentally. Since cache miss penalty has direct contact on the system’s performance. To overtake from this, we proposed a new model in which we use revisited second chance FIFO (RS FIFO), which uses partial comparator and use multiple search to increase cache hit rate, by reducing cache miss rate. The Above proposed technique gives maximum hit rate when using revisited SFIFO rather than FIFO. Keywords: cache mapping technique; hit rate; miss rate; cache optimisation; cache replacement policies.

Design and Implementation of Task Reprocessing on Medium-large Multi-core Architecture

EDF-Based real-time scheduling approach is one of the efficient way of scheduling the recurrent real-time tasks in both soft real-time and hard real-time systems. EDF algorithms entile heavy overhead due to lower scheduling and huge migration. In this paper, an exploratory procedure has been presented for reprocessing of neglected task sets and also examine, whether an improved heuristic set attributed as Enhanced shared CAche Performance (ENCAP) can improve the shared cache performance which processes the task set based on Earliest-Deadline-First (EDF). An Object-Oriented real-time code reusable scheduling components which is designed from the scratch referred as LITMUSRT has been used to test the efficiency and average-case accomplishments where per task utilizations are shortened and overall utilization are not restricted. We also discuss the implementation and experimental evaluation of Task-Repo procedure for co-scheduling task sets, under the utilization of 50-60% (per-task) on small to medium-large multicore LINUX SYSTEM.

Locality-Aware CTA Clustering for Modern GPUs

Cache is designed to exploit locality; however, the role of on-chip L1 data caches on modern GPUs is often awkward. The locality among global memory requests from different SMs (Streaming Multiprocessors) is predominantly harvested by the commonly-shared L2 with long access latency; while the in-core locality, which is crucial for performance delivery, is handled explicitly by user-controlled scratchpad memory. In this work, we disclose another type of data locality that has been long ignored but with performance boosting potential --- the inter-CTA locality. Exploiting such locality is rather challenging due to unclear hardware feasibility, unknown and inaccessible underlying CTA scheduler, and small in-core cache capacity. To address these issues, we first conduct a thorough empirical exploration on various modern GPUs and demonstrate that inter-CTA locality can be harvested, both spatially and temporally, on L1 or L1/Tex unified cache. Through further quantification process, we prove the significance and commonality of such locality among GPU applications, and discuss whether such reuse is exploitable. By leveraging these insights, we propose the concept of CTA-Clustering and its associated software-based techniques to reshape the default CTA scheduling in order to group the CTAs with potential reuse together on the same SM. Our techniques require no hardware modification and can be directly deployed on existing GPUs. In addition, we incorporate these techniques into an integrated framework for automatic inter-CTA locality optimization. We evaluate our techniques using a wide range of popular GPU applications on all modern generations of NVIDIA GPU architectures. The results show that our proposed techniques significantly improve cache performance through reducing L2 cache transactions by 55%, 65%, 29%, 28% on average for Fermi, Kepler, Maxwell and Pascal, respectively, leading to an average of 1.46x, 1.48x, 1.45x, 1.41x (up to 3.8x, 3.6x, 3.1x, 3.3x) performance speedups for applications with algorithm-related inter-CTA reuse.

Locality-Aware CTA Clustering for Modern GPUs

Cache is designed to exploit locality; however, the role of on-chip L1 data caches on modern GPUs is often awkward. The locality among global memory requests from different SMs (Streaming Multiprocessors) is predominantly harvested by the commonly-shared L2 with long access latency; while the in-core locality, which is crucial for performance delivery, is handled explicitly by user-controlled scratchpad memory. In this work, we disclose another type of data locality that has been long ignored but with performance boosting potential --- the inter-CTA locality. Exploiting such locality is rather challenging due to unclear hardware feasibility, unknown and inaccessible underlying CTA scheduler, and small in-core cache capacity. To address these issues, we first conduct a thorough empirical exploration on various modern GPUs and demonstrate that inter-CTA locality can be harvested, both spatially and temporally, on L1 or L1/Tex unified cache. Through further quantification process, we prove the significance and commonality of such locality among GPU applications, and discuss whether such reuse is exploitable. By leveraging these insights, we propose the concept of CTA-Clustering and its associated software-based techniques to reshape the default CTA scheduling in order to group the CTAs with potential reuse together on the same SM. Our techniques require no hardware modification and can be directly deployed on existing GPUs. In addition, we incorporate these techniques into an integrated framework for automatic inter-CTA locality optimization. We evaluate our techniques using a wide range of popular GPU applications on all modern generations of NVIDIA GPU architectures. The results show that our proposed techniques significantly improve cache performance through reducing L2 cache transactions by 55%, 65%, 29%, 28% on average for Fermi, Kepler, Maxwell and Pascal, respectively, leading to an average of 1.46x, 1.48x, 1.45x, 1.41x (up to 3.8x, 3.6x, 3.1x, 3.3x) performance speedups for applications with algorithm-related inter-CTA reuse.

Optimal Symbiosis and Fair Scheduling in Shared Cache

On multi-core processors, applications are run sharing the cache. This paper presents optimization theory to co-locate applications to minimize cache interference and maximize performance. The theory precisely specifies MRC-based composition, optimization, and correctness conditions. The paper also presents a new technique called footprint symbiosis to obtain the best shared cache performance under fair CPU allocation as well as a new sampling technique which reduces the cost of locality analysis. When sampling and optimization are combined, the paper shows that it takes less than 0.1 second analysis per program to obtain a co-run that is within 1.5 percent of the best possible performance. In an exhaustive evaluation with 12,870 tests, the best prior work improves co-run performance by 56 percent on average. The new optimization improves it by another 29 percent. Without single co-run test, footprint symbiosis is able to choose co-run choices that are just 8 percent slower than the best co-run solutions found with exhaustive testing.

Energy-aware cache hierarchy assessment targeting HEVC encoder execution

This article presents a framework for assessing the behavior and energy impact of cache hierarchies when encoding HEVC on general-purpose processors. The memory energy estimation framework estimates energy consumption of cache hierarchies based on mathematical models combined with memory access profiling tools. The energy analysis of several cache hierarchies targeting HEVC encoders with different input parameters is also carried out. This article provides relevant information on the energy consumption of HEVC encoders by taking into account the different tradeoffs between energy efficiency, coding efficiency, and other important cache memory design parameters, such as miss rates and access latency. The first analysis explores cache performance for different specifications, such as capacity, line size. Results show that most of the energy is spent on reading operations (almost 73% on the first level cache), indicating that HEVC encoders could benefit from memory technologies with low reading energy costs. This analysis also pointed that increasing the capacity affects more the energy performance of the first level cache, which represents 34.78% (on average) more energy consumption than the last level cache. Based on this investigation, we report the most suited cache specifications for HEVC encoders for each video resolution. The second analysis discusses the impact of HEVC input parameters in cache performance, demonstrating that it is possible to save up to 30% of energy with a small increase of 2% in BD-BR. A comparative analysis between HM (HEVC model) and x265 (H.265 video codec) HEVC software models is presented, demonstrating that x265 is faster (speedup to 648x), and more cache efficient providing less memory energy (31.38% on average) compared to the HM implementation. The results obtained with the proposed framework indicate that the management of video encoding parameters combined with application-tuned cache specifications has a high potential to reduce energy consumption of video coding systems while keeping video quality.