Cache Performance Research Articles

LPCE-Based Replacement Scheme for Enhancing Caching Performance in Named Data Networking

The substantial surge in users has adversely impacted the performance of the present IP-based Internet. Named data networking (NDN) emerges as a future alternative, given its distributed content caching system, where data can be cached in multiple routers and retrieved from the closest one instead of the original producer, enhancing content availability, reducing latency, and minimizing data loss. This paper introduces the less popular content eviction (LPCE) policy, a novel cache replacement scheme designed to enhance the caching performance of the conventional LFU (least frequently used) policy in NDN routers, thereby improving overall network efficiency. The proposed method subsumes LFU and FIFO (first in first out) policies and employs an additional list controlled by the LRU (least recently used) policy. Utilizing the ccnSim simulator, we conduct a comparison of LPCE’s performance with that of the LFU technique and other competing caching techniques, considering variations in several simulation parameters. Experimental results reveal that the proposed LPCE algorithm excels over others across a majority of performance metrics, such as cache hit ratio, content delivery delay, upstream hop count, network traffic, and producers’ load. Besides, the findings indicate that LPCE outperforms LFU, with an increase in cache hit ratio ranging from 1.32% to 5.75%.

Mixed Micro/Macro Cache for Device-to-Device Caching Systems in Multi-Operator Environments.

In a device-to-device (D2D) caching system that utilizes a device's available storage space as a content cache, a device called a helper can provide content requested by neighboring devices, thereby reducing the burden on the wireless network. To enhance the efficiency of a limited-size cache, one can consider not only macro caching, which is content-based caching based on content popularity, but also micro caching, which is chunk-based sequential prefetching and stores content chunks slightly behind the one that a nearby device is currently viewing. If the content in a cache can be updated intermittently even during peak hours, the helper can improve the hit ratio by performing micro caching, which stores chunks that are expected to be requested by nearby devices in the near future. In this paper, we discuss the performance and effectiveness of micro D2D caching when there are multiple operators, the helpers can communicate with the devices of other operators, and the operators are under a low load independently of each other. We also discuss the ratio of micro caching in the cache area when the cache space is divided into macro and micro cache areas. Good performance can be achieved by using micro D2D caching in conjunction with macro D2D caching when macro caching alone does not provide sufficient performance, when users are likely to continue viewing the content they are currently viewing, when the content update cycle for the cache is short and a sufficient number of chunks can be updated for micro caching, and when there are multiple operators in the region.

FASA-DRAM: Reducing DRAM Latency with Destructive Activation and Delayed Restoration

DRAM memory is a performance bottleneck for many applications, due to its high access latency. Previous work has mainly focused on data locality, introducing small but fast regions to cache frequently accessed data, thereby reducing the average latency. However, these locality-based designs have three challenges in modern multi-core systems: (1) inter-application interference leads to random memory access traffic, (2) fairness issues prevent the memory controller from over-prioritizing data locality, and (3) write-intensive applications have much lower locality and evict substantial dirty entries. With frequent data movement between the fast in-DRAM cache and slow regular arrays, the overhead induced by moving data may even offset the performance and energy benefits of in-DRAM caching. In this article, we decouple the data movement process into two distinct phases. The first phase is Load-Reduced Destructive Activation (LRDA), which destructively promotes data into the in-DRAM cache. The second phase is Delayed Cycle-Stealing Restoration (DCSR), which restores the original data when the DRAM bank is idle. LRDA decouples the most time-consuming restoration phase from activation, and DCSR hides the restoration latency through prevalent bank-level parallelism. We propose FASA-DRAM, incorporating destructive activation and delayed restoration techniques to enable both in-DRAM caching and proactive latency-hiding mechanisms. Our evaluation shows that FASA-DRAM improves the average performance by 19.9% and reduces average DRAM energy consumption by 18.1% over DDR4 DRAM for four-core workloads, with less than 3.4% extra area overhead. Furthermore, FASA-DRAM outperforms state-of-the-art designs in both performance and energy efficiency.

Learning from the present for the future: The Jülich LOFAR Long-term Archive

The Forschungszentrum Jülich has been hosting the German part of the LOFAR archive since 2013. It is Germany’s most extensive radio astronomy archive, currently storing nearly 22 petabytes (PB) of data. Future radio telescopes are expected to require a dramatic increase in long-term data storage. Here, we take stock of the current data management of the Jülich LOFAR Data Archive, describe the ingestion, the storage system, the export to the long-term archive, and the request chain. We analysed the data availability over the last 10 years and searched for the underlying data access pattern and the energy consumption of the process. We determine hardware-related limiting factors, such as network bandwidth and cache pool availability and performance, and software aspects, e.g. workflow adjustment and parameter tuning, as the main data storage bottlenecks. By contrast, the challenge in providing the data from the archive for the users lies in retrieving the data from the tape archive and staging them. Building on this analysis, we suggest how to avoid/mitigate these problems in the future and define the requirements for future even more extensive long-term data archives.

Enhancing Lifetime and Performance of MLC NVM Caches Using Embedded Trace Buffers

Large volumes of on-chip and off-chip memory are required by contemporary applications. Emerging non-volatile memory technologies including STT-RAM, PCM, and ReRAM are becoming popular for on-chip and off-chip memories as a result of their desirable properties. Compared to traditional memory technologies such as SRAM and DRAM, they have minimal leakage current and high packing density. Non Volatile Memories (NVM), however, have a low write endurance, a high write latency, and high write energy. Non-volatile Single Level Cell (SLC) memories can store a single bit of data in each memory cell, whereas Multi Level Cells (MLC) can store two or more bits in each memory cell. Although MLC NVMs have substantially higher packing density than SLCs, their lifetime and access speed are key concerns. For a given cache size, MLC caches consume 1.84× less space and 2.62× less leakage power than SLC caches. We propose Trace buffer Assisted Non-volatile Memory Cache (TANC), an approach that increases the lifespan and performance of MLC-based last-level caches using the underutilized Embedded Trace Buffers (ETB). TANC improves the lifetime of MLC LLCs up to 4.36× and decreases average memory access time by 4% compared to SLC NVM LLCs and by 6.41× and 11%, respectively, compared to baseline MLC LLCs.

SLAP: Segmented Reuse-Time-Label Based Admission Policy for Content Delivery Network Caching

‘‘Learned” admission policies have shown promise in improving Content Delivery Network (CDN) cache performance and lowering operational costs. Unfortunately, existing learned policies are optimized with a few fixed cache sizes while in reality, cache sizes often vary over time in an unpredictable manner. As a result, existing solutions cannot provide consistent benefits in production settings. We present SLAP , a learned CDN cache admission approach based on segmented object reuse time prediction. SLAP predicts an object’s reuse time range using the Long-Short-Term-Memory model and admits objects that will be reused (before eviction) given the current cache size. SLAP decouples model training from cache size, allowing it to adapt to arbitrary sizes. The key to our solution is a novel segmented labeling scheme that makes SLAP without requiring precise prediction on object reuse time. To further make SLAP a practical and efficient solution, we propose aggressive reusing of computation and training on sampled traces to optimize model training, and a specialized predictor architecture that overlaps prediction computation with miss object fetching to optimize model inference. Our experiments using production CDN traces show that SLAP achieves significantly lower write traffic (38%-59%), longer SSDs lifetime (104%-178%), a consistently higher hit rate (3.2%-11.7%), and requires no effort to adapt to changing cache sizes, outperforming existing policies.

Optimized Two-Tier Caching with Hybrid Millimeter-Wave and Microwave Communications for 6G Networks

Data caching is a promising technique to alleviate the data traffic burden from the backhaul and minimize data access delay. However, the cache capacity constraint poses a significant challenge to obtaining content through the cache resource that degrades the caching performance. In this paper, we propose a novel two-tier caching mechanism for data caching on mobile user equipment (UE) and the small base station (SBS) level in ultra-dense 6G heterogeneous networks for reducing data access failure via cache resources. The two-tier caching enables users to retrieve their desired content from cache resources through device-to-device (D2D) communications from neighboring users or the serving SBS. The cache-enabled UE exploits millimeter-wave (mmWave)-based D2D communications, utilizing line-of-sight (LoS) links for high-speed data transmission to content-demanding mobile UE within a limited connection time. In the event of D2D communication failures, a dual-mode hybrid system, combining mmWave and microwave μWave technologies, is utilized to ensure effective data transmission between the SBS and UE to fulfill users’ data demands. In the proposed framework. the data transmission speed is optimized through mmWave signals in line-of-sight (LoS) conditions. In non-LoS scenarios, the system switches to μWave mode for obstacle-penetrating signal transmission. Subsequently, we propose a reinforcement learning (RL) approach to optimize cache decisions through the approximation of the Q action-value function. The proposed technique undergoes iterative learning, adapting to dynamic network conditions to enhance the content placement policy and minimize delay. Extensive simulations demonstrate the efficiency of our proposed approach in significantly reducing network delay compared with benchmark schemes.

FASA-DRAM: Reducing DRAM Latency with Destructive Activation and Delayed Restoration

DRAM memory is a performance bottleneck for many applications, due to its high access latency. Previous work has mainly focused on data locality, introducing small-but-fast regions to cache frequently accessed data, thereby reducing the average latency. However, these locality-based designs have three challenges in modern multi-core systems: 1) Inter-application interference leads to random memory access traffic. 2) Fairness issues prevent the memory controller from over-prioritizing data locality. 3) Write-intensive applications have much lower locality and evict substantial dirty entries. With frequent data movement between the fast in-DRAM cache and slow regular arrays, the overhead induced by moving data may even offset the performance and energy benefits of in-DRAM caching. In this paper, we decouple the data movement process into two distinct phases. The first phase is Load-Reduced Destructive Activation (LRDA), which destructively promotes data into the in-DRAM cache. The second phase is Delayed Cycle-Stealing Restoration (DCSR), which restores the original data when DRAM bank is idle. LRDA decouples the most time-consuming restoration phase from activation, and DCSR hides the restoration latency through prevalent bank-level parallelism. We propose FASA-DRAM incorporating destructive activation and delayed restoration techniques to enable both in-DRAM caching and proactive latency-hiding mechanisms. Our evaluation shows that FASA-DRAM improves the average performance by 19.9% and reduces average DRAM energy consumption by 18.1% over DDR4 DRAM for four-core workloads, with less than 3.4% extra area overhead. Furthermore, FASA-DRAM outperforms state-of-the-art designs in both performance and energy efficiency.

On the design of cost minimization for D2D‐enabled wireless caching networks: A joint recommendation, caching, and routing perspective

AbstractCache‐enabled device‐to‐device (D2D) network has been deemed as an effective technique to offload the data traffic. However, the gain of the caching schemes is closely related to the homogeneity among users' preference distribution. To tackle this issue, recommendation is a promising proactive approach. It increases the request probability of recommended contents, reshaping users' contents demand patterns, and improving caching performance. Moreover, considering the heterogeneous network settings, i.e. content retrieval costs vary, the routing design becomes a non‐negligible factor on caching performance optimization. On these grounds, the average system cost of D2D‐enabled wireless caching networks with multiple BSs is first described. Then the routing strategies are designed together with caching and recommendation policies by minimizing the average cost of these networks. The optimization problem is proven as NP‐hard. To facilitate the analysis, the original problem is decoupled into two sub‐problems and solve them respectively. Afterwards, all the variables are optimized in an alternating manner until the convergence is achieved. The proposed algorithm's convergence performance and benefits over benchmark strategies in terms of total cost and cache hit ratio are supported by Monte‐Carlo simulation results.

BERT4Cache: a bidirectional encoder representations for data prefetching in cache.

Cache plays a crucial role in improving system response time, alleviating server pressure, and achieving load balancing in various aspects of modern information systems. The data prefetch and cache replacement algorithms are significant factors influencing caching performance. Due to the inability to learn user interests and preferences accurately, existing rule-based and data mining caching algorithms fail to capture the unique features of the user access behavior sequence, resulting in low cache hit rates. In this article, we introduce BERT4Cache, an end-to-end bidirectional Transformer model with attention for data prefetch in cache. BERT4Cache enhances cache hit rates and ultimately improves cache performance by predicting the user's imminent future requested objects and prefetching them into the cache. In our thorough experiments, we show that BERT4Cache achieves superior results in hit rates and other metrics compared to generic reactive and advanced proactive caching strategies.

Exploring XRootD Optimisations Using Advanced Monitoring in the UK

XRootD servers are commonplace to many parts of HEP data management and are a key component to data access and management strategies in both the WLCG and OSG. Deployments of XRootD instances across the UK have demonstrated the versatility and expandability of this data management software. As we become more reliant on these services, there is a requirement to collect low-level metrics to monitor service performance and behaviour. This presentation will share our recent experiences in the collection, monitoring and analysis of such metrics from servers at UK WLCG-Tier2 sites. Building on work presented at VCHEP-2021 we have been collecting XRootD service metrics from various UK sites. In addition to this, we have developed novel technologies for recording and verifying metrics from our grid storage systems at the Edinburgh Tier2. Our custom tooling is integrated with our recently deployed monitoring platform. This work complements ongoing WLCG efforts focussing on capturing and analysing XRootD based traffic flow. Through capturing and analysing additional metrics we’re able to better remotely assess service health and provide insights to assist in debugging. The goal of this is to offer a centralised resource that monitors the performance and health of remote storage services across the UK. Making use of these additional service metrics also allows us a greater insight into the performance of XRootD Proxy File Caches. By applying machine learning techniques to already collected metrics, we aim to determine if gains can potentially be made in optimising the behaviour of these caching systems in production at a UK WLCG-Tier2. The result of this potentially being the building and deployment of custom decision-making libraries for use with XRootDPFC services.

Improving key-value cache performance with heterogeneous memory tiering: A case study of CXL-based memory expansion

Improving key-value cache performance with heterogeneous memory tiering: A case study of CXL-based memory expansion

Analyzing Data Reference Characteristics of Deep Learning Workloads for Improving Buffer Cache Performance

Due to the recent growing data size of deep learning workloads, loading data from storage is increasingly becoming a performance bottleneck for neural network systems. In this article, we analyze the data reference characteristics of neural network workloads and observe that they are significantly different from conventional desktop workloads. In particular, during the training phase of deep learning, data blocks are referenced in a fully random manner, which significantly degrades the performance of a buffer cache. To handle this situation, this article suggests a new data shuffling scheme that aims to accelerate data loading in deep neural networks. Unlike the default shuffling method used in PyTorch that randomly shuffles full dataset in every epoch, the proposed scheme defines a shuffling unit called bundle, and enhances the locality of data references to improve buffer cache performances. Specifically, the proposed scheme performs data shuffling by the unit of a bundle, and the bundles used in each epoch are arranged alternately, thereby improving the locality of references at the viewpoint of the buffer cache. Based on simulation and measurement studies, we show that the hit rate of the buffer cache is improved by 37.2%, and the data loading time is also shortened by 11.4% without degrading the model’s training efficiency.

On the Performance of Cache- and Energy Harvesting-Assisted NOMA in D2D Communications With Hardware Impairments

On the Performance of Cache- and Energy Harvesting-Assisted NOMA in D2D Communications With Hardware Impairments

PP-Bridge: Establishing a Bridge between the Prefetching and Cache Partitioning

— Modern computer processors are equipped with multiple cores, each boasting its own dedicated cache memory, while collectively sharing a generously sized Last Level Cache (LLC). To ensure equitable utilization of the LLC space and bolster system security, partitioning techniques have been introduced to allocate the shared LLC space among the applications running on different cores. This partition dynamically adapts to the requirements of these applications. Prefetching plays a vital role in enhancing cache performance by proactively loading data into the cache before it get requested explicitly by a core. Each core employs prefetch engines to decide which data blocks to fetch preemptively. However, a haphazard prefetcher may bring in more data blocks than necessary, leading to cache pollution and a subsequent degradation in system performance. To maximize the benefits of prefetching, it is essential to keep cache pollution to a minimum. Intriguingly, our research has uncovered that when existing prefetching techniques are combined with partitioning methods, they tend to exacerbate cache pollution within the LLC, resulting in a noticeable decline in system performance. In this paper, we present a novel approach aimed at mitigating cache pollution when combining prefetching with partitioning techniques.

VMIFresh: Efficient and fresh caches for virtual machine introspection

Virtual machine introspection (VMI) is the process of extracting knowledge about the inner state of a virtual machine from the outside. Traditional passive introspection mechanisms have proved themselves ineffective in many application domains due to their low performance. As a remedy for this issue, caching at the level of the introspection application was introduced. However, this sacrificed the freshness of VMI and led to an inconsistent outside view.In this work, we propose a multi-purpose hybrid caching scheme with freshness and consistency guarantees that is interleaved with the guest's MMU. This scheme can easily be integrated into existing applications and frameworks such as libvmi and Volatility 3. We demonstrate its feasibility by developing a prototype for such applications. Furthermore, the experimental evaluation of our approach suggests that it even significantly exceeds the performance of previous inconsistent caches.

Design and Development of Effective Multi-Level Cache Memory Model

An algorithm to determine the effectiveness and efficiency of a multi-level cache was developed in this paper. The developed model was used to test the efficiency rate, the relationships and the performance output level of a computer concerning the cache properties. This research paper showed that the level of cache and access time increases with the absolute hit rate but decreases with the relative hit rate. The number of cache levels varies directly with absolute access time and inversely with relative access time. The level one cache with set associativity of one has the highest access time and as the associativity and cache levels increase the memory access time decreases. The increase in the number of set associativity leads to an increase in cache performance and as well increases the performance speed of a computer.

Cache Optimization and Performance Modeling of Batched, Small, and Rectangular Matrix Multiplication on Intel, AMD, and Fujitsu Processors

Factorization and multiplication of dense matrices and tensors are critical, yet extremely expensive pieces of the scientific toolbox. Careful use of low rank approximation can drastically reduce the computation and memory requirements of these operations. In addition to a lower arithmetic complexity, such methods can, by their structure, be designed to efficiently exploit modern hardware architectures. The majority of existing work relies on batched BLAS libraries to handle the computation of many small dense matrices. We show that through careful analysis of the cache utilization, register accumulation using SIMD registers and a redesign of the implementation, one can achieve significantly higher throughput for these types of batched low-rank matrices across a large range of block and batch sizes. We test our algorithm on three CPUs using diverse ISAs – the Fujitsu A64FX using ARM SVE, the Intel Xeon 6148 using AVX-512, and AMD EPYC 7502 using AVX-2, and show that our new batching methodology is able to obtain more than twice the throughput of vendor optimized libraries for all CPU architectures and problem sizes.

Intelligent Cooperative Caching at Mobile Edge based on Offline Deep Reinforcement Learning

Cooperative edge caching enables edge servers to jointly utilize their cache to store popular contents, thus drastically reducing the latency of content acquisition. One fundamental problem of cooperative caching is how to coordinate the cache replacement decisions at edge servers to meet users’ dynamic requirements and avoid caching redundant contents. Online deep reinforcement learning (DRL) is a promising way to solve this problem by learning a cooperative cache replacement policy using continuous interactions (trial and error) with the environment. However, the sampling process of the interactions is usually expensive and time-consuming, thus hindering the practical deployment of online DRL-based methods. To bridge this gap, we propose a novel Delay-awarE Cooperative cache replacement method based on Offline deep Reinforcement learning (DECOR), which can exploit the existing data at the mobile edge to train an effective policy while avoiding expensive data sampling in the environment. A specific convolutional neural network is also developed to improve the training efficiency and cache performance. Experimental results show that DECOR can learn a superior offline policy from a static dataset compared to an advanced online DRL-based method. Moreover, the learned offline policy outperforms the behavior policy used to collect the dataset by up to 35.9%.

ESA: An efficient sequence alignment algorithm for biological database search on Sunway TaihuLight

In computational biology, biological database search has been playing a very important role. Since the COVID-19 outbreak, it has provided significant help in identifying common characteristics of viruses and developing vaccines and drugs. Sequence alignment, a method finding similarity, homology and other information between gene/protein sequences, is the usual tool in the database search. With the explosive growth of biological databases, the search process has become extremely time-consuming. However, existing parallel sequence alignment algorithms cannot deliver efficient database search due to low utilization of the resources such as cache memory and performance issues such as load imbalance and high communication overhead. In this paper, we propose an efficient sequence alignment algorithm on Sunway TaihuLight, called ESA, for biological database search. ESA adopts a novel hybrid alignment algorithm combining local and global alignments, which has higher accuracy than other sequence alignment algorithms. Further, ESA has several optimizations including cache-aware sequence alignment, capacity-aware load balancing and bandwidth-aware data transfer. They are implemented in a heterogeneous processor SW26010 adopted in the world’s 6th fastest supercomputer, Sunway TaihuLight. The implementation of ESA is evaluated with the Swiss-Prot database on Sunway TaihuLight and other platforms. Our experimental results show that ESA has a speedup of 34.5 on a single core group (with 65 cores) of Sunway TaihuLight. The strong and weak scalabilities of ESA are tested with 1 to 1024 core groups of Sunway TaihuLight. The results show that ESA has linear weak scalability and very impressive strong scalability. For strong scalability, ESA achieves a speedup of 338.04 with 1024 core groups compared with a single core group. We also show that our proposed optimizations are also applicable to GPU, Intel multicore processors, and heterogeneous computing platforms.