Cache Management Scheme Research Articles

Program context-assisted address translation for high-capacity SSDs

As the capacity of NAND flash-based SSDs keeps increasing, it becomes crucial to design a memory-efficient address translation algorithm that offers high performance when a translation table cannot be entirely loaded in a controller DRAM. Existing flash translation layers (FTL) employ demand-based address translation which caches popular mapping information in DRAM by leveraging locality of I/O references. Owing to the lack of information about detailed behaviors of applications, however, existing demand-based FTLs often suffer from many translation-table misses and thus result in sub-optimal performance. In this paper, we propose a new Program context-AssisteDFlash Translation Layer, called PADFTL. Unlike existing FTLs which are implemented as the form of firmware, PADFTL is vertically integrated with a host-level I/O classifier which provides useful hints for an FTL in an SSD to make a better decision in managing a translation table. The host-level I/O classifier monitors unique behaviors of applications by analyzing their program contexts and categorizes I/O patterns into four types, (1) Loop, (2) Hot, (3) Sequential, and (4) Random, which are then delivered to an SSD through extended interfaces. The SSD-side module of PADFTL partitions a controller DRAM into four zones and isolates mapping information associated with different I/O patterns into separate zones. By employing cache management strategies optimized for individual zones, PADFTL can lower the overall translation-table miss ratio. To evaluate the effectiveness of PADFTL, we implement the host-level classifier in the Linux kernel and PADFTL’s FTL in a trace-driven FTL simulator. In our experimental results, compared to the state-of-the-art FTL, PADFTL increases the overall table hit ratio by 16% while reducing the address translation time by up to 20% on average.

A Machine Learning-Empowered Cache Management Scheme for High-Performance SSDs

A Machine Learning-Empowered Cache Management Scheme for High-Performance SSDs

Catalyst: Optimizing Cache Management for Large In-memory Key-value Systems

In-memory key-value cache systems, such as Memcached and Redis, are essential in today's data centers. A key mission of such cache systems is to identify the most valuable data for caching. To achieve this, the current system design keeps track of each key-value item's access and attempts to make accurate estimation on its temporal locality. All it aims is to achieve the highest cache hit ratio. However, as cache capacity quickly increases, the overhead of managing metadata for a massive amount of small key-value items rises to an unbearable level. Put it simply, the current fine-grained, heavy-cost approach cannot continue to scale. In this paper, we have performed an experimental study on the scalability challenge of the current key-value cache system design and quantitatively analyzed the inherent issues related to the metadata operations for cache management. We further propose a key-value cache management scheme, called Catalyst , based on a highly efficient metadata structure, which allows us to make effective caching decisions in a scalable way. By offloading non-essential metadata operations to GPU, we can further dedicate the limited CPU and memory resources to the main service operations for improved throughput and latency. We have developed a prototype based on Memcached. Our experimental results show that our scheme can significantly enhance the scalability and improve the cache system performance by a factor of up to 4.3.

Adaptive Management With Request Granularity for DRAM Cache Inside nand-Based SSDs

Most flash-based solid-state drives (SSDs) adopt an on-board Dynamic Random Access Memory (DRAM) to buffer hot write data. Then, the write or overwrite operations can be absorbed by the DRAM cache, given that there is sufficient locality in the applications’ I/O access pattern, to consequently avoid flushing the write data onto underlying SSD cells. After analyzing typical real-world workloads over SSDs, we observed that the buffered data of small size requests are more likely to be re-accessed than those of large write requests. To efficiently utilize the limited space of DRAM cache, this paper proposes an adaptive request granularity-based cache management scheme for SSDs. First, we introduce a request block corresponding to a write request, as the cache management granularity, and propose a dynamic manner for classifying small and large request blocks. Next, we design three-level linked lists for supporting different routines of upgradation for small and large request blocks, once their data have been hit in the cache. At last, we present a scheme of evicting the request blocks having the minimum cost in cache replacement, by taking both factors of access hotness and time discounting into account. Experimental results show that our proposal can yield improvements on cache hits and the overall I/O latency by 21.8% and 14.7% on average, compared to state-of-the-art cache management schemes inside SSDs.

Visibility Graph-based Cache Management for DRAM Buffer Inside Solid-state Drives

Most solid-state drives (SSDs) adopt an on-board Dynamic Random Access Memory (DRAM) to buffer the write data, which can significantly reduce the amount of write operations committed to the flash array of SSD if data exhibits locality in write operations. This article focuses on efficiently managing the small amount of DRAM cache inside SSDs. The basic idea is to employ the visibility graph technique to unify both temporal and spatial locality of references of I/O accesses, for directing cache management in SSDs. Specifically, we propose to adaptively generate the visibility graph of cached data pages and then support batch adjustment of adjacent or nearby (hot) cached data pages by referring to the connection situations in the visibility graph. In addition, we propose to evict the buffered data pages in batches by also referring to the connection situations, to maximize the internal flushing parallelism of SSD devices without worsening I/O congestion. The trace-driven simulation experiments show that our proposal can yield improvements on cache hits by between 0.8 % and 19.8 %, and the overall I/O latency by 25.6 % on average, compared to state-of-the-art cache management schemes inside SSDs.

An efficient adaptive cache management scheme for named data networks

With the advent of modern technologies and applications, NDN networks have been known as a reliable approach to meeting the needs ahead. A prominent feature of these networks is the ability to cache comprehensive content within network nodes. Separating the content from the original location reduces the content retrieval latency, network traffic, and overhead within the applications containing considerable required data volume. This approach also improves the user experience. In this article, first, the idea of intra-network caching with an approach based on the cooperation of adjacent nodes along the path is proposed to form a local virtual cluster with distributed cache space. Second, popular cached data will be shared by exchanging minimal notification messages between them to develop content storage decisions. Two new modules are defined in each node, called RIT and NCT which have offered the possibility of utilizing the resources of nodes adjacent to the delivered route. In the next stage, a novel approach is proposed to determine the content caching threshold in each node by considering the parameters affecting the content, as well as the network topology and real-time status of the nodes. Finally, the decision mechanism regarding the storage of content in each node along the path is presented by using an adaptive approach based on set thresholds. In such a manner, undesirable redundancy of content that wastes network resources will be removed, and more caching space in each area of the network will be provided. The results of simulations using ndnSim revealed that the performance of the proposed algorithm (cache hit ratio, average data delivery latency and path stretch) is better than other existing benchmark strategies. Considering other different parameters confirmed the effectiveness of the presented approach.

DAC: A dynamic active and collaborative cache management scheme for solid state disks

NAND Flash memory-based SSDs have been widely used in modern storage systems to improve the performance and energy consumption. Existing caching management schemes rarely consider the utilization of parallel resources in flash memory array inside an SSD while these parallel resources cannot be fully exploited. The I/Os-access conflicts deteriorate the SSD performance and enlarge the response time under write-intensive workloads. Most cache replacement strategies give priority to clean data to improve the I/O-access conflicts, thereby boosting the SSD performance. However, clean data entirely depends on missed read-I/Os requests. Ideally, data replacement ought to evict clean data in the cache thanks to low replacement cost. As such, flash memory-based caching schemes give the highest priority to clean data. However, the proportion of replaced clean data is low in cache data. To solve this issue, we propose a dynamic active and collaborative cache management named DAC, in which the cache is composed of cold cache, hot cache, ghost cold cache, and ghost hot cache. DAC determines hot-cold changes of the I/O requests in workloads based on ghost cache size, thereby adjusting the real cache size to well serve I/O requests. The dynamic write-back window (DWW) mechanism dynamically adjusts the write-back window size in the cold cache. The write-back thresholds automatically update according to the changes in I/O patterns in workloads. When a flash chip is in the idle state, DAC produces clean data by proactively writing normal cold cache data back into flash memory, and the clean data is migrated into the active cold cache. DAC prioritizes replacing data in the active cold cache, aiming to avoid evicting data that are just read from flash memory. This method is adept at boosting the replacement rate of clean data, thereby optimizing SSD performance. We undertake extensive experiments in the latest NVMe SSD simulator to validate the DAC’s edge over the seven existing caching schemes including LRU, CFLRU, GCaR-CFLRU, LCR, ARC, AD-LRU, and MQsim. The results unveil that our DAC shortens the response time by up to 60.8% with an average optimization rate of 24.4%. When it comes to response-time cliff, the maximum improvement offered by DAC reaches approximately 68.71%; meanwhile, DAC delivers an average optimization rate of 26.9%. Besides, the number of erase count reduces by up to 93.3% with an average improvement rate of 17.04%.

An Effective Management Model for Data Caching in MANET Environment

A mobile ad-hoc (MANET) network has the main challenge to provide the needed data for the desired mobile nodes. An efficient on request routing protocol for MANET is Ad-hoc on-demand Distance Vector (AODV), which is based on two main methods: route discovery and route maintenance. Route discovery is the process used to detect a route to the destination from the packet source, while route maintenance is the process used to detect a link failure and repair it. Cooperative caching tends improving data availability in mobile ad-hoc networks, the coordination of cache discovery and cache management strategies is very significant in the cooperative caching of MANETs because requests for data and answers to requested data can be reduced simply due to interference, network congestion, or when a forwarding node is out of reach and the route breaks down. Cooperative cache management is much more complicated in cooperative caching because it also depends on neighbouring nodes to decide what to cache. In this paper, three algorithms were proposed: (1) a combination algorithm for cache admission control based on cache data and location of data to save space and reduce data redundancy, (2) a value-based policy for cache placement and replacement instead of the more common least recently used strategy, depending on metrics that describe cached items to increase the local cache hit ratio, and (3) a combined algorithm for cache consistency that includes time-to-live, pull, and push policies to enhance data availability and system scalability. The proposed algorithm implemented by the NS3 simulation program; which used to create a network using the AODV protocol in several parameters and achieve better system performance.

An Effective Management Model for Data Caching in MANET Environment

A mobile ad-hoc (MANET) network has the main challenge to provide the needed data for the desired mobile nodes. An efficient on request routing protocol for MANET is Ad-hoc on-demand Distance Vector (AODV), which is based on two main methods: route discovery and route maintenance. Route discovery is the process used to detect a route to the destination from the packet source, while route maintenance is the process used to detect a link failure and repair it. Cooperative caching tends improving data availability in mobile ad-hoc networks, the coordination of cache discovery and cache management strategies is very significant in the cooperative caching of MANETs because requests for data and answers to requested data can be reduced simply due to interference, network congestion, or when a forwarding node is out of reach and the route breaks down. Cooperative cache management is much more complicated in cooperative caching because it also depends on neighbouring nodes to decide what to cache. In this paper, three algorithms were proposed: (1) a combination algorithm for cache admission control based on cache data and location of data to save space and reduce data redundancy, (2) a value-based policy for cache placement and replacement instead of the more common least recently used strategy, depending on metrics that describe cached items to increase the local cache hit ratio, and (3) a combined algorithm for cache consistency that includes time-to-live, pull, and push policies to enhance data availability and system scalability. The proposed algorithm implemented by the NS3 simulation program; which used to create a network using the AODV protocol in several parameters and achieve better system performance.

A Space-Efficient Fair Cache Scheme Based on Machine Learning for NVMe SSDs

Non-volatile memory express (NVMe) solid-state drives (SSDs) have been widely adopted in multi-tenant cloud computing environments or multi-programming systems. The on-board DRAM cache inside NVMe SSDs can efficiently reduce the disk accesses and extend the lifetime of SSDs. Current SSD cache management research either improves cache hit ratio while ignoring fairness, or improves fairness while sacrificing overall performance. In this paper, we present MLCache, a space-efficient shared cache management scheme for NVMe SSDs. By learning the impact of reuse distance on cache allocation, a workload-generic neural network model is built. At runtime, MLCache continuously monitors the reuse distance distribution for the neural network module to obtain space-efficient allocation decisions. MLCache also proposes an efficient parallel writing back strategy based on hit ratio and response time, to improve fairness. Experimental results show MLCache improves the write hit ratio when compared to baseline, and MLCache strongly safeguards the fairness of SSDs with parallel write-back and maintains a low level of degradation.

Weight-Aware Cache for Application-Level Proportional I/O Sharing

Virtualization technology has enabled server consolidation where multiple servers are co-located on a single physical machine to improve resource utilization. In such systems, proportional I/O sharing is critical to meet the SLO (Service-Level Objectives) of the applications running in each virtual instance. However, previous studies focus on block-level I/O proportionality without considering the upper-layer I/O caches, which handle I/O requests on behalf of the underlying storage devices, thereby failing to achieve application-level proportional I/O sharing. To overcome this limitation, we propose a new cache management scheme, Weight-aware Cache (WaC), which reflects the I/O weights on cache allocation and reclamation. Specifically, WaC prioritizes higher-weighted applications in the lock acquisition process of cache allocation by re-ordering the lock waiting queue based on I/O weight. Additionally, WaC keeps the number of cache entries of each application proportional to its I/O weight, through weight-aware cache reclamation. To verify the efficacy of our scheme, we implement and evaluate WaC on both the page cache and bcache. The experimental results demonstrate that our scheme improves I/O proportionality with negligible overhead in various cases.

Energy-Saving SSD Cache Management for Video Servers with Heterogeneous HDDs

Dynamic adaptive streaming over HTTP (DASH) technique, the most popular streaming method, requires a large number of hard disk drives (HDDs) to store multiple bitrate versions of many videos, consuming significant energy. A solid-state drive (SSD) can be used to cache popular videos, thus reducing HDD energy consumption by allowing I/O requests to be handled by an SSD, but this requires effective HDD power management due to limited SSD bandwidth. We propose a new SSD cache management scheme to minimize the energy consumption of a video storage system with heterogeneous HDDs. We first present a technique that caches files with the aim of saving more HDD energy as a result of I/O processing on an SSD. Based on this, we propose a new HDD power management algorithm with the goal of increasing the number of HDDs operated in low-power mode while reflecting the heterogeneous HDD power characteristics. For this purpose, it assigns a separate parameter value to each I/O task based on the ratio of HDD energy to bandwidth and greedily selects the I/O tasks handled by the SSD within limits on its bandwidth. Simulation results show that our scheme consumes between 12% and 25% less power than alternative schemes under the same HDD configuration.

Research on Load Balancing and Caching Strategy for Central Network

The central network architecture takes the information content as the processing object and can better meet the user access needs. However, the traditional network node load balancing and caching strategy has been difficult to adapt to the development of the central network architecture. Therefore, this article proposed a load balancing and cache management strategy based on the central network in order to provide a reference for the architecture and application of the central network. Based on the analysis of the existing network architecture, this article expounded the networking form and characteristics of the central network system. By analyzing the load scheduling requirements of network nodes, combined with the transmission characteristics of resource requests in the central network, a cache node load balancing method suitable for the requirements of the central network was proposed based on the existing load balancing algorithms. In addition, by analyzing the cache management strategy and its impact on the performance of the network system, a cache node information forwarding model with resource requests as the object in the central network environment was given. Finally, through experimental test and comparative analysis, the results showed that the method proposed in this article had certain feasibility and effectiveness. Compared with other typical strategies, the load balancing and caching strategy proposed in this article can better meet the user needs of the central network.

Dynamic Cooperative Cache Management Scheme Based on Social and Popular Data in Vehicular Named Data Network

Vehicular Named Data Network (VNDN) is considered a strong paradigm to deploy in vehicular applications. In VNDN, each node has its cache, but due to limited cache, it directly affects the performance in a highly dynamic environment, which requires massive and fast content delivery. To reduce these issues, the cooperative caching plays an efficient role in VNDN. Most studies regarding cooperative caching focus on content replacement and caching algorithms and implement these methods in a static environment rather than a dynamic environment. In addition, few existing approaches addressed the cache diversity and latency in VNDN. This paper proposes a Dynamic Cooperative Cache Management Scheme (DCCMS) based on social and popular data, which improves the cache efficiency and implements it in a dynamic environment. We designed a two-level dynamic caching scheme, in which we choose the right caching node that frequently communicates with other nodes, keep the copy of the most popular content, and distribute it with the requester’s node when needed. The main intention of DCCMS is to improve the cache performance in terms of reducing latency, server load, cache hit ratio, average hop count, cache utilization, and diversity. The simulation results show that our proposed DCCMS scheme improves the cache performance than other state-of-the-art approaches.

Adaptive DSR routing caching strategy based on local connection mechanism

An optimisation solution for DSR routing protocol cache management strategy based on local connection mechanism is proposed to solve the problem that DSR routing protocol can easily cause incorrect routing and cause network performance degradation when dealing with dynamically changing network topology. A local connection mechanism is introduced to ensure the reachability of links between nodes and improve the accuracy of cached routing entries. Finally, using the network simulation tool NS-3 to test the improved DSR routing protocol after the cache strategy, compare the packet delivery rate of the nodes before and after the optimisation, the route initiation frequency and the average delay. Simulation results show that the proposed scheme can effectively improve the packet data delivery rate of the DSR routing protocol and reduce the average delay and enhance the overall performance of the protocol.

Providing Cache Consistency Guarantee for ICN-Based IoT Based on Push Mechanism

This paper develops POPCache (POpularity-aware Push-based Cache), a cache management strategy to support cache consistency in ICN-based IoT. Based on the proposed push mechanism, POPCache can proactively update the slated content to impose strong consistency while achieving low overhead, because POPCache relieves the heavy burden of nodes (which can be resource-constrained IoT devices) of maintaining extra information. POPCache makes caching decisions by jointly considering several determining factors. We use the theoretical study to analyze POPCache’s overhead. Moreover, we conduct simulations to compare POPCache with existing works; results show that POPCache improves cache consistency and achieves good cache and network performance.

Performance-oriented cache management scheme based on a retention state for energy-harvesting nonvolatile processors

In this paper, we address the challenges of frequent backup and recovery operations associated with an unstable energy supply in energy-harvesting nonvolatile processors (NVPs). To address this problem, this paper adopts a retention state, which can allow the system to retain data in a volatile cache to wait for power recovery instead of immediately backing up the data. In addition, in an energy-harvesting NVP, important issues include how to back up and recover data, how to perform cache replacement, and how to transmit data to different parts of the cache in such a way that the performance and energy consumption can be improved and a successful backup can be guaranteed. Therefore, we propose a performance-oriented cache management scheme based on a retention state to generate near-optimal data migration, replacement and backup solutions for applications in polynomial time. This scheme uses two energy thresholds to pre-back up and back up data to reduce the backup and recovery operations according to the retention state. We evaluate the performance and energy consumption of our proposed algorithms in comparison with those of the dual-threshold SLC STT-RAM scheme and morphable cache scheme. The evaluation shows that the proposed scheme can achieve 18.58% and 4.54% lower average energy consumption than the SLC STT-RAM and morphable cache schemes, respectively, with comparable performance. The experimental results also show that compared with the SLC STT-RAM and morphable cache schemes, the proposed algorithm can achieve 17.90% and 8.85% performance improvement, respectively, on average.

Penalty- and Locality-aware Memory Allocation in Redis Using Enhanced AET

Due to large data volume and low latency requirements of modern web services, the use of an in-memory key-value (KV) cache often becomes an inevitable choice (e.g., Redis and Memcached). The in-memory cache holds hot data, reduces request latency, and alleviates the load on background databases. Inheriting from the traditional hardware cache design, many existing KV cache systems still use recency-based cache replacement algorithms, e.g., least recently used or its approximations. However, the diversity of miss penalty distinguishes a KV cache from a hardware cache. Inadequate consideration of penalty can substantially compromise space utilization and request service time. KV accesses also demonstrate locality, which needs to be coordinated with miss penalty to guide cache management. In this article, we first discuss how to enhance the existing cache model, the Average Eviction Time model, so that it can adapt to modeling a KV cache. After that, we apply the model to Redis and propose pRedis, Penalty- and Locality-aware Memory Allocation in Redis, which synthesizes data locality and miss penalty, in a quantitative manner, to guide memory allocation and replacement in Redis. At the same time, we also explore the diurnal behavior of a KV store and exploit long-term reuse. We replace the original passive eviction mechanism with an automatic dump/load mechanism, to smooth the transition between access peaks and valleys. Our evaluation shows that pRedis effectively reduces the average and tail access latency with minimal time and space overhead. For both real-world and synthetic workloads, our approach delivers an average of 14.0%∼52.3% latency reduction over a state-of-the-art penalty-aware cache management scheme, Hyperbolic Caching (HC), and shows more quantitative predictability of performance. Moreover, we can obtain even lower average latency (1.1%∼5.5%) when dynamically switching policies between pRedis and HC.

GCS: Collaborative video cache management strategy in multi-access edge computing

With the rapid development of multimedia services in wireless mobile networks, the video data traffic has increased exponentially. Traditional video traffic service based on cloud computing caused a large amount of traffic load and longer access delay, which severely reduces the quality of service (QoS) of users. Multi-Access Edge Computing (MEC) can directly serve user requests so as to greatly reduce the traffic load and shorten access delay. At present, distributed caching is widely used in the caching deployment of base stations (BSs). However, the caching capacity of a single BS is generally particularly limited, which will degrade the performance of wireless mobile network. In this paper, a collaborative caching scheme in the heterogeneous MEC networks is designed, and the edge caching of macro base station (MBS) and small base stations (SBSs) is utilized to bring storage resources closer to users. In addition, an optimization problem of content caching is formulated to minimize the total delay cost of all users requesting content in the MEC networks. In order to solve this problem, cache management strategy (GCS) is proposed, which consists of greedy cache placement strategy and greedy cache update strategy. Finally, numerical simulations demonstrate that the GCS scheme effectively improves the cache hit rate, and significantly reduces the average delay and backhaul traffic load.

A-Cache: Asymmetric Buffer Cache for RAID-10 Systems Under a Single-Disk Failure to Significantly Boost Availability

The RAID-10 architecture has been widely deployed in commercial and industrial storage environments over the past two decades due to its high reliability, availability, and performance. However, during the recovery process of a single disk failure, which accounts for more than 99.75% of the disk failure scenarios, it is still at a high risk of data loss and suffers from a degradation of user I/O performance, which results from the severe interference between the user and recovery I/Os. Based on our observations and analyses, we find highly asymmetric disk bandwidth utilization and disk I/O interference during the recovery process of the RAID-10 systems under the single faulty-disk condition. Motivated by the fact that this asymmetry can be leveraged to significantly and simultaneously speed up the recovery and user I/O performances. As a result, we propose a novel asymmetric buffer cache management scheme, called A-Cache, to mitigate this asymmetry by allocating cache space asymmetrically between the disks that do not participate in the recovery process and those that do. To verify the effectiveness of the A-cache, we have integrated A-Cache into the popular cache algorithms LRU, named A-LRU. The evaluation of our prototype system demonstrates that A-LRU is able to significantly speed up the recovery speed and average user I/O latency under various typical configurations, compared to the original LRU, without any additional hardware cost.