Scalable Cache Research Articles

A High-Performance Accelerator for Real-Time Super-Resolution on Edge FPGAs

In the digital era, the prevalence of low-quality images contrasts with the widespread use of high-definition displays, primarily due to low-resolution cameras and compression technologies. Image super-resolution (SR) techniques, particularly those leveraging deep learning, aim to enhance these images for high-definition presentation. However, real-time execution of deep neural network (DNN)-based SR methods at the edge poses challenges due to their high computational and storage requirements. To address this, field-programmable gate arrays (FPGAs) have emerged as a promising platform, offering flexibility, programmability, and adaptability to evolving models. Previous FPGA-based SR solutions have focused on reducing computational and memory costs through aggressive simplification techniques, often sacrificing the quality of the reconstructed images. This paper introduces a novel SR network specifically designed for edge applications, which maintains reconstruction performance while managing computation costs effectively. Additionally, we propose an architectural design that enables the real-time and end-to-end inference of the proposed SR network on embedded FPGAs. Our key contributions include a tailored SR algorithm optimized for embedded FPGAs, a DSP-enhanced design that achieves a significant four-fold speedup, a novel scalable cache strategy for handling large feature maps, optimization of DSP cascade consumption, and a constraint optimization approach for resource allocation. Experimental results demonstrate that our FPGA-specific accelerator surpasses existing solutions, delivering superior throughput, energy efficiency, and image quality.

Adaptive and Scalable Caching With Erasure Codes in Distributed Cloud-Edge Storage Systems

Erasure codes have been widely used to enhance data resiliency with low storage overheads. However, in geo-distributed cloud storage systems, erasure codes may incur high service latency as they require end users to access remote storage nodes to retrieve data. An elegant solution to achieving low latency is to deploy caching services at the edge servers close to end users. In this paper, we propose adaptive and scalable caching schemes to achieve low latency in the cloud-edge storage system. Based on the measured data popularity and network latencies in real time, an adaptive content replacement scheme is proposed to update caching decisions upon the arrival of requests. Theoretical analysis shows that the reduced data access latency of the replacement scheme is at least 50% of the maximum reducible latency. With the low computation complexity of our design, nearly no extra overheads will be introduced when handling intensive data flows. For further performance improvements without sacrificing its efficiency, an adaptive content adjustment scheme is presented to replace the subset of cached contents that incur the aforementioned performance loss. Driven by real-world data traces, extensive experiments based on Amazon Simple Storage Service demonstrate the effectiveness and efficiency of our design.

Proactive Content Caching Based on Actor–Critic Reinforcement Learning for Mobile Edge Networks

Mobile edge caching/computing (MEC) has emerged as a promising approach for addressing the drastic increasing mobile data traffic by bringing high caching and computing capabilities to the edge of networks. Under MEC architecture, content providers (CPs) are allowed to lease some virtual machines (VMs) at MEC servers to proactively cache popular contents for improving users’ quality of experience. The scalable cache resource model rises the challenge for determining the ideal number of leased VMs for CPs to obtain the minimum expected downloading delay of users at the lowest caching cost. To address these challenges, in this paper, we propose an actor-critic (AC) reinforcement learning based proactive caching policy for mobile edge networks without the prior knowledge of users’ content demand. Specifically, we formulate the proactive caching problem under dynamical users’ content demand as a Markov decision process and propose a AC based caching algorithm to minimize the caching cost and the expected downloading delay. Particularly, to reduce the computational complexity, a branching neural network is employed to approximate the policy function in the actor part. Numerical results show that the proposed caching algorithm can significantly reduce the total cost and the average downloading delay when compared with other popular algorithms.

Intelligent Resource Management at the Edge for Ubiquitous IoT: An SDN-Based Federated Learning Approach

The ubiquitous nature of Internet of Things (IoT) devices has posited many challenges that need innovative solutions in the 5G era. Software defined networks (SDNs) are becoming indispensable in managing several aspects of next-generation IoT networking that arise from the need to control highly heterogeneous, geographically dispersed, mobile IoT devices. One such aspect is cache management at the edge. Recently, multiple forms of edge resources, including mobile device clouds and micro-edge data centers have emerged to provide scalable cache placement locations that reduce the costs for the mobile network operator (MNO). As all of these service locations are registered with the MNO (or links established after registration with the 5G base station, BS), content should be placed according to the user's demand and the cost the user is willing to pay to receive the desired level of QoS. To this end, it is important to understand the future popularity of the content for its optimal placement considering the highly dynamic user mobility. In this article, we address two key aspects of a mobile IoT network: security and seamless connectivity for data delivery. We rely on the federated learning (FL) architecture, which enables harnessing data and computational capabilities at end-user devices to train machine learning models. We study FL concepts in the domain of edge computing for IoT use cases, such as caching. We draw conclusions from various state-of-the-art models and posit several challenges that can be overcome via a novel proposed control algorithm.

Pache: A Packet Management Scheme of Cache in Data Center Networks

The communication traffic among servers is a principal bottleneck for data center networks (DCNs), in which redundant traffic has a significant impact on the performance of DCNs. In this paper, we propose Pache , a distributed and efficient cache scheme to achieve traffic redundancy elimination in DCNs. In Pache, nodes cache packets to reduce bandwidth consumption and gain performance revenue. Each cache node manages the local cache and improves cache efficiency with counting bloom filter, hash table and Trie. Each cache node broadcasts the local cache information by a bloom filter to other servers, and servers have cache information tables (cache-table) to record them, which achieves a cache sharing mechanism. Moreover, Pache employs false positive information table (fp-table) to reduce the false positive rate caused by bloom filters. A server determines an original packet or the corresponding fingerprint packet to send by querying cache-table and fp-table. Cache node placement mechanism is also designed to select which node to cache a packet. For evaluating the performance of Pache, we execute extensive simulations and conduct a case study on Amazon EC2 platform from different aspects. The results show that Pache can eliminate about 40 percent redundant traffic on average, and only increase 10 percent runtime, and it is also a scalable and feasible cache scheme in DCNs.

A Fine-grain and scalable set-based cache partitioning through threadclassification

As contemporary processors utilize more and more cores, cache partitioning algorithms tend to preserve cache associativity with a finer-grain of control to achieve higher throughput and fairness goals. In this study, we propose a scalable set-based cache partitioning mechanism, which welds an allocation policy and an enforcement scheme together. We also propose a set-based classifier to better allocate partitions to more deserving threads, a fast set redirection logic to map accesses to dedicated cache sets, and a double access mechanism to overcome the performance penalty due to a repartitioning phase. We compare our work to the best line-grain cache partitioning scheme that is available in the literature. Our results show that set-based partitioning improves throughput and fairness by 5.6 % and 4.8 % on average, respectively. The maximum achievable gains are as high as 33 % in terms of throughput and 23 % in terms of fairness.

Progressive Caching System for Video Streaming Services Over Content Centric Network

This paper presents a metafile-based progressive caching system over the content-centric networking (CCN) tree that supports seamless video streaming services with a high network utilization. In the proposed caching system, each CCN node uses a metafile made by a scalable caching algorithm for efficient and fast chunk caching management, and the reserved area of the CCN interest/data packet headers is used to deliver caching information among the CCN nodes. Based on this caching information, the proposed caching system determines the caching range of video data to minimize the required peak bandwidth for each link. The proposed caching system is implemented using the NS-3 based named data networking simulator. Furthermore, a real cellular wireless network testbed is realized with C/C++, open sources such as CCNx and Ubuntu MATE, and a Raspberry PIs to examine the performance of the proposed caching system. The experiment results demonstrate the performance improvement achieved by the proposed caching system.

Belle II Conditions Database

The Belle II experiment at KEK is preparing for taking first collision data in early 2018. For the success of the experiment it is essential to have information about varying conditions available to systems worldwide in a fast and efficient manner that is straightforward for both the user and maintainer. The Belle II Conditions Database was designed to make maintenance as easy as possible. To this end, a HTTP REST service was developed with industry-standard tools such as Swagger for the API interface development, Payara for the Java EE application server, and the Hazelcast in-memory data grid for support of scalable caching as well as transparent distribution of the service across multiple sites.On the client side, the online and offline software has to be able to obtain conditions data from the Belle II Conditions Database in a robust and reliable way under very different situations. As such the client side interface to the Belle II Conditions Database has been designed with a variety of access mechanisms which allow the software to be used with and without an internet connection. Different methods to access the payload information are implemented to allow for a high level of customization per site and to simplify testing of new payloads locally. Changes to the conditions data are usually handled transparently but users can actively check whether an object has changed or register callback functions to be called whenever a conditions data object is updated. In addition a command line user interface has been developed to simplify inspection and modification of the database contents.

Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities

Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this work, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.

(FPL 2015) Scavenger

High-level abstractions separate algorithm design from platform implementation, allowing programmers to focus on algorithms while building complex systems. This separation also provides system programmers and compilers an opportunity to optimize platform services on an application-by-application basis. In field-programmable gate arrays (FPGAs), platform-level malleability extends to the memory system: Unlike general-purpose processors, in which memory hardware is fixed at design time, the capacity, associativity, and topology of FPGA memory systems may all be tuned to improve application performance. Since application kernels may only explicitly use few memory resources, substantial memory capacity may be available to the platform for use on behalf of the user program. In this work, we present Scavenger, which utilizes spare resources to construct program-optimized memories, and we also perform an initial exploration of methods for automating the construction of these application-specific memory hierarchies. Although exploiting spare resources can be beneficial, naïvely consuming all memory resources may cause frequency degradation. To relieve timing pressure in large block RAM (BRAM) structures, we provide microarchitectural techniques to trade memory latency for design frequency. We demonstrate, by examining a set of benchmarks, that our scalable cache microarchitecture achieves performance gains of 7% to 74% (with a 26% geometric mean on average) over the baseline cache microarchitecture when scaling the size of first-level caches to the maximum.

Scalable Cache Management for ISP-Operated Content Delivery Services

Content delivery networks (CDNs) have been the prevalent method for the efficient delivery of content across the Internet. Management operations performed by CDNs are usually applied only based on limited information about Internet Service Provider (ISP) networks, which can have a negative impact on the utilization of ISP resources. To overcome these issues, previous research efforts have been investigating ISP-operated content delivery services, by which an ISP can deploy its own in-network caching infrastructure and implement its own cache management strategies. In this paper, we extend our previous work on ISP-operated content distribution and develop a novel scalable and efficient distributed approach to control the placement of content in the available caching points. The proposed approach relies on parallelizing the decision-making process and the use of network partitioning to cluster the distributed decision-making points, which enables fast reconfiguration and limits the volume of information required to take reconfiguration decisions. We evaluate the performance of our approach based on a wide range of parameters. The results demonstrate that the proposed solution can outperform previous approaches in terms of management overhead and complexity while offering similar network and caching performance.

Innovative Mobile E-Healthcare Systems: A New Rule-Based Cache Replacement Strategy Using Least Profit Values

Providing and managing e-health data from heterogeneous and ubiquitous e-health service providers in a content distribution network (CDN) for providing e-health services is a challenging task. A content distribution network is normally utilized to cache e-health media contents such as real-time medical images and videos. Efficient management, storage, and caching of distributed e-health data in a CDN or in a cloud computing environment of mobile patients facilitate that doctors, health care professionals, and other e-health service providers have immediate access to e-health information for efficient decision making as well as better treatment. Caching is one of the key methods in distributed computing environments to improve the performance of data retrieval. To find which item in the cache can be evicted and replaced, cache replacement algorithms are used. Many caching approaches are proposed, but the SACCS—Scalable Asynchronous Cache Consistency Scheme—has proved to be more scalable than the others. In this work, we propose a new cache replacement algorithm—Profit SACCS—that is based on the rule-based least profit value. It replaces the least recently used strategy that SACCS uses. A comparison with different cache replacement strategies is also presented.

Are distributed sharing codes a solution to the scalability problem of coherence directories in manycores? An evaluation study

The development of efficient and scalable cache coherence protocols is a key aspect in the design of manycore chip multiprocessors. In this work, we present a comprehensive evaluation of a kind of ...

I2CC: Interleaving two-level cache with network coding in peer-to-peer VoD system

As video technologies have advanced, peer-to-peer video-on-demand services have increasingly proliferated. In this paper, an interleaving two-level cache with network coding (I2CC) scheme is proposed to provide rapid content searching and efficiently support random seeking with the dynamic nature of peers. In I2CC, a video segment is composed of chunks, and each chunk is divided into blocks. These blocks are encoded by a network coding technique and distributed to different peers for storage in a local two-level cache, which includes a prefix cache and suffix cache. The prefix cache helps achieve rapid parent peer searching, low startup delays, and quick responses for jump operations. The suffix cache primarily provides a scalable cache size adjustment based on the limited storage space of peers. Additionally, enhancement and compensation mechanisms are provided to reduce playback delay and more efficiently utilize limited uploading bandwidth. We also present an analytical model for determining appropriate system parameters to reduce the chunk loss probability. Experimental results show that the proposed I2CC scheme outperforms competing schemes in terms of startup delay, jump delay, and server stress under different peer arrival times and video bitrates.

A Resource-Constrained Asymmetric Redundancy Elimination Algorithm

We focus on the problem of efficient communications over access networks with asymmetric bandwidth and capability. We propose a resource-constrained asymmetric redundancy elimination algorithm (RCARE) to leverage downlink bandwidth and receiver capability to accelerate the uplink data transfer. RCARE can be deployed on a client or a proxy. Different from existing asymmetric algorithms, RCARE uses a flexible matching mechanism to identify redundant data and allocates a small sender cache to absorb the high downlink traffic overhead. Compared to existing redundancy elimination algorithms, RCARE provides a scalable sender cache that is adaptive based on resource and performance. We evaluate RCARE with real traffic traces collected from multiple servers and a campus gateway. The trace-driven simulation results indicate that RCARE achieves higher goodput gains and reduces downlink traffic compared to existing asymmetric communication algorithms. We design an adaptation algorithm for resource-constrained senders sending multiple data streams. Our algorithm takes samples from data streams and predicts how to invest cache size on individual data streams to achieve maximal uplink goodput gain. The adaptation algorithm improves the goodput gain by up to 87% compared to the baseline. In first 10% of data streams (sorted by the optimal goodput gains), RCARE achieves up to 42% goodput gain on average.

Static Task Partitioning for Locked Caches in Multicore Real-Time Systems

Growing processing demand on multitasking real-time systems can be met by employing scalable multicore architectures. For such environments, locking cache lines for hard real-time systems ensures timing predictability of data references and may lower worst-case execution time. This work studies the benefits of cache locking on massive multicore architectures with private caches in the context of hard real-time systems. In shared cache architectures, the cache is a single resource shared among all of the tasks. However, in scalable cache architectures with private caches, conflicts exist only among the tasks scheduled on one core. This calls for a cache-aware allocation of tasks onto cores. The objective of this work is to increase the predictability of memory accesses resolved by caches while reducing the number of cores for a given task set. This allows designers to reduce the footprint of their subsystem of real-time tasks and thereby cost, either by choosing a product with fewer cores as a target or to allow more subsystems to be co-located on a given fixed number of cores. Our work proposes a novel variant of the cache-unaware First Fit Decreasing (FFD) algorithm called Naive locked First Fit Decreasing (NFFD) policy. We propose two cache-aware static scheduling schemes: (a) Greedy First Fit Decreasing (GFFD) and (b) Colored First Fit Decreasing (CoFFD) for task sets where tasks do not have intratask conflicts among locked regions (Scenario A). NFFD is capable of scheduling high utilization task sets that FFD cannot schedule. Experiments also show that CoFFD consistently outperforms GFFD, resulting in a lower number of cores and lower system utilization. CoFFD reduces the number of core requirements by 30% to 60% compared to NFFD. For a more generic case where tasks have intratask conflicts, we split the task partitioning between two phases: task selection and task allocation (Scenario B). Instead of resolving conflicts at a global level, these algorithms resolve conflicts among regions while allocating a task onto a core and unlocking at region level instead of task level. We show that a combination of dynamic ordering (task selection) with Chaitin’s Coloring (task allocation) scheme reduces the number of cores required by up to 22% over a basic scheme (in a combination of monotone ordering and regional FFD). Regional unlocking allows this scheme to outperform CoFFD for medium utilization task sets from Scenario A. However, CoFFD performs better than any other scheme for high utilization task sets from Scenario A. Overall, this work is unique in considering the challenges of future multicore architectures for real-time systems and provides key insights into task partitioning and cache-locking mechanisms for architectures with private caches.

Improving multiprocessor performance with fine-grain coherence bypass

Efficient and scalable cache coherence protocol is crucial to high-performance servers with shared-memory. The directory-based cache coherence protocol is more desirable than the snooping-based protocol with respect to the scalability. However, even for the former protocol, scaling to a large number of cores is also challenging due to the additional area requirements of the directories. We observed that a significant percentage of the referenced memory blocks were only accessed by a single core (even in parallel applications) which could be considered as private memory blocks. An intuitive motivation from this observation is that memory blocks accessed by a single core do not require coherence maintenance. The issue is to identify the private block and track the change of its access pattern. We propose a novel hardware approach to (1) dynamically identify the shared memory blocks at the cache block level, and (2) bypass the coherence procedure for the private memory blocks. This approach increases the effectiveness of the directory-based approach and therefore improves the system performance. Experimental results showed that, our approach can on an average (1) avoid the coherence tracking of about 54% referenced memory blocks, (2) reduce the coherence overhead by 77%, (3) avoid 8% L2 cache misses, and (4) shorten the execution time of parallel applications by 13%.

Bandwidth Adaptive Cache Coherence Optimizations for Chip Multiprocessors

Chip Multiprocessors (CMPs) have different technological parameters and physical constraints than earlier multi-processor systems, which should be taken into consideration when designing cache coherence protocols. Also, contemporary cache coherence protocols use invalidate schemes that are known to generate a high number of coherence misses. This is especially true under producer-consumer sharing patterns that can become a performance bottleneck as the number of cores increases. This paper presents two mechanisms to design efficient and scalable cache coherence protocols for CMPs. First, we propose an adaptive hybrid protocol to reduce coherence misses observed in write-invalidate based protocols. The proposed protocol is based on a write-invalidate scheme. However, adaptively, it can push updates to potential consumers based on observed producer-consumer sharing patterns. Secondly, we extend this adaptive protocol with an interconnection resource aware mechanism. Experimental evaluations, conducted on a tiled-CMP via full-system simulation, were used to assess the performance from our proposed dynamic hybrid protocols. Performance analysis is presented on a set of scientific applications from the SPLASH-2 and NAS parallel benchmark suites. Results showed that the proposed mechanisms reduce cache-to-cache sharing misses up to 48 % and speed up application performance up to 34 %. In addition, the proposed interconnection resource aware mechanism is proven to perform well under varying interconnection utilizations.

Increasing the effectiveness of directory caches by deactivating coherence for private memory blocks

To meet the demand for more powerful high-performance shared-memory servers, multiprocessor systems must incorporate efficient and scalable cache coherence protocols, such as those based on directory caches. However, the limited directory cache size of the increasingly larger systems may cause frequent evictions of directory entries and, consequently, invalidations of cached blocks, which severely degrades system performance. A significant percentage of the referred memory blocks are only accessed by one processor (even in parallel applications) and, therefore, do not require coherence maintenance. Taking advantage of techniques that dynamically identify those private blocks, we propose to deactivate the coherence protocol for them and to treat them as uniprocessor systems do. The protocol deactivation allows directory caches to omit the tracking of an appreciable quantity of blocks, which reduces their load and increases their effective size. Since the operating system collaborates on the detection of private blocks, our proposal only requires minor modifications. Simulation results show that, thanks to our proposal, directory caches can avoid the tracking of about 57% of the accessed blocks and their capacity can be better exploited. This contributes either to shorten the runtime of parallel applications by 15% while keeping directory cache size or to maintain system performance while using directory caches 8 times smaller.

Elastic SI-Cache: consistent and scalable caching in multi-tier architectures

The new vision of cloud computing demands scalable, available and autonomic software platforms in order to deploy applications and services accessible anywhere and anytime. Multi-tier architectures are an important building block for many applications that are deployed in the cloud. This paper presents a novel caching and replication infrastructure that facilitates the scalable and elastic deployment of multi-tier architectures. Our Elastic SI-Cache is a novel multi-version cache that attains high performance and consistency in multi-tier systems. In contrast to most existing caches, Elastic SI-Cache provides snapshot isolation coherently across all tiers. Furthermore, Elastic SI-Cache supports scalable replication of the different tiers where replicas can be added or removed dynamically as needed, making the cache amenable for cloud computing environments. Elastic SI-Cache has been implemented and integrated into an open source JEE application server and its performance evaluated with the industrial benchmark SPECjAppServer.