Shared Cache Research Articles

Neural Network-based Performance Prediction for Task Migration on S-NUCA Many-Cores

The performance of a task running on a many-core with distributed shared last-level cache (LLC) strongly depends on two parameters: the power budget needed to guarantee thermally-safe operation and the LLC latency. The task's thread-to-core mapping determines both the parameters and needs to make a trade-off because both cannot be simultaneously optimal. Arrival and departure of tasks on a many-core deployed in an open system can change its state significantly in terms of available cores and power budgets. Task migrations can thereupon be used as a tool to keep the many-core operating at peak performance. Furthermore, the relative impacts of power budget and LLC latency on a task's performance may change with its different execution phases mandating its migration on-the-fly. We propose the first run-time algorithm <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PCMig</i> that increases the performance of a many-core with distributed shared LLC by migrating tasks based on their phases and the many-core's state. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PCMig</i> is based on a model that predicts the performance impact of migrations. We propose a performance prediction model based on a lightweight neural network (NN). To serve as a reference, we also propose an analytical model of the many-core that operates on CPI stacks. We demonstrate an NN-based model achieves a higher prediction accuracy at a lower overhead than an analytical model. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PCMig</i> is based on the NN prediction model and results in an up to 7.3 percent increase in performance under a thermal constraint for mixed workloads compared to architecture-aware state-of-the-art (up to 20 percent increase for individual applications). This is achieved with a run-time overhead of less than 0.5 percent.

Linear Network Coded Wireless Caching in Cloud Radio Access Network

This paper investigates a cache-aided cloud radio access network (C-RAN), comprising a central unit, K base stations (BSs) each with N <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</sub> antennas, and M users each with N <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</sub> antennas, where each BS and user have local caches to store some popular contents from the central unit. For this cache-aided network, we propose the linear network coded (NC) wireless caching that consists of linear wireless network coding assisted cache placement phase and signal-space alignment (SSA) enabled content delivery phase. In the cache placement phase, we design a joint NC caching function at the BSs to store linear combinations of messages from the central unit, as a form of linear wireless network coding. In the content delivery phase, we design the SSA pattern based on the NC caching to guide the precoding designs at BSs. Then, each user can reliably decode its requested messages by receiver shaping and reverse NC operation. The primary contribution of this work is to achieve the coding gain induced by the integration of linear wireless network coding and SSA, which has been not exploited in the field of wireless coded caching. In particular, to deal with high temporal variability of user requests, we show that the proposed cache placement is invariant to different user requests in the worst-case caching, without any shared caching messages at different BSs. Furthermore, we verify that the proposed scheme is also compatible with the insufficient caching scenario at the BSs. In addition, we analyze the achievable sum degrees of freedom (DoF) for the proposed caching network. Both analytical and numerical results verify that the proposed caching scheme achieves a higher sum DoF than the existing related works.

Fast distributed compilation and testing of large C++ projects

High energy physics experiments traditionally have large software codebases primarily written in C++ and the LHCb physics software stack is no exception. Compiling from scratch can easily take 5 hours or more for the full stack even on an 8-core VM. In a development workflow, incremental builds often do not significantly speed up compilation because even just a change of the modification time of a widely used header leads to many compiler and linker invokations. Using powerful shared servers is not practical as users have no control and maintenance is an issue. Even though support for building partial checkouts on top of published project versions exists, by far the most practical development workflow involves full project checkouts because of off-the-shelf tool support (git, intellisense, etc.) This paper details a deployment of distcc, a distributed compilation server, on opportunistic resources such as development machines. The best performance operation mode is achieved when preprocessing remotely and profiting from the shared CernVM File System. A 10 (30) fold speedup of elapsed (real) time is achieved when compiling Gaudi, the base of the LHCb stack, when comparing local compilation on a 4 core VM to remote compilation on 80 cores, where the bottleneck becomes non-distributed work such as linking. Compilation results are cached locally using ccache, allowing for even faster rebuilding. A recent distributed memcached-based shared cache is tested as well as a more modern distributed system by Mozilla, sccache, backed by S3 storage. These allow for global sharing of compilation work, which can speed up both central CI builds and local development builds. Finally, we explore remote caching and execution services based on Bazel, and how they apply to Gaudi-based software for distributing not only compilation but also linking and even testing.

Fundamental Limits of Coded Caching With Multiple Antennas, Shared Caches and Uncoded Prefetching

The work explores the fundamental limits of coded caching in the setting where a transmitter with potentially multiple ( $N_{0}$ ) antennas serves different users that are assisted by a smaller number of caches. Under the assumption of uncoded cache placement, the work derives the exact optimal worst-case delay and DoF, for a broad range of user-to-cache association profiles where each such profile describes how many users are helped by each cache. This is achieved by presenting an information-theoretic converse based on index coding that succinctly captures the impact of the user-to-cache association, as well as by presenting a coded caching scheme that optimally adapts to the association profile by exploiting the benefits of encoding across users that share the same cache. The work reveals a powerful interplay between shared caches and multiple senders/antennas, where we can now draw the striking conclusion that, as long as each cache serves at least $N_{0}$ users, adding a single degree of cache-redundancy can yield a DoF increase equal to $N_{0}$ , while at the same time — irrespective of the profile — going from 1 to $N_{0}$ antennas reduces the delivery time by a factor of $N_{0}$ . Finally some conclusions are also drawn for the related problem of coded caching with multiple file requests.

REAL

Shared last level caches (LLC) of multicore systems-on-chip are subject to a significant amount of contention over a limited bandwidth, resulting in major performance bottlenecks that make the issue a first-order concern in modern multiprocessor systems-on-chip. Even though shared cache space partitioning has been extensively studied in the past, the problem of cache bandwidth partitioning has not received sufficient attention. We demonstrate the occurrence of such contention and the resulting impact on the overall system performance. To address the issue, we perform detailed simulations to study the impact of different parameters and propose a novel cache bandwidth partitioning technique, called REAL , that arbitrates among cache access requests originating from different processor cores. It monitors the LLC access patterns to dynamically assign a priority value to each core. Experimental results on different mixes of benchmarks show up to 2.13× overall system speedup over baseline policies, with minimal impact on energy.

Cooperative Inter-Domain Cache Sharing for Information-Centric Networking via a Bargaining Game Approach

Caching popular content locally has been shown to be an efficient way for internet service providers (ISPs) to reduce transit fees and improve the quality of their services. Recently, with the development of information-centric networking, content-peering, which allows peering ISPs to access each other's cache, has attracted interest. However, due to the economic structure and policy routing of the internet, content peering is only applicable between settlement-free peering ISPs, which constraints the benefit of inter-domain cache sharing. In this study, we consider extending the inter-domain cache sharing to ISPs with different tiers; this is done to take full advantage of the caching resources. Specifically, we identify the special properties of the inter-domain cache-sharing market with ISPs of different tiers, analyze the ISPs’ interactions, and employ Nash bargaining solutions to address the resource allocation issues in the market. In addition, we present decentralized algorithms to realize the proposed mechanism for large-scale networks. We show the correctness of the proposed method through rigorous analysis, and we verify the benefits to ISPs by carrying out extensive experiments.

Enhanced Methods of Mobile Cache Sharing and Pre-fetching for Required Web Contents

Availability of content over the web is increasing exponentially. The demand for content by users is also increasing rapidly. The problem of making the right content available to user at the right time will continue to be a crucial issue. As variety of contents are available and variety of users are involved, there is no single way of matching the availability versus need and deliver content instantly especially in a limited mobile environment. Hence a hybrid method is proposed in this paper by combining the different techniques such as caching, pre-fetching and cache sharing with noise reduction to improve the overall performance of mobile for optimal cache memory utilisation, efficient bandwidth utilization, network traffic reduction and latency reduction. Efficiency of mobile caching and pre-fetching is improved using Enhanced Bloom Filter technique and data is shared among cooperative users by establishing a voluntary hub. The unwanted contents in the web page can be considered as noise which is removed when storing the web pages in cache or pre-fetch area. The success of the proposed method greatly depends on the hit ratio of contents rendered locally rather than getting it from server. In order to reduce server hits, sharing the contents of cache and pre-fetch area amongst mobile users is effective. Whenever any user requires new content, even if it is not available in browser cache or local cache of that user, the content can be rendered from the cache or pre-fetch area of collaborative mobile users rather than hitting the server. This hybrid cooperative cache sharing and pre-fetching for accessing the required contents improve the overall performance and hit ratio than the existing methods.

Modeling and Analysis of a Shared Edge Caching System for Connected Cars and Industrial IoT-Based Applications

The next revolution of industrial applications, known as smart industry or Industry 4.0, will rely on Internet of Things (IoT) to automate the monitoring, inspection, and control of industrial equipment and processes. In Industry 4.0, efficient content delivery is one of the fundamental challenges to be addressed. Nowadays, the promising solution for content delivery in smart industrial applications is the use of hierarchical caching systems at the network edge (5G small cells). This approach reduces the delay for content delivery and helps improve the performance of smart industrial applications. However, the caching management is a challenging and complex task, especially in those scenarios of shared storage resources on edge devices to support multiple concurrent applications (e.g., industrial, mobile users, and connected cars applications). In this article, we study the performance of a shared edge caching system for content delivery in smart industry and connected cars applications. To do so, we propose a mathematical framework to model the performance of a hierarchical shared edge caching system. The proposed mathematical framework considers the distinct content catalogs of the different applications (e.g., industrial and connected cars applications) and content request characteristics from industrial IoT devices and vehicles. Numerical results show that the performance of the shared edge caching system is sensitive to vehicular mobility (i.e., vehicular speed).

A Skewed Multi-banked Cache for Many-core Vector Processors

As the number of cores and the memory bandwidth have increased in a balanced fashion, modern vector processors achieve high sustained performances, especially in memory-intensive applications in the fields of science and engineering. However, it is difficult to significantly increase the off-chip memory bandwidth owing to the limitation of the number of input/output pins integrated on a single chip. Under the circumstances, modern vector processors have adopted a shared cache to realize a high sustained memory bandwidth. The shared cache can effectively reduce the pressure to the off-chip memory bandwidth by keeping reusable data that multiple vector cores require. However, as the number of vector cores sharing a cache increases, more different blocks requested from multiple cores simultaneously use the same set. As a result, conflict misses caused by these blocks degrade the performance. In order to avoid increasing the conflict misses in the case of the increasing number of cores, this paper proposes a skewed cache for many-core vector processors. The skewed cache prevents the simultaneously requested blocks from being stored into the same set. This paper discusses how the most important two features of the skewed cache should be implemented in modern vector processors: hashing function and replacement policy. The proposed cache adopts the oddmultiplier displacement hashing for effective skewing and the static re-reference interval prediction policy for reasonable replacing. The evaluation results show that the proposed cache significantly improves the performance of a many-core vector processor by eliminating conflict misses.

Efficient Data Transfer in a Heterogeneous Multicore-Based CE Systems using Cache Performance Optimization

Multiple CPU and GPU cores are integrated together in a typical CE system with shared caches. As the CPU and GPU applications have different characteristics in cache accesses there can be performance penalty. This paper presents a bypass-based shared cache management method to improve such penalty in CE systems. This method dynamically analyzes the cache sensitive features of CPU and GPU application during the execution. It considers the current access characteristics of applications when dealing with the GPU access request, real time to determine whether the GPU application is accessing the memory or accessing the shared cache, through the dynamic adjustment to make it better to adapt to different applications.

An Efficient GPU Cache Architecture for Applications with Irregular Memory Access Patterns

GPUs provide high-bandwidth/low-latency on-chip shared memory and L1 cache to efficiently service a large number of concurrent memory requests. Specifically, concurrent memory requests accessing contiguous memory space are coalesced into warp-wide accesses. To support such large accesses to L1 cache with low latency, the size of L1 cache line is no smaller than that of warp-wide accesses. However, such L1 cache architecture cannot always be efficiently utilized when applications generate many memory requests with irregular access patterns especially due to branch and memory divergences that make requests uncoalesced and small. Furthermore, unlike L1 cache, the shared memory of GPUs is not often used in many applications, which essentially depends on programmers. In this article, we propose Elastic-Cache, which can efficiently support both fine- and coarse-grained L1 cache line management for applications with both regular and irregular memory access patterns to improve the L1 cache efficiency. Specifically, it can store 32- or 64-byte words in non-contiguous memory space to a single 128-byte cache line. Furthermore, it neither requires an extra memory structure nor reduces the capacity of L1 cache for tag storage, since it stores auxiliary tags for fine-grained L1 cache line managements in the shared memory space that is not fully used in many applications. To improve the bandwidth utilization of L1 cache with Elastic-Cache for fine-grained accesses, we further propose Elastic-Plus to issue 32-byte memory requests in parallel, which can reduce the processing latency of memory instructions and improve the throughput of GPUs. Our experiment result shows that Elastic-Cache improves the geometric-mean performance of applications with irregular memory access patterns by 104% without degrading the performance of applications with regular memory access patterns. Elastic-Plus outperforms Elastic-Cache and improves the performance of applications with irregular memory access patterns by 131%.

Time-sensitivity-aware shared cache architecture for multi-core embedded systems

In embedded systems such as automotive systems, multi-core processors are expected to improve performance and reduce manufacturing cost by integrating multiple functions on a single chip. However, inter-core interference in shared last-level cache (LLC) results in increased and unpredictable execution times for time-sensitive tasks (TSTs), which have (soft) timing constraints, thereby increasing the deadline miss rates of such systems. In this paper, we propose a time-sensitivity-aware dead block-based shared LLC architecture to mitigate these problems. First, a time-sensitivity indication bit is added to each cache block, which allows the proposed LLC architecture to be aware of instructions/data belonging to TSTs. Second, portions of the LLC space are allocated to general tasks without interfering with TSTs by developing a time-sensitivity-aware dead block-based cache partitioning technique. Third, to reduce the deadline miss rate of TSTs further, we propose a task matching in shared caches and a cache partitioning scheme that considers the memory access characteristics and the time-sensitivity of tasks (TATS). The TATS is combined with our proposed dead block-based scheme. Our evaluation shows that the proposed schemes reduce deadline miss rates of TSTs compared to conventional shared caches. On a dual-core system, compared to a baseline, equal partitioning, and state-of-the-art quality-of-service-aware cache partitioning, our proposed dead block-based cache partitioning provides 9.3%, 30.5%, and 2.6% lower average deadline miss rates, respectively. On a quad-core system, compared to the baseline, equal partitioning, and state-of-the-art quality-of-service-aware cache partitioning, the combination of our proposed schemes provides 21.2%, 17.7%, and 4.1% lower average deadline miss rates, respectively.

FOS: a low-power cache organization for multicores

The cache hierarchy of current multicore processors typically consists of one or two levels of private caches per core and a large shared last-level cache. This approach incurs area and energy wasting due to oversizing the private cache space, data replication through the inclusive cache levels, as well as the use of highly set-associative caches. In this paper, we claim that although this is the commonly adopted approach, it presents important design issues that can be addressed by a more energy efficient organization. This work proposes Flat On-chip Storage (FOS), a novel cache organization that, aimed at addressing energy and area on low-power processors, resolves the mentioned issues. For this purpose, FOS combines L2 and L3 cache levels into a single one, organized as a flat space, and composed of a pool of private small cache slices. These slices are initially powered off to save energy, and they are powered on and assigned to cores provided that the system performance is expected to improve. To provide fast and uniform access from the private L1 caches to the FOS’s cache slices, multiple architectural challenges are overcome, which entails the design of a custom optical network-on-chip. Experimental results show that FOS achieves significant energy savings on both static and dynamic energy over conventional cache organizations with the same storage capacity. FOS static energy savings are as much as 60% over an electrically connected shared cache; these savings grow up to 75% compared to optically connected baselines. Moreover, despite deactivating part of the cache space, FOS achieves similar performance values as those achieved by conventional approaches.

SCaN-Mob: An opportunistic caching strategy to support producer mobility in named data wireless networking

In recent years, Information-Centric Networking (ICN) architectures have been proposed to deal with many of the existing constraints of the current IP-based host-centric Internet model. The Named Data Networking (NDN) is an example of ICN, which is designed to meet many of the mobility challenges of IP-based networks. However, despite its benefits in supporting mobility of consumers, such architectures still lack efficient solutions for mobile producers, in particular, to improve contents availability during a producer’s handoff. In this paper, we introduce the Shared Caching in Named Data Networking Mobile (SCaN-Mob) strategy. It minimizes the effects of the mobile producers’ unavailability in NDN by creating an interest forwarding strategy and a cache replacement policy for seamless handoff of producer nodes in wireless networks. We proposed two SCaN-Mob approaches to mobile producer support: SCaN-Mob Full that uses all edge caching capacity, and SCaN-Mob AP that uses only the access point’s caching capacity. Simulations results show that SCaN-Mob improves the interest satisfaction rate, while decreases the delivery delay when compared to naive NDN and increases cache hit ratio concerning the other compared solutions.

Efficient File Delivery for Coded Prefetching in Shared Cache Networks With Multiple Requests Per User

We consider a centralized caching network, where a server serves several groups of users, each having a common shared homogeneous fixed-size cache and requesting arbitrary multiple files. An existing coded prefetching scheme is employed where each file is broken into multiple fragments and each cache stores multiple coded packets, each formed by XORing fragments from different files. For such a system, we propose an efficient file delivery scheme by the server to meet the arbitrary multi-requests of all user-groups. Specifically, the stored coded packets of each cache are classified into four types based on the composition of the file fragments encoded. A delivery strategy is developed, which separately delivers a part of each packet type first, and then combinatorially delivers the remaining different packet types in the last stage. The rate, as well as the worst rate of the proposed delivery scheme, are analyzed. We show that our caching model and delivery scheme can incorporate some existing coded caching schemes as special cases. Moreover, for the special case of uniform requests and uncoded prefetching, we make a comparison with existing results, and show that our approach can achieve a lower delivery rate. We also provide numerical results on the delivery rate for the proposed scheme.

Efficient Algorithms for Coded Multicasting in Heterogeneous Caching Networks.

Coded multicasting has been shown to be a promising approach to significantly improve the performance of content delivery networks with multiple caches downstream of a common multicast link. However, the schemes that have been shown to achieve order-optimal performance require content items to be partitioned into several packets that grows exponentially with the number of caches, leading to codes of exponential complexity that jeopardize their promising performance benefits. In this paper, we address this crucial performance-complexity tradeoff in a heterogeneous caching network setting, where edge caches with possibly different storage capacity collect multiple content requests that may follow distinct demand distributions. We extend the asymptotic (in the number of packets per file) analysis of shared link caching networks to heterogeneous network settings, and present novel coded multicast schemes, based on local graph coloring, that exhibit polynomial-time complexity in all the system parameters, while preserving the asymptotically proven multiplicative caching gain even for finite file packetization. We further demonstrate that the packetization order (the number of packets each file is split into) can be traded-off with the number of requests collected by each cache, while preserving the same multiplicative caching gain. Simulation results confirm the superiority of the proposed schemes and illustrate the interesting request aggregation vs. packetization order tradeoff within several practical settings. Our results provide a compelling step towards the practical achievability of the promising multiplicative caching gain in next generation access networks.

Dynamic directory table with victim cache: on-demand allocation of directory entries for active shared cache blocks

In this paper, we present a novel directory architecture that can dynamically allocate a directory entry for a cache block on demand at runtime only when the block is shared by more than a single core. Thus, we do not maintain coherence for private blocks, substantially reducing the number of directory entries. Even for shared blocks, we allocate directory entry dynamically only when the block is actively shared, further reducing the number of directory entries at runtime. For this, we propose a new directory architecture called dynamic directory table (DDT), which is a decoupled directory storage from the shared cache and dynamically maintains directory entries only for actively shared blocks. Also, we add a small additional victim cache to its original DDT in order to reduce invalidation broadcasts caused by DDT eviction. Through our detailed simulation on PARSEC benchmarks, we show that DDT can outperform the expensive full-map directory by a slight margin with only 16.09% of directory area across a variety of different workloads. This is achieved by its faster access and high hit rates in the small directory. In addition, we demonstrate that even smaller DDTs can give comparable or higher performance compared to recent directory optimization schemes such as SPACE and DGD with considerably less area.

A Gaussian Set Sampling Model for Efficient Shared Cache Profiling on Multi-Cores

The last level cache (LLC) has significant impact to system performance on modern multi-core processors. But as cache sizes reach several megabytes and more, the overhead of exploring performance on LLC greatly increases as well. To improve the efficiency of performance analysis, we propose a set-sampling-based cache profiling model for the performance analysis on multi-core LLC. We first explore the memory access distributions on LLC by developing a low-overhead stress-application-based method. The results show that memory access distributions can be approximated by Gaussian distribution function. Based on this observation, a Gaussian-distribution-based set sampling model is proposed which can predict program performance with limited representative samples. We evaluate our model on a contemporary multi-core machine and show that 1) the proposed method can precisely predict program performance on LLC under different contention intensities and 2) our method can achieve similar precision with less samples compared to widely adopted set sampling methods such as the random sampling and the continuous address sampling.

A Low-power Shared Cache Design with Modified PID Controller for Efficient Multicore Embedded Systems

A Low-power Shared Cache Design with Modified PID Controller for Efficient Multicore Embedded Systems

Filter router: An enhanced router design for efficient stacked shared cache network

Filter router: An enhanced router design for efficient stacked shared cache network