Cache Architecture Research Articles

On Mobile Edge Caching

With the widespread adoption of various mobile applications, the amount of traffic in wireless networks is growing at an exponential rate, which exerts a great burden on mobile core networks and backhaul links. Mobile edge caching, which enables mobile edges with cache storages, is a promising solution to alleviate this problem. In this paper, we aim to review the state-of-the-art of mobile edge caching. We first present an overview of mobile edge caching and its advantages. We then discuss the locations where mobile edge caching can be realized in the network. We also analyze different caching criteria and their respective effects on the caching performances. Moreover, we compare several caching schemes and discuss their pros and cons. We further present a detailed and in-depth discussion on the caching process, which can be delineated into four phases including content request, exploration, delivery, and update. For each phase, we identify different issues and review related works in addressing these issues. Finally, we present a number of challenges faced by current mobile edge caching architectures and techniques for further studies.

Upper bounds for multi-level multi-server paging

The distributed multi-level multi-server paging problem (DMLMSP) defined in this paper extends and generalizes the classical distributed paging problem [3] to a distributed concurrent multi-level setting, in which multiple servers share caches at multiple levels. The DMLMSP can be used for modeling algorithms for efficient distributed storage systems, in which multiple servers use caches for accelerating access to a centralized storage, maintaining cache coherency across multiple nodes while minimizing access latency and optimizing cache-hit ratio. The DMLMSP model fits basic principles of Non-Uniform Cache Architectures (NUCAs) and can be used for analyzing multiple-level centralized storage. We present an optimal offline algorithm for the DMLMSP model, with minimum number of page faults and minimum makespan, whose time complexity is polynomial in the length of the servers' page request sequences. The new algorithm generalizes and simplifies the state of the art algorithms [3,1].

Performance improvement by Software controlled Cache Architecture

Performance improvement by Software controlled Cache Architecture

Reuse-Distance-Aware Write-Intensity Prediction of Dataless Entries for Energy-Efficient Hybrid Caches

Emerging nonvolatile memory technologies act as a prominent choice for the larger on-chip caches on account of high density, good scalability, and low static power consumption. However, costly write operations reduce their possibility as a successor of SRAM. To mitigate this problem, a spin-transfer torque random-access memory (STT-RAM)-SRAM hybrid cache architecture is proposed. In such cache architectures, allocation of a write-intensive block is the key challenge for energy efficiency. This paper presents a data allocation policy that reduces the number of writes and energy consumption of the STT-RAM region in the last-level cache by considering the existence of private blocks. Dataless entries are allocated in STT region for such private blocks, and actual data is written only when the block is written back from L1. Heavily written blocks are subsequently migrated to SRAM region. We also present a predictor that helps to redirect the write backs from L1 of dataless entries directly to SRAM region, depending on the predicted reuse-distance-aware write intensity. Experimental evaluation shows that this technique reduces the energy consumption by 34.3% (19.6%) and 23.3% (14.1%), respectively, over two existing techniques in the case of dual (quad) core system.

TriZone: A Design of MLC STT-RAM Cache for Combined Performance, Energy, and Reliability Optimizations

Spin-transfer torque random access memory (STT-RAM) is a promising technology for future nonvolatile caches and memories. To increase the storage density, multilevel cell (MLC) technique was recently introduced to STT-RAM designs at the cost of degraded access speed, reliability, and energy efficiency. Existing MLC STT-RAM cache architectures primarily focus on the performance and energy optimizations but ignore the crucial demand for reliability. In this paper, we propose “TriZone”—a holistic design scheme for MLC STT-RAM cache to simultaneously meet the requirements of performance, energy, and reliability. Three cache block configurations, namely hard, soft, and mixed, are constructed with the hard-bit, soft-bit, and both hard-bit and soft-bit of MLC STT-RAM, respectively. By observing the difference of these cache blocks, a nonuniform strength ECC (NUS-ECC) is developed to guarantee the operational reliability of a cache block with a variable decoding delay adapting to the needs of error correction (e.g., the number of the erroneous bits). The whole MLC STT-RAM cache is then partitioned into three regions, each of which is composed of different cache blocks. In order to achieve the best tradeoff among performance, energy, and reliability, we then introduce the dynamic cache partitioning to determine the partition of this tri-way MLC STT-RAM cache according to the runtime characteristic of various applications. Experiment results show that compared with conventional performance-driven MLC STT-RAM cache design with pessimistic ECC, TriZone can improve the system performance and energy by averagely 11.7% (10.0%) and 13.3% (15.7%), respectively, for single-threaded (multiprogram) applications. The additional area overhead associated with NUS-ECC is limited by ~ 3%.

ARMOR: A Recompilation and Instrumentation-Free Monitoring Architecture for Detecting Memory Exploits

Software written in programming languages that permit manual memory management, such as C and C++, are often littered with exploitable memory errors. These memory bugs enable attackers to leak sensitive information, hijack program control flow, or otherwise compromise the system and are a critical concern for computer security. Many runtime monitoring and protection approaches have been proposed to detect memory errors in C and C++ applications, however, they require source code recompilation or binary instrumentation, creating compatibility challenges for applications using proprietary or closed source code, libraries, or plug-ins. This paper introduces a new approach for detecting heap memory errors that does not require applications to be recompiled or instrumented. We show how to leverage the calling convention of a processor to track all dynamic memory allocations made by an application during runtime. We also present a transparent tracking and caching architecture to efficiently verify program heap memory accesses. Performance simulations of our architecture using SPEC benchmarks and real-world application workloads show our architecture achieves hit rates over 95 percent for a 256-entry cache, resulting in only 2.9 percent runtime overhead. Security analysis using a software prototype shows our architecture detects 98 percent of heap memory errors from selected test cases in the Juliet Test Suite and real-world exploits.

Performance and Power-Efficient Design of Dense Non-Volatile Cache in CMPs

In this paper, we present a novel cache design based on Multi-Level Cell Spin-Transfer Torque RAM (MLC STT-RAM) that can dynamically adjust the set capacity and associativity to efficiently use the full potential of MLC STT-RAM technology. We exploit the asymmetric nature of the MLC storage scheme to build cache lines featuring heterogeneous performances, that is, half of the cache lines are read-friendly, while the other half are write-friendly. Furthermore, we propose to opportunistically deactivate cache ways in underutilized sets to convert MLC to Single-Level Cell (SLC) mode, which features overall better performance and lifetime. Our ultimate goal is to build a cache architecture that combines the capacity advantages of MLC and performance/energy advantages of SLC. Our experimental evaluations show an average improvement of 43 percent in total numbers of conflict misses, 27 percent in memory access latency, 12 percent in system performance, and 26 percent in L3 access energy, with a slight degradation in lifetime (about 7 percent) compared to an SLC cache.

ReCA: An Efficient Reconfigurable Cache Architecture for Storage Systems with Online Workload Characterization

In recent years, SSDs have gained tremendous attention in computing and storage systems due to significant performance improvement over HDDs. The cost per capacity of SSDs, however, prevents them from entirely replacing HDDs in such systems. One approach to effectively take advantage of SSDs is to use them as a caching layer to store performance critical data blocks to reduce the number of accesses to disk subsystem. Due to characteristics of Flash-based SSDs such as limited write endurance and long latency on write operations, employing caching algorithms at the Operating System (OS) level necessitates to take such characteristics into consideration. Previous caching techniques are optimized towards only one type of application, which affects both generality and applicability. In addition, they are not adaptive when the workload pattern changes over time. This paper presents an efficient Reconfigurable Cache Architecture (ReCA) for storage systems using a comprehensive workload characterization to find an optimal cache configuration for I/O intensive applications. For this purpose, we first investigate various types of I/O workloads and classify them into five major classes. Based on this characterization, an optimal cache configuration is presented for each class of workloads. Then, using the main features of each class, we continuously monitor the characteristics of an application during system runtime and the cache organization is reconfigured if the application changes from one class to another class of workloads. The cache reconfiguration is done online and workload classes can be extended to emerging I/O workloads in order to maintain its efficiency with the characteristics of I/O requests. Experimental results obtained by implementing ReCA in a server running Linux show that the proposed architecture improves performance and lifetime up to 24\% and 33\%, respectively.

Proactive caching for edge computing-enabled industrial mobile wireless networks

As manufacturing systems shift from automated patterns to smart frameworks such as smart factories in Industry 4.0, industrial wireless networks (IWNs) are serving as promising communication systems that can be applied to the manufacturing field. When the mobile elements and static nodes are introduced into the system, large amounts of data downloaded from mobile networks or tele-servers can be one of the greatest challenges for industrial mobile wireless networks (IMWNs). Mobility and industrial properties have rarely been considered by previous research on download strategies and caching methods. In this paper, we present a three-layer cache architecture based on edge computing and other heritage traditional networks. Then, useful spatial and temporal mobility properties are mapped using different groups and edge computing servers that contain mobile nodes. Then, according to the sojourn time, the capacity of edge computing servers and other neighbouring nodes, we propose a proactive caching strategy for large amounts of data downloaded by mobile networks that considers location and mobile trajectories. Moreover, the superiority of our proposed scheme is demonstrated by comparison case studies of widely used classical schemes. The numerical results show that our proposed strategy achieves higher goodput and real-time and other performance.

Cooperative Content Caching in 5G Networks with Mobile Edge Computing

Along with modern wireless networks being content-centric, the demand for rich multimedia services has been growing at a tremendous pace, which brings significant challenges to mobile networks in terms of the need for massive content delivery. Edge caching has emerged as a promising approach to alleviate the heavy burden on data transmission through caching and forwarding contents at the edge of networks. However, existing studies always treat storage and computing resources separately, and neglect the mobility characteristic of both the content caching nodes and end users. Driven by these issues, in this work, we propose a new cooperative edge caching architecture for 5G networks, where mobile edge computing resources are utilized for enhancing edge caching capability. In the architecture, we focus on mobility-aware hierarchical caching, where smart vehicles are taken as collaborative caching agents for sharing content cache tasks with base stations. To further utilize the caching resource of smart vehicles, we introduce a new vehicular caching cloud concept, and propose a vehicle-aided edge caching scheme, where the caching and computing resources at the wireless network edge are jointly scheduled. Numerical results indicate that the proposed scheme minimizes content access latency and improves caching resource utilization.

Performance and Energy-Efficient Design of STT-RAM Last-Level Cache

Recent research has proposed having a die-stacked last-level cache (LLC) to overcome the memory wall. Lately, spin-transfer-torque random access memory (STT-RAM) caches have received attention, since they provide improved energy efficiency compared with DRAM caches. However, recently proposed STT-RAM cache architectures unnecessarily dissipate energy by fetching unneeded cache lines (CLs) into the row buffer (RB). In this paper, we propose a selective read policy for the STT-RAM which fetches those CLs into the RB that are likely to be reused. In addition, we propose a tags-update policy that reduces the number of STT-RAM writebacks. This reduces the number of reads/writes and thereby decreases the energy consumption. To reduce the latency penalty of our selective read policy, we propose the following performance optimizations: 1) an RB tags-bypass policy that reduces STT-RAM access latency; 2) an LLC data cache that stores the CLs that are likely to be used in the near future; 3) an address organization scheme that simultaneously reduces LLC access latency and miss rate; and 4) a tags-to-column mapping policy that improves access parallelism. For evaluation, we implement our proposed architecture in the Zesto simulator and run different combinations of SPEC2006 benchmarks on an eight-core system. We compare our approach with a recently proposed STT-RAM LLC with subarray parallelism support and show that our synergistic policies reduce the average LLC dynamic energy consumption by 75% and improve the system performance by 6.5%. Compared with the state-of-the-art DRAM LLC with subarray parallelism, our architecture reduces the LLC dynamic energy consumption by 82% and improves system performance by 6.8%.

Combining Software Cache Partitioning and Loop Tiling for Effective Shared Cache Management

One of the biggest challenges in multicore platforms is shared cache management, especially for data-dominant applications. Two commonly used approaches for increasing shared cache utilization are cache partitioning and loop tiling. However, state-of-the-art compilers lack efficient cache partitioning and loop tiling methods for two reasons. First, cache partitioning and loop tiling are strongly coupled together, and thus addressing them separately is simply not effective. Second, cache partitioning and loop tiling must be tailored to the target shared cache architecture details and the memory characteristics of the corunning workloads. To the best of our knowledge, this is the first time that a methodology provides (1) a theoretical foundation in the above-mentioned cache management mechanisms and (2) a unified framework to orchestrate these two mechanisms in tandem (not separately). Our approach manages to lower the number of main memory accesses by an order of magnitude keeping at the same time the number of arithmetic/addressing instructions to a minimal level. We motivate this work by showcasing that cache partitioning, loop tiling, data array layouts, shared cache architecture details (i.e., cache size and associativity), and the memory reuse patterns of the executing tasks must be addressed together as one problem, when a (near)-optimal solution is requested. To this end, we present a search space exploration analysis where our proposal is able to offer a vast deduction in the required search space.

SABR

State-of-the-art software-defined wide area networks (SD-WANs) provide the foundation for flexible and highly resilient networking. In this work, we design, implement, and evaluate a novel architecture (denoted as SABR) that leverages the benefits of software-defined networking (SDN) to provide network-assisted adaptive bitrate streaming. With clients retaining full control of their streaming algorithms, we clearly show that by this network assistance, both the clients and the content providers benefit significantly in terms of quality of experience (QoE) and content origin offloading. SABR utilizes information on available bandwidths per link and network cache contents to guide video streaming clients with the goal of improving the viewer’s QoE. In addition, SABR uses SDN capabilities to dynamically program flows to optimize the utilization of content delivery network caches. Backed by our study of SDN-assisted streaming, we discuss the change in the requirements for network-to-player APIs that enables flexible video streaming. We illustrate the difficulty of the problem and the impact of SDN-assisted streaming on QoE metrics using various well-established player algorithms. We evaluate SABR together with state-of-the-art dynamic adaptive streaming over HTTP (DASH) quality adaptation algorithms through a series of experiments performed on a real-world, SDN-enabled testbed network with minimal modifications to an existing DASH client. In addition, we compare the performance of different caching strategies in combination with SABR. Our trace-based measurements show the substantial improvement in cache hit rates and QoE metrics in conjunction with SABR indicating a rich design space for jointly optimized SDN-assisted caching architectures for adaptive bitrate video streaming applications.

The Quest for Energy-Efficient I$ Design in Ultra-Low-Power Clustered Many-Cores

High performance and extreme energy efficiency are strong requirements for a fast-growing number of edge-node Internet of Things (IoT) applications. While traditional Ultra-Low-Power designs rely on single-core micro-controllers (MCU), a new generation of architectures leveraging fully programmable tightly-coupled clusters of near-threshold processors is emerging, joining the performance gain of parallel execution over multiple cores with the energy efficiency of low-voltage operation. In this work, we tackle one of the most critical energy-efficiency bottlenecks for these architectures: instruction memory hierarchy. Exploiting the instruction locality typical of data-parallel applications, we explore two different shared instruction cache architectures, based on energy-efficient latch-based memory banks: one leveraging a crossbar between processors and single-port banks (SP), and one leveraging banks with multiple read ports (MP). We evaluate the proposed architectures on a set of signal processing applications with different executable sizes and working-sets. The results show that the shared cache architectures are able to efficiently execute a much wider set of applications (including those featuring large memory footprint and irregular access patterns) with a much smaller area and with much better energy efficiency with respect to the private cache. The multi-port cache is suitable for sizes up to a few kB, improving performance by up to 40 percent, energy efficiency by up to 20 percent, and energy $\times$ area efficiency by up to 30 percent with respect to the private cache. The single-port solution is more suitable for larger cache sizes (up to 16 kB), providing up to 20 percent better energy $\times$ area efficiency than the multi-port, and up to 30 percent better energy efficiency than private cache.

An energy-efficient 3D-stacked STT-RAM cache architecture for cloud processors: the effect on emerging scale-out workloads

This paper focuses on energy consumption which is a major problem in the dark silicon era. As energy consumption becomes a key issue for operation and maintenance of cloud data centers, cloud computing providers are becoming significantly concerned. Here, we show how spin-transfer torque random access memory (STT-RAM) can be used as an on-chip L2 cache to obtain lower energy compared to conventional L2 caches, like SRAM. High density, fast read access and non-volatility make STT-RAM a significant technology for on-chip memories. Previous studies have mainly studied specific schemes based on common applications and do not provide a thorough analysis of emerging scale-out applications with multiple design options. Here, we discuss different outlooks consisting of performance and energy efficiency in cloud processors by running emerging scale-out workloads. Experiment results on the CloudSuite benchmarks show that the proposed method reduces energy by 51% (on average) and improves energy delay product by 37% (on average) where instruction per cycle degradation is only 22% (on average) compared to the SRAM method.

A Duplication-Aware SSD-Based Cache Architecture for Primary Storage in Virtualization Environment

With the advantages of low latency, high performance, and low power consumption, solid state drives (SSDs) have been widely deployed as the cache layer between memory and back-end low-speed storage devices to narrow the performance gap between CPU and storage system. However, in virtualization environment, the high integration of virtual machines can introduce a lot of duplicate data blocks in the cache device. Existing cache architectures and replacement algorithms rarely take this situation into consideration. This greatly limits the efficient use of the cache device. For this case, we proposed a duplication-aware SSD-based cache architecture. In this architecture, duplicate data blocks can be significantly reduced, and the utilization efficiency of the cache device will be notably improved. Furthermore, to reduce the cache replacement overhead, we also proposed an improved adaptive replacement cache (ARC)-based replacement strategy, which we named D-ARC. Experiment results show that, in some situations, our cache architecture can improve the cache hit ratio by five times, reduce the average I/O latency by 63%, and eliminate SDD writes by 81%. Compared with the ARC-based replacement strategy, D-ARC can provide a performance improvement by about 16% if the configuration is set appropriately.

A Novel Architecture of Large Hybrid Cache With Reduced Energy

Energy becomes an inevitable challenge when using a large die-stacking dynamic random access memory (DRAM) cache. Although emerging spin-transfer-torque-RAM (STT-RAM) technology can efficiently reduce the static energy of large cache, it cannot completely replace DRAM cache due to the high write energy of STT-RAM. Recently, researchers have observed that there are many redundant bits written in the row buffer and futile bits written back to STT-RAM cells, which do not change the cells’ value but still cost high write energy. In this paper, we first design a large hybrid cache architecture with the DRAM region and the STT-RAM region to reduce the high static energy of DRAM cache. The selective write back to row buffer and selective write back to cell array optimizations are proposed to reduce high write energy of the STT-RAM region by removing the unnecessary bit-writes. Furthermore, we propose reuse distance-oriented data movement to reduce write operations in the STT-RAM region. Finally, we propose a novel tag design for the hybrid cache by moving all tag arrays to the STT-RAM region. The SPEC CPU2006 benchmarks show an average 28.3% energy reduction and 6.7% performance improvement for the write optimizations and 7.3% energy savings and 27.5% instructions per cycle speedups for the proposed tag design.

Access-Time-Aware Cache Algorithms

Most of the caching algorithms are oblivious to requests’ timescale, but caching systems are capacity constrained and, in practical cases, the hit rate may be limited by the cache’s impossibility to serve requests fast enough. In particular, the hard-disk access time can be the key factor capping cache performance. In this article, we present a new cache replacement policy that takes advantage of a hierarchical caching architecture, and in particular of access-time difference between memory and disk. Our policy is optimal when requests follow the independent reference model and significantly reduces the hard-disk load, as shown also by our realistic, trace-driven evaluation. Moreover, we show that our policy can be considered in a more general context, since it can be easily adapted to minimize any retrieval cost, as far as costs add over cache misses.

Locality-aware task scheduling for homogeneous parallel computing systems

In systems with complex many-core cache hierarchy, exploiting data locality can significantly reduce execution time and energy consumption of parallel applications. Locality can be exploited at various hardware and software layers. For instance, by implementing private and shared caches in a multi-level fashion, recent hardware designs are already optimised for locality. However, this would all be useless if the software scheduling does not cast the execution in a manner that promotes locality available in the programs themselves. Since programs for parallel systems consist of tasks executed simultaneously, task scheduling becomes crucial for the performance in multi-level cache architectures. This paper presents a heuristic algorithm for homogeneous multi-core systems called locality-aware task scheduling (LeTS). The LeTS heuristic is a work-conserving algorithm that takes into account both locality and load balancing in order to reduce the execution time of target applications. The working principle of LeTS is based on two distinctive phases, namely; working task group formation phase (WTG-FP) and working task group ordering phase (WTG-OP). The WTG-FP forms groups of tasks in order to capture data reuse across tasks while the WTG-OP determines an optimal order of execution for task groups that minimizes the reuse distance of shared data between tasks. We have performed experiments using randomly generated task graphs by varying three major performance parameters, namely: (1) communication to computation ratio (CCR) between 0.1 and 1.0, (2) application size, i.e., task graphs comprising of 50-, 100-, and 300-tasks per graph, and (3) number of cores with 2-, 4-, 8-, and 16-cores execution scenarios. We have also performed experiments using selected real-world applications. The LeTS heuristic reduces overall execution time of applications by exploiting inter-task data locality. Results show that LeTS outperforms state-of-the-art algorithms in amortizing inter-task communication cost.

NVM Way Allocation Scheme to Reduce NVM Writes for Hybrid Cache Architecture in Chip-Multiprocessors

Hybrid cache architectures (HCAs) containing both SRAM and non-volatile memory (NVM) have been proposed to overcome the disadvantages of NVM-based cache architecture. Most previous works have concentrated on managing write-intensive blocks by storing these blocks to SRAM to reduce the number of the write operations to NVM. However, they have not focused on reducing linefill operations which also occupy a large portion of overall NVM write counts in chip-multiprocessor (CMP) environments. This paper proposes an NVM way allocation scheme, taking into account the NVM linefill counts as well as cache miss rate and the NVM write hit counts during victim selection. Three metrics are introduced to estimate the effectiveness of NVM way allocation: Miss counts change (ΔM), write counts change (ΔW), and NVM write counts change (ΔNVMW). An algorithm to minimize the write counts of NVM based on these metrics is proposed as well. Our experimental results show that dynamic energy consumption is reduced by 37.5 percent on average.