Last-level Cache Research Articles

Timing-aware analysis of shared cache interference for non-preemptive scheduling

AbstractIn multi-core architectures, the last-level cache (LLC) is often shared between cores. Sharing the LLC leads to inter-core interference, which impacts system performance and predictability. This means that tasks running in parallel on different cores may experience additional LLC misses as they compete for cache space. To compute a task’s worst-case execution time (WCET), a safe bound on the inter-core cache interference has to be determined. We propose an interference analysis for set-associative shared least-recently-used caches. The analysis leverages timing information to establish tight bounds on the worst-case interference and classifies individual accesses as either cache hits or potential cache misses. We evaluated the analysis performance for systems containing 2 and 4 cores using shared caches up to 64 KB. The evaluation shows an average WCET reduction of up to 23.3% for dual-core systems and 8.5% for quad-core systems.

High Performance and Predictable Shared Last-level Cache for Safety-Critical Systems

High Performance and Predictable Shared Last-level Cache for Safety-Critical Systems

Thermal optimization of two-terminal SOT-MRAM

While magnetoresistive random-access memory (MRAM) stands out as a leading candidate for embedded nonvolatile memory and last-level cache applications, its endurance is compromised by substantial self-heating due to the high programming current density. The effect of self-heating on the endurance of the magnetic tunnel junction (MTJ) has primarily been studied in spin-transfer torque (STT)-MRAM. Here, we analyze the transient temperature response of two-terminal spin–orbit torque (SOT)-MRAM with a 1 ns switching current pulse using electro-thermal simulations. We estimate a peak temperature range of 350–450 °C in 40 nm diameter MTJs, underscoring the critical need for thermal management to improve endurance. We suggest several thermal engineering strategies to reduce the peak temperature by up to 120 °C in such devices, which could improve their endurance by at least a factor of 1000× at 0.75 V operating voltage. These results suggest that two-terminal SOT-MRAM could significantly outperform conventional STT-MRAM in terms of endurance, substantially benefiting from thermal engineering. These insights are pivotal for thermal optimization strategies in the development of MRAM technologies.

Enhancing Lifetime and Performance of MLC NVM Caches Using Embedded Trace Buffers

Large volumes of on-chip and off-chip memory are required by contemporary applications. Emerging non-volatile memory technologies including STT-RAM, PCM, and ReRAM are becoming popular for on-chip and off-chip memories as a result of their desirable properties. Compared to traditional memory technologies such as SRAM and DRAM, they have minimal leakage current and high packing density. Non Volatile Memories (NVM), however, have a low write endurance, a high write latency, and high write energy. Non-volatile Single Level Cell (SLC) memories can store a single bit of data in each memory cell, whereas Multi Level Cells (MLC) can store two or more bits in each memory cell. Although MLC NVMs have substantially higher packing density than SLCs, their lifetime and access speed are key concerns. For a given cache size, MLC caches consume 1.84× less space and 2.62× less leakage power than SLC caches. We propose Trace buffer Assisted Non-volatile Memory Cache (TANC), an approach that increases the lifespan and performance of MLC-based last-level caches using the underutilized Embedded Trace Buffers (ETB). TANC improves the lifetime of MLC LLCs up to 4.36× and decreases average memory access time by 4% compared to SLC NVM LLCs and by 6.41× and 11%, respectively, compared to baseline MLC LLCs.

SGXFault: An Efficient Page Fault Handling Mechanism for SGX Enclaves

Trusted Execution Environments (TEEs) such as Intel SGX are becoming a commonplace for the security in server processors. Intel SGX provides the guarantees of confidentiality, integrity and replay attack protection for a specific region of memory. However, the downside of it is the performance bottleneck due to the context switching overhead and the page faults for the larger memory footprint applications. In this paper, we propose a scheme <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SGXFault</i> which ensures for major part of the execution, the pages are available in the secure memory when needed. We do this decision at the last level cache (LLC) by locking the blocks of the frequent pages to the LLC using Cache Lockdown mechanism. In addition to this, we do a page-level prediction based prefetching when there is a miss in the LLC. Using the combination of both the approaches, we are able to outperform baseline Intel SGX and recent competing scheme by around 18.6% and 17.8% respectively.

Designing a Deep Neural Network engine for LLC block reuse prediction to mitigate Soft Error in Multicore

Last level cache (LLC), a major contender of chip area, exhibits the highest sensitivity to soft error. Block reuse prediction is used to exhibit selective protection of LLC blocks. However, traditional reuse predictors fail to accurately understand the cache access pattern. This work explores machine learning for block reuse prediction that realizes the features that affect reuse likelihood and therefore, better predicts the reusable blocks. To off-load the reuse calculation from the processing cores, a dedicated reuse predictor engine is designed. This engine implements a deep neural network with unsupervised learning and calculates reuse probabilities offline without affecting the execution path. Priority blocks with high reuse likelihood are protected using replications in shared LLC. Low priority blocks are invalidated to reduce their vulnerable time. A replication aware replacement policy is also developed to protect the replicated/reserved blocks from eviction. The scheme is evaluated in Multi2Sim 5.0 simulation framework with SPEC-CPU benchmarks considering bulk-CMOS, FDSOI and FinFET technologies. The results indicate 41.29% and 43.14% reductions in miss rate, 92.29% and 89.56% reductions in vulnerability, 5.73% and 6.31% increase in write-back rate, 17.55% and 16.27% reductions in average memory access time, 2.17% and 1.54% increase in Instructions-per-cycle (IPC) and 1.31% and 0.77% reductions in normalized-execution-cycle over the baseline (with SECDED protection) for integer and floating point benchmarks respectively. Marginal increase in power consumption by 1.33% and dynamic energy by 0.43% are observed at 6.32% area overhead.

Characterizing Multi-Chip GPU Data Sharing

Multi-chip Graphics Processing Unit (GPU) systems are critical to scale performance beyond a single GPU chip for a wide variety of important emerging applications. A key challenge for multi-chip GPUs, though, is how to overcome the bandwidth gap between inter-chip and intra-chip communication. Accesses to shared data, i.e., data accessed by multiple chips, pose a major performance challenge as they incur remote memory accesses possibly congesting the inter-chip links and degrading overall system performance. This article characterizes the shared dataset in multi-chip GPUs in terms of (1) truly versus falsely shared data, (2) how the shared dataset scales with input size, (3) along which dimensions the shared dataset scales, and (4) how sensitive the shared dataset is with respect to the input’s characteristics, i.e., node degree and connectivity in graph workloads. We observe significant variety in scaling behavior across workloads: some workloads feature a shared dataset that scales linearly with input size, whereas others feature sublinear scaling (following a \(\sqrt {2}\) or \(\sqrt [3]{2}\) relationship). We further demonstrate how the shared dataset affects the optimum last-level cache organization (memory-side versus SM-side) in multi-chip GPUs, as well as optimum memory page allocation and thread scheduling policy. Sensitivity analyses demonstrate the insights across the broad design space.

PP-Bridge: Establishing a Bridge between the Prefetching and Cache Partitioning

— Modern computer processors are equipped with multiple cores, each boasting its own dedicated cache memory, while collectively sharing a generously sized Last Level Cache (LLC). To ensure equitable utilization of the LLC space and bolster system security, partitioning techniques have been introduced to allocate the shared LLC space among the applications running on different cores. This partition dynamically adapts to the requirements of these applications. Prefetching plays a vital role in enhancing cache performance by proactively loading data into the cache before it get requested explicitly by a core. Each core employs prefetch engines to decide which data blocks to fetch preemptively. However, a haphazard prefetcher may bring in more data blocks than necessary, leading to cache pollution and a subsequent degradation in system performance. To maximize the benefits of prefetching, it is essential to keep cache pollution to a minimum. Intriguingly, our research has uncovered that when existing prefetching techniques are combined with partitioning methods, they tend to exacerbate cache pollution within the LLC, resulting in a noticeable decline in system performance. In this paper, we present a novel approach aimed at mitigating cache pollution when combining prefetching with partitioning techniques.

TB-TBP: a task-based adaptive routing algorithm for network-on-chip in heterogenous CPU-GPU architectures

With the rapid development of heterogeneous network-on-chip (NoC), a vast amount of shared resources are integrated into NoC. Intense resource competition exists between CPUs and GPUs, leading to congestion and a decrease in overall network performance. Reasonable node placement can minimize network conflicts at the topology level. This paper first discusses the placement of shared last-level cache and memory controller, then selects a more rational placement method and optimizes the path. To solve the hot spots problem in center placement method, a task-based routing algorithm is designed to plan the path. Simulation results demonstrate that, compared to the traditional routing algorithm, the overall network latency is reduced by 9%, and the CPU performance is improved by 13.6%. Furthermore, a dynamic task-based routing algorithm is proposed. Compared to the static task routing algorithm, the overall network latency is reduced by 2.08%, and the CPU performance is improved by 4.09%.

ZPP: A Dynamic Technique to Eliminate Cache Pollution in NoC based MPSoCs

Data prefetching efficiently reduces the memory access latency in NUCA architectures as the Last Level Cache (LLC) is shared and distributed across multiple cores. But cache pollution generated by prefetcher reduces its efficiency by causing contention for shared resources such as LLC and the underlying network. The paper proposes Zero Pollution Prefetcher (ZPP) that eliminates cache pollution for NUCA architecture. For this purpose, ZPP uses L1 prefetcher and places the prefetched blocks in the data locations of LLC where modified blocks are stored. Since modified blocks in LLC are stale and request for such blocks are served from the exclusively owned private cache, their space unnecessary consumes power to maintain such stale data in the cache. The benefits of ZPP are (a) Eliminates cache pollution in L1 and LLC by storing prefetched blocks in LLC locations where stale blocks are stored. (b) Insufficient cache space is solved by placing prefetched blocks in LLC as LLCs are larger in size than L1 cache. This helps in prefetching more cache blocks, thereby increasing prefetch aggressiveness. (c) Increasing prefetch aggressiveness increases its coverage. (d) It also maintains an equivalent lookup latency to L1 cache for prefetched blocks. Experimentally it has been found that ZPP increases weighted speedup by 2.19x as compared to a system with no prefetching while prefetch coverage and prefetch accuracy increases by 50%, and 12%, respectively compared to the baseline. 1

An edge re-ordering based acceleration architecture for improving data locality in graph analytics applications

Data structure is the key in Edge Computing where various types of data are continuously generated by ubiquitous devices. Within all common data structures, graphs are used to express relationships and dependencies among human identities, objects, and locations. They are also expected to become one of the most important data infrastructures in the near future. Furthermore, as graph processing often requires random accesses to vast memory spaces, conventional memory hierarchies with caches cannot work efficiently. To alleviate such memory access bottlenecks in graph processing, we present a solution through vertex accesses scheduling and edge array re-ordering, in parallel with the execution of graph processing applications to improve both temporal and spatial locality of memory accesses, especially for edge-centric graph analytics which are popular means in handling dynamic graphs. Our proposed architecture is evaluated and tested through both trace-based cache simulations and cycle-accurate FPGA-based prototyping. Evaluation results show that our proposal has a potential of significantly reducing the Last Level Cache (LLC) misses by 62.60% in general among PageRank and BFS algorithms. Meanwhile, evaluations with the FPGA prototype successfully reduce the quantity of Miss-Per-Kilo-Instructions (MPKI) for LLC by 56.27% on average.

NFVPermit: Toward Ensuring Performance Isolation in NFV-Based Systems

Network Functions Virtualization (NFV) promises programmability and cost savings by replacing hardware middleboxes with more flexible Virtual Network Functions (VNFs) on commodity servers. But, the current virtualization technologies do not fully isolate the system resources like Last Level Cache (LLC) and Memory Bandwidth (MB); therefore, co-location of VNFs on the same commodity server causes interference effects which might severely impact the performance of VNFs in terms of throughput, latency, etc. Contention at LLC is one of the root causes of this performance degradation and it is addressed by LLC resource partitioning. But, it remains unexplored the impact of both LLC and MB on VNF performance. In this work, we investigate the importance of MB partitioning along with LLC partitioning to achieve performance isolation in NFV-based systems. Allocating these system resources among co-located VNFs to meet Service Level Agreements (SLAs) is challenging due to the dynamic nature of traffic and varying functionality of VNFs. In this work, we formulate the resource allocation problem as an Integer Linear Programming (ILP) for maximizing the number of accepted VNF requests with SLA guarantees. Since the problem is NP-hard, we present a polynomial time -approximation scheme. Further, we propose a heuristic approach named NFVPermit, a resource manager for NFV-based systems that tries to ensure performance isolation among co-located VNFs based on their current traffic rates and SLA requirements. Through extensive experiments, we show how NFVPermit outperforms state-of-the-art and baseline approaches.

An Efficient NVM-Based Architecture for Intermittent Computing Under Energy Constraints

Batteryless technology evolved to replace battery technology. Nonvolatile memory (NVM)-based processors were explored to store the program state during a power failure. The energy stored in a capacitor is used for a backup during a power failure. Since the size of a capacitor is fixed and limited, the available energy in a capacitor is also limited and fixed. Thus, the capacitor energy is insufficient to store the entire program state during frequent power failures. This article proposes an architecture that assures safe backup of volatile contents during a power failure under energy constraints. Using a proposed dirty block table (DBT) and writeback queue (WBQ), this work limits the number of dirty blocks in the L1 cache at any given time. We further conducted a set of experiments by varying the parameter sizes to help the user make appropriate design decisions concerning their energy requirements. The proposed architecture decreases energy consumption by 17.56%, the number of writes to NVM by 18.97% at last level cache (LLC), and 10.66% at a main-memory level compared to baseline architecture.

Lazy Load Scheduling for Mixed-criticality Applications in Heterogeneous MPSoCs

Newly emerging multiprocessor system-on-a-chip (MPSoC) platforms provide hard processing cores with programmable logic (PL) for high-performance computing applications. In this article, we take a deep look into these commercially available heterogeneous platforms and show how to design mixed-criticality applications such that different processing components can be isolated to avoid contention on the shared resources such as last-level cache and main memory. Our approach involves software/hardware co-design to achieve isolation between the different criticality domains. At the hardware level, we use a scratchpad memory (SPM) with dedicated interfaces inside the PL to avoid conflicts in the main memory. At the software level, we employ a hypervisor to support cache-coloring such that conflicts at the shared L2 cache can be avoided. In order to move the tasks in/out of the SPM memory, we rely on a DMA engine and propose a new CPU-DMA co-scheduling policy, called Lazy Load , for which we also derive the response time analysis. The results of a case study on image processing demonstrate that the contention on the shared memory subsystem can be avoided when running with our proposed architecture. Moreover, comprehensive schedulability evaluations show that the newly proposed Lazy Load policy outperforms the existing CPU-DMA scheduling approaches and is effective in mitigating the main memory interference in our proposed architecture.

L2C2: Last-level compressed-contents non-volatile cache and a procedure to forecast performance and lifetime.

Several emerging non-volatile (NV) memory technologies are rising as interesting alternatives to build the Last-Level Cache (LLC). Their advantages, compared to SRAM memory, are higher density and lower static power, but write operations wear out the bitcells to the point of eventually losing their storage capacity. In this context, this paper presents a novel LLC organization designed to extend the lifetime of the NV data array and a procedure to forecast in detail the capacity and performance of such an NV-LLC over its lifetime. From a methodological point of view, although different approaches are used in the literature to analyze the degradation of an NV-LLC, none of them allows to study in detail its temporal evolution. In this sense, this work proposes a forecasting procedure that combines detailed simulation and prediction, allowing an accurate analysis of the impact of different cache control policies and mechanisms (replacement, wear-leveling, compression, etc.) on the temporal evolution of the indices of interest, such as the effective capacity of the NV-LLC or the system IPC. We also introduce L2C2, a LLC design intended for implementation in NV memory technology that combines fault tolerance, compression, and internal write wear leveling for the first time. Compression is not used to store more blocks and increase the hit rate, but to reduce the write rate and increase the lifetime during which the cache supports near-peak performance. In addition, to support byte loss without performance drop, L2C2 inherently allows N redundant bytes to be added to each cache entry. Thus, L2C2+N, the endurance-scaled version of L2C2, allows balancing the cost of redundant capacity with the benefit of longer lifetime. For instance, as a use case, we have implemented the L2C2 cache with STT-RAM technology. It has affordable hardware overheads compared to that of a baseline NV-LLC without compression in terms of area, latency and energy consumption, and increases up to 6-37 times the time in which 50% of the effective capacity is degraded, depending on the variability in the manufacturing process. Compared to L2C2, L2C2+6 which adds 6 bytes of redundant capacity per entry, that means 9.1% of storage overhead, can increase up to 1.4-4.3 times the time in which the system gets its initial peak performance degraded.

CAPMIG: Coherence-Aware Block Placement and Migration in Multiretention STT-RAM Caches

In recent years, the increased working set size of applications craves more memory demand in terms of large-sized last-level caches (LLCs). To fulfill this, one of the promising technology is STTRAM. However, high write energy and write latency make it challenging to adopt it on a wide scale. Multiretention STTRAM caches have been considered an improvisation over standard STTRAM caches by reducing their retention time which reduces the write latency. Here, we have to negotiate with a refresh operation by applying various refresh management techniques. However, its retention period consumes significant refresh energy in the periodic refresh. In this article, we take help from the coherence protocol and decide the best retention type for each block. A block loaded on a write access is likely to get more writes in the future and is therefore loaded in the lowest retention time region. Similarly, instruction blocks are loaded in the highest retention time region. This helps in reducing the number of refreshes incurred by the blocks. During runtime, the blocks may change their access patterns, requiring a change in their retention region. This article also proposes a migration policy to relocate the blocks to appropriate regions during runtime. Identification of zero data value blocks and not refreshing them is an additional augmentation to our proposal. The experimental result using full system simulation shows a good reduction in the number of refreshes and energy consumption over the baseline designs.

Energy-Efficient Cache Partitioning Using Machine Learning for Embedded Systems

Nowadays, embedded device applications have become partially correlated and can share platform resources. Cross-execution and sharing resources can cause memory access conflicts, especially in the Last Level Cache (LLC). LLC is a promising candidate for improving system performance on multicore embedded systems. It leads to a reduction in the number of high-latency main memory accesses. Currently, commercial devices can use cache partitioning. The software could better utilize the LLC and conserve energy by caching. This paper proposes a new energy-optimization model for embedded multicore systems based on a reconfigurable artificial neural network LLC architecture. The proposed model uses a machine-learning approach to express the reconfiguration of LLC, and can predict each task’s next interval LLC partitioning factor at runtime. The obtained experimental results reveal that the proposed model - compared to other algorithms - improves energy consumption by 28%, and gives 33% reduction in the LLC miss rate.

Casper: Accelerating Stencil Computations Using Near-Cache Processing

Stencil computations are commonly used in a wide variety of scientific applications, ranging from large-scale weather prediction to solving partial differential equations. Stencil computations are characterized by three properties: (1) low arithmetic intensity, (2) limited temporal data reuse, and (3) regular and predictable data access pattern. As a result, stencil computations are typically bandwidth-bound workloads, which only experience limited benefits from the deep cache hierarchy of modern CPUs. In this work, we propose Casper, a near-cache accelerator consisting of specialized stencil computation units connected to the last-level cache (LLC) of a traditional CPU. Casper is based on two key ideas: (1) avoiding the cost of moving rarely reused data throughout the cache hierarchy, and (2) exploiting the regularity of the data accesses and the inherent parallelism of stencil computations to increase overall performance.With minimal changes in LLC address decoding logic and data placement, Casper performs stencil computations at the peak LLC bandwidth. We show that by tightly coupling lightweight stencil computation units near LLC, Casper improves performance of stencil kernels by 1.65× on average (up to 4.16×) compared to a commercial high-performance multi-core processor, while reducing system energy consumption by 35% on average (up to 65%). Casper provides 37× (up to 190×) improvement in performance-per-area compared to a state-of-the-art GPU.

Last-Level Cache Insertion and Promotion Policy in the Presence of Aggressive Prefetching

The last-level cache (LLC) is the last chance for memory accesses from the processor to avoid the costly latency of going to main memory. LLC management has been the topic of intense research focusing on two main techniques: replacement and prefetching. However, these two ideas are often evaluated separately, with one being studied outside the context of the state-of-the-art in the other. We find that high-performance replacement and highly accurate pattern-based prefetching do not result in synergistic improvements in performance. The overhead of complex replacement policies is wasted in the presence of aggressive prefetchers. We find that a simple replacement policy with minimal overhead provides at least the same benefit as a state-of-the-art replacement policy in the presence of aggressive pattern-based prefetching. Our proposal is based on the idea of using a genetic algorithm to search the space of insertion and promotion policies that generalize transitions in the recency stack for the least-recently-used policy.

TPPD: Targeted Pseudo Partitioning based Defence for cross-core covert channel attacks

Contemporary computing employs cache hierarchy to fill the speed gap between processors and main memories. In order to optimise system performance, Last Level Caches (LLC) are shared among all the cores. Cache sharing has made them an attractive surface for cross-core timing channel attacks. In these attacks, an attacker running on another core can exploit the access timing of the victim process to infiltrate the secret information. One such attack is called a cross-core Covert Channel Attack (CCA). Timely detection and then prevention of cross-core CCA is critical for maintaining the integrity and security of users, especially in a shared computing environment. In this work, we have proposed an efficient cross-core CCA mitigation technique. We propose a way-wise cache partitioning on targeted sets, only for the processes suspected to be attackers. In this way, the performance impact on the entire LLC is minimised, and benign applications can utilise the LLC to its full capacity. We have used a cycle-accurate simulator (gem5) to analyse the performance of the proposed method and its security effectiveness. It has been successful in abolishing the cross-core covert timing channel attack with no significant performance impact on benign applications. It causes 23% less cache misses in comparison to existing partitioning based solutions while requiring ≈0.26% storage overhead.