Cache Memory Research Articles

2nd Generation 3D V-Cache TM Enablement

Here we present AMD’s 2nd generation 3D V-Cache TM technology and package level results. Key highlights are: 1. Use of face to back (F2B) stacking of 7 nm node cache memory chiplet (SRAM chiplet) on 5 nm node logic chiplet (“Zen 4” compute chiplet). 2. Bond pad via (BPV) landing on top Aluminum pad (AP) instead of top metal layer With the slowdown in Moore’s Law, heterogeneously integrated chiplet architecture enabled by advanced packaging technologies is becoming essential to enabling the continued economically viable growth of power efficient computing. Decreasing interconnect pitches between the various chiplets, beyond what the conventional solder-based packaging techniques offer, is needed to achieve the required power, performance, area, and cost (PPAC) improvements. We demonstrate hybrid bonding (HB) of stacked chiplets at 9 um bond pitches with die to wafer (D2W) stacking to enable an order of magnitude higher bandwidth between chiplets at lower power and latency compared to existing packaging technologies. Moreover, in the top chiplet, the BPV lands on the top AP instead of the M13 top metal layer, enabling shorter connectivity path. Extensive reliability of these parts in an LGA MCM package has been completed, with the units successfully passing suite of package reliability tests.

Twinkle: A GPU-based Binary-lens Microlensing Code with the Contour Integration Method

Abstract With the rapidly increasing rate of microlensing planet detections, microlensing modeling software faces significant challenges in computation efficiency. Here, we develop the Twinkle code, an efficient and robust binary-lens modeling software suite optimized for heterogeneous computing devices, especially GPUs. Existing microlensing codes have the issue of catastrophic cancellation that undermines the numerical stability and precision, and Twinkle resolves them by refining the coefficients of the binary-lens equation. We also devise an improved method for robustly identifying ghost images, thereby enhancing computational reliability. We have advanced the state of the art by optimizing Twinkle specifically for heterogeneous computing devices by taking into account the unique task and cache memory dispatching patterns of GPUs, while the compatibility with the traditional computing architectures of CPUs is still maintained. Twinkle has demonstrated an acceleration of approximately 2 orders of magnitude (≳102×) on contemporary GPUs. The enhancement in computational speed of Twinkle will translate to the delivery of accurate and highly efficient data analysis for ongoing and upcoming microlensing projects. Both GPU and CPU versions of Twinkle are open-source and publicly available.

An Efficient and Secure Data Sharing Scheme for Edge-Enabled IoT

The project proposes an Edge-based Blockchain Secure Data Sharing Scheme (EBSDSS) to enhance the efficiency and security of data transmission between IoT devices, edge servers, and cloud servers, addressing latency and security concerns inherent in traditional cloud systems [1][6]. By utilizing edge servers that receive data directly from IoT devices, the scheme significantly reduces data transfer times, offering better performance in comparison to centralized cloud models [7][14]. The integration of Blockchain technology ensures data integrity and security by storing tamper-proof records through hash-based transactions, aligning with blockchainbased secure data sharing models in IoT environments [2][15]. To protect data, AES symmetric encryption is employed, while Local Sensitive Hashing (LSH) facilitates efficient and rapid searches across Blockchain-stored IoT data, ensuring quick access without compromising security [16][18]. Furthermore, a certificate-less signature scheme is introduced to verify the authenticity of IoT devices and the data they share, reducing the need for complex certificate management [12][13]. Shamir’s secret sharing scheme is used to enhance the security of cryptographic keys, ensuring that sensitive data is securely shared and stored, which aligns with previous work on data security and privacy in IoT systems [3][5]. As an extension, the project introduces two enhancements: the use of Elliptic Curve Cryptography (ECC) for more efficient encryption, ensuring stronger security with smaller key sizes [4][19], and the implementation of cache memory to accelerate LSH search operations, reducing the need for repeated searches for identical queries [17]. This innovative approach not only improves the overall efficiency and security of IoT data sharing but also paves the way for more resilient and responsive IoT applications across various domains such as industrial IoT, healthcare, and smart cities [20][8].

The Role of Cache Memory In Enhancing Microprocessor Performance in PT. Srikandi Sinergi Sakti

Cache memory in microprocessors has an important role in improving computer system performance by reducing data access time. This research aims to test the hypothesis that increasing the size and level of cache memory can significantly improve microprocessor performance. The research methodology involves a literature study on the concept of cache memory and experimental simulations using computer architecture simulators, such as Gem5, to model scenarios with varying cache sizes and levels. In these simulations, performance parameters such as memory access latency, throughput, and Instructions Per Cycle (IPC) were measured and analyzed. The results show that increasing cache size and level generally contributes towards improving microprocessor performance by reducing data access time. Further statistical analysis supports the hypothesis that there is a positive correlation between cache size and level and system efficiency. These findings provide useful insights in future microprocessor architecture design and memory system optimization.

Experimental Comparison of Encryption Algorithms On Smart Devices

Now, the technological environments rely on a large number of users and devices interconnected in such a way as to be able to share information and exchange resources. For such reasons, security becomes of prime importance inside these networks. Especially, encryption algorithms widely used in smart metering systems form the very backbone of ensuring security. It is unique according to every single parameter: the level of security achieved, speed, operational complexity, length, and type of key used. A comparison of performance and throughput for the most used encryption algorithms, such as AES-128, AES-192, RC4, Blowfish, and ECDSA, is presented here. Such devices, for instance, smart meters, usually represent very resource-constrained computational capability, memory, and data transfer. For these reasons, the experiments investigate performance impacts of using different encryption algorithms on a smart meter environment in a scaled setup. Experiments were run on a laptop device and then downscaled according to the limitation characteristics of the smart meter target device. As for the scaling, it has been performed concerning key factors: CPU of the smart meter, RAM, and cache memory. Execution times of the encryption and decryption processes were measured, as well as the throughput of messages for various file sizes. Quantitative key results obtained included: RC4 with a throughput of 25.37 Kbytes/sec, whereas AES-128 has 4.87 Kbytes/sec, while in ECDSA-256 the performance is much lower: its verification throughput amounts to 0.0023 Kbytes/sec. The results showed significant variations in performance, speed, and throughput between these algorithms. Even considering small smart devices, the encryption and decryption processes yielded efficient throughputs. These findings underscore the importance of choosing the right encryption algorithm for smart meters, balancing both security requirements and resource limitations to ensure optimal performance.

IMPLEMENTED RECONFIGURABLE CACHE MEMORY ARCHITECTURE BASED ON 32-BITS MIPS PROCESSOR

This work presents the design of a reconfigurable cache memory utilizing a 32-bit MIPS processor, implemented through the utilization of VHDL (Very high-speed IC Hardware Description Language). A comprehensive implementation of a 32-bit, single-cycle MIPS processor is presented in VHDL. This processor is capable of supporting 50 instructions, comprising 25-R-type, 16-I-type, and 9-J-type instructions. The processor incorporates a three-dimensional reconfigurable cache memory architecture, enabling the modification of cache memory size, cache organization (in terms of associativity), and cache block size. The reconfigurable cache memory concept proposes the use of multi-cache controller units to execute reconfiguration operations and ensure that all permitted cache memory sets can be utilized with varying levels of associativity. The design is built on an FPGA using the Xilinx ISE design suite, which is based on the hardware description language. To ensure that assess functionality of the proposed reconfigurable cache memory design for various reconfiguration scenarios, numerous assembly language programs have been developed and performed on the MIPS processor. The results validated the efficacy of the suggested reconfigurable cache memory concept and demonstrated its simulation using the Xilinx ISim simulator.

Low power process tolerant nano cache memory for military applications

ABSTRACT Rapid growth in high-performance battery-powered computing applications, Internet of Things (IoT) and artificial intelligence (AI), has raised the need for low-power integrated circuits. Particularly in military applications, wireless sensor nodes are used to detect border intrusion, weapon attacks, biometric wearables, internet of battlefield things (IoBT), etc. The computing and security devices utilised by the soldiers are powered by batteries. To increase the life and performance of these devices, the integrated circuits (IC) used within these devices need careful design. Particularly, an appropriate memory needs more attention because the design of memory has a substantial impact on IC performance as it occupies most of the chip space. In this article, a novel 8T transistor low-power cache memory cell is proposed and simulations for nominal chirality and dual chirality are done using HSPICE-compatible CNFET (carbon nanotube field effect transistor) technology. The power consumption is minimized in dual chirality case than nominal chirality. But still the power saving percentage of the proposed cache memory cell remains good compared to conventional memory structures. Thus the proposed cache memory cell provides low power, delay and energy apart from being process tolerant, thus proving that it is compatible for warfare and military applications.

Design of reliable and fast Schmitt trigger 10T SRAM cells using in-memory computing

Abstract Microprocessors, integrated data storage systems, and cache memories are just a few applications that can benefit from Static random access memory (SRAM). Computing In memory provides a solution for the von Neumann bottleneck problems. Simultaneously, it removes needless repeated data transmission between CPU and memory. Schmitt trigger was utilized for energy-efficient operation to get high reading and writing capacity. Two cross-coupled inverters with a transistor in between increase the design writing process. Furthermore, voltage boost logic is prioritized to maximize the node capacity. Using two dummy columns and dummy rows in RAM, the Replica technique controls the flow of signals via the core. The simulated results are compared to the designs that already exist. The simulation results show that the leakage power of the proposed design is 0.75 nw at 0.6 volts. The write static noise margin (WSNM) of the proposed cell is twice that of SEDF10T, which can be helpful for high-performance computing applications.

Data caching technologies in modern microprocessors

Objective. The study presented in the paper is aimed at studying the methods for improving the efficiency of software in modern computing systems with a hierarchical memory structure. Method. The study is based on data caching technologies in microprocessors. Result. The article presents the results of the analysis of various approaches to the development of efficient software taking into account the characteristics of the memory subsystem of the computing system, which made it possible to prove the importance of cache memory in improving the performance and interaction of computer components. Conclusion. Cache memory is a critical element in the architecture of microprocessors, playing a key role in determining the performance of the computing system. Optimizing the use of cache can significantly improve data access time and, as a result, overall system performance. Software developers need to pay special attention to the characteristics of the memory subsystem when designing and implementing solutions.

Open-AI model Efficient Memory Reduce Management for the Large Language Models (LLMs) Serving with Paged Attention of sharing the KV Cashes

Abstract: High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically. When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem we proposed a Paged Attention. An alternative algorithm inspired by the classical virtual memory and paging techniques in operating systems. An LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage. Our evaluations show that LLM improves the throughput of popular LLMs by 2-4× with the same level of latency compared to the state-of-the-art systems, such as Faster Transformer and Orca. The improvement is more pronounced with longer sequences, larger models, and more complex decoding algorithms

Memory architecture to mitigate side channel attacks for cryptographic application using loop cut technique

Cryptography is crucial in embedded systems for safeguarding sensitive information, maintaining data integrity, and facilitating secure communication. L1 cache memory enhances overall performance by temporarily storing cryptographic keys. Post-quantum cryptography (PQC) focuses on creating algorithms that stay secure even against quantum computing threats. However, PQC doesn't fully protect against side-channel attacks (SCA). As IoT devices like wearable health monitors and industrial sensors become more widespread, the demand for lightweight cryptography increases to balance security with resource constraints. Yet, lightweight cryptography can face challenges, including reduced security and increased susceptibility to SCA due to smaller key sizes. Attackers can exploit power consumption patterns through side-channel attacks (SCA), such as Power Analysis, to extract these secret keys. Once a key is compromised, encrypted data becomes vulnerable. In cloud computing cache side-channel attacks take advantage of multiple virtual machines running simultaneously on the same hardware, enabling them to extract sensitive information from encryption processes. Although many SRAM cells have been designed in the literature, none offer complete security against SCA. This paper presents a new architecture for secure cache memory using a proposed 10T SRAM cell that secures all three cell operations (read, write, and hold) against SCA attacks. Additionally, this architecture also prevents half selection issue in cache memory. The security measure of proposed-10T cell is 97.19 % which is highest among all other cells and reliability measure is 82.03 %.The cell also offers highest hold stability of 421 mV along with good read stability and write ability of 220 mV and 334 mV respectively. The leakage of Proposed-10T cell is also less i.e., 13.56 pA. The Performance and Security Factor (PSF) is computed for all considered cells, encompassing various performance and security metrics. The normalized PSF (PSF)N of the Proposed-10T cell is 2.17, the highest among all cells considered for comparison. Therefore, the Proposed-10T cell architecture is fully secure against SCA and achieves better performance during cell operations.

Exploration and optimization of novel replacement and prefetching strategies for inefficiencies of advanced MRAM-based hybrid cache systems

With the emergence of cutting-edge hardware systems such as cloud computing, edge computing, and on-chip neural network accelerators, how to design advanced memory strategies to substitute the traditional ones for maximizing the potential performance of non-volatile memory (NVM) under the existing hardware conditions, has become an urgent research issue for both academia and industrial communities. It is promising and innovative to improve computer systems in the layer of data exchanging with the emerging advanced semiconductor devices. In the paper, to address the inefficiencies of write-intensive, high power consumption, low hit rate and so on, which exist in hybrid magnetic random access memory cache systems, three novel cache replacement strategies and two cache prefetching strategies are put forward. The proposed triple novel replacement strategies, including historical frequency and time judgments, duplicate data-aware deletion, and dynamic relevance factors computing, can be utilized to compensate for the shortcomings of the traditional least recently used replacement strategy, respectively. In the two novel prefetching strategies, region distribution parameters and Listnet ranking network are imported into the caching process, respectively, to achieve optimized hitting performance. The simulation results demonstrate that the proposed replacement strategies can achieve up to 61.76%, 84.91%, 56.49%, and 53.21% optimization of write count, hit rate, dynamic power, and IPC compared to the conventional one. The proposed prefetching strategy can achieve up to 91.27%, 49.25% hit rate and IPC optimization. Meanwhile, the synthetic evaluation of the replacement and prefetching strategies are elaborated in the paper, including multi-core characteristics, information entropy, interplays and the performance constraints between replacement and prefetching mechanism, which would facilitate more credible ideas for future memory inefficiencies management and strategy design.

Thermal optimization of two-terminal SOT-MRAM

While magnetoresistive random-access memory (MRAM) stands out as a leading candidate for embedded nonvolatile memory and last-level cache applications, its endurance is compromised by substantial self-heating due to the high programming current density. The effect of self-heating on the endurance of the magnetic tunnel junction (MTJ) has primarily been studied in spin-transfer torque (STT)-MRAM. Here, we analyze the transient temperature response of two-terminal spin–orbit torque (SOT)-MRAM with a 1 ns switching current pulse using electro-thermal simulations. We estimate a peak temperature range of 350–450 °C in 40 nm diameter MTJs, underscoring the critical need for thermal management to improve endurance. We suggest several thermal engineering strategies to reduce the peak temperature by up to 120 °C in such devices, which could improve their endurance by at least a factor of 1000× at 0.75 V operating voltage. These results suggest that two-terminal SOT-MRAM could significantly outperform conventional STT-MRAM in terms of endurance, substantially benefiting from thermal engineering. These insights are pivotal for thermal optimization strategies in the development of MRAM technologies.

Enhancing Big Data Performance Through Graph Coloring-Based Locality of Reference

Efficiency is a crucial factor when handling the retrieval and storage of data from vast amounts of records in a Big Data repository. These systems require a subset of data that can be accommodated within the combined physical memory of a cluster of servers. It becomes impractical to analyze all of the data if its size exceeds the available memory capacity. Retrieving data from virtual storage, primarily hard disks, is significantly slower compared to accessing data from main memory, resulting in increased access time and diminished performance. To address this, a proposed model aims to enhance performance by identifying the most suitable data locality structure within a big data set and reorganizing the data schema accordingly; by locality, it has been referred to as a particular access pattern. This allows transactions to be executed on data residing in the fastest memory layer, such as cache, main memory, or disk cache

Minimizing cache usage with fixed-priority and earliest deadline first scheduling

Cache partitioning is a technique to reduce interference among tasks running on the processors with shared caches. To make this technique effective, cache segments should be allocated to tasks that will benefit the most from having their data and instructions stored in the cache. The requests for cached data and instructions can be retrieved faster from the cache memory instead of fetching them from the main memory, thereby reducing overall execution time. The existing partitioning schemes for real-time systems divide the available cache among the tasks to guarantee their schedulability as the sole and primary optimization criterion. However, it is also preferable, particularly in systems with power constraints or mixed criticalities where low- and high-criticality workloads are executing alongside, to reduce the total cache usage for real-time tasks. Cache minimization as part of design space exploration can also help in achieving optimal system performance and resource utilization in embedded systems. In this paper, we develop optimization algorithms for cache partitioning that, besides ensuring schedulability, also minimize cache usage. We consider both preemptive and non-preemptive scheduling policies on single-processor systems with fixed- and dynamic-priority scheduling algorithms (Rate Monotonic (RM) and Earliest Deadline First (EDF), respectively). For preemptive scheduling, we formulate the problem as an integer quadratically constrained program and propose an efficient heuristic achieving near-optimal solutions. For non-preemptive scheduling, we combine linear and binary search techniques with different fixed-priority schedulability tests and Quick Processor-demand Analysis (QPA) for EDF. Our experiments based on synthetic task sets with parameters from real-world embedded applications show that the proposed heuristic: (i) achieves an average optimality gap of 0.79% within 0.1× run time of a mathematical programming solver and (ii) reduces average cache usage by 39.15% compared to existing cache partitioning approaches. Besides, we find that for large task sets with high utilization, non-preemptive scheduling can use less cache than preemptive to guarantee schedulability.

Survey on Redundancy Based-Fault tolerance methods for Processors and Hardware accelerators - Trends in Quantum Computing, Heterogeneous Systems and Reliability

Rapid progress in CMOS technology since the late 1990s has increased the vulnerability of processors toward faults. Subsequently, the focus of computer architects has shifted toward designing fault-tolerance methods for processor architectures. Concurrently, chip designers have encountered high-order challenges for designing fault-tolerant processor architectures. For processor cores, redundancy-based fault-tolerance methods for fault detection at the core, micro-architectural, thread, and software levels are discussed. Similar applicable redundancy-based fault-tolerance methods for cache memory and hardware accelerators are also presented in the article. Recent trends in fault-tolerant quantum computing and quantum error correction are also discussed. The classification of state-of-the-art techniques presented will help researchers organize their work on established lines.

Accelerating Graph Analytics Using Attention-Based Data Prefetcher

Graph analytics shows promise for solving challenging problems on relational data. However, memory constraints arise from the large size of graphs and the high complexity of algorithms. Data prefetching is a crucial technique to hide memory access latency by predicting and fetching data into the memory cache beforehand. Traditional prefetchers struggle with fixed rules in adapting to complex memory access patterns in graph analytics. Machine learning (ML) algorithms, particularly long short-term memory (LSTM) models, excel in memory access prediction. However, they encounter challenges such as difficulty in learning interleaved access patterns and high storage costs when predicting in large memory address space. In addition, there remains a gap between designing a high-performance ML-based memory access predictor and developing an effective ML-based prefetcher for an existing memory system. In this work, we propose a novel Attention-based prefetching framework to accelerate graph analytics applications. To achieve high-performance memory access prediction, we propose A2P, a novel Attention-based memory Access Predictor for graph analytics. We use the multi-head self-attention mechanism to extract features from memory traces. We design a novel bitmap labeling method to collect future deltas within a spatial range, making interleaved patterns easier to learn. We introduce a novel super page concept, allowing the model to surpass physical page constraints. To integrate A2P into a memory system, we design a three-module prefetching framework composed of an existing memory hierarchy, a prefetch controller, and the predictor A2P. In addition, we propose a hybrid design to combine A2P and existing hardware prefetchers for higher prefetching performance. We evaluate A2P and the prefetching framework using the widely used GAP benchmark. Prediction experiments show that for the top three predictions, A2P outperforms the widely used state-of-the-art LSTM-based model by 23.1% w.r.t. Precision, 21.2% w.r.t. Recall, and 10.4% w.r.t. Coverage. Prefetching experiments show that A2P provides 18.4% IPC Improvement on average, outperforming state-of-the-art prefetchers BO by 17.2%, ISB by 15.0%, and Delta-LSTM by 10.9%. The hybrid prefetcher combining A2P and ISB achieves 21.7% IPC Improvement, outperforming the hybrid of BO and ISB by 16.3%.

Lightweight Hardware-Based Cache Side-Channel Attack Detection for Edge Devices (Edge-CaSCADe)

Cache Side-Channel Attacks (CSCAs) have been haunting most processor architectures for decades now. Existing approaches to mitigation of such attacks have certain drawbacks, namely software mishandling, performance overhead, and low throughput due to false alarms. Hence, “mitigation only when detected” should be the approach to minimize the effects of such drawbacks. We propose a novel methodology of fine-grained detection of timing-based CSCA using a hardware-based detection module. We discuss the design, implementation, and use of our proposed detection module in processor architectures. Our approach successfully detects attacks that flush secret victim information from cache memory like Flush+Reload, Flush+Flush, Prime+Probe, Evict+Probe, and Prime+Abort, commonly known as cache timing attacks. Detection is on time with minimal performance overhead. The parameterizable number of counters used in our module allows detection of multiple attacks on multiple sensitive locations simultaneously. The fine-grained nature ensures negligible false alarms, severely reducing the need for any unnecessary mitigation. The proposed work is evaluated by synthesizing the entire detection algorithm as an attack detection block, Edge-CaSCADe, in a RISC-V processor as a target example. The detection results are checked under different workload conditions with respect to the number of attackers and the number of victims having RSA-, AES-, and ECC-based encryption schemes like ECIES, and on benchmark applications like MiBench and Embench. More than 98% detection accuracy within 2% of the beginning of an attack can be achieved with negligible false alarms. The detection module has an area and power overhead of 0.9% to 2% and 1% to 2.1% for the targeted RISC-V processor core without cache for one to five counters, respectively. The detection module does not affect the processor critical path and hence has no impact on its maximum operating frequency.

Analysis of the parasitic capacitance effects on the layout of latch-based sense amplifiers for improving SRAM performance

Static random-access memory (SRAM) technology is utilized in designing cache memory to enhance the processing performance of computer systems. The sense amplifier (SA) circuit, a crucial component of memory design, significantly impacts data access time and power consumption. In comparison to conventional differential sense amplifiers (DSA) designs, latch-based sense amplifiers (LSA) used in memory-based computing platforms have specific requirements, including robust noise resistance in harsh working environments and low power consumption, particularly for internet of thing (IoT) embedded computing applications. However, the performance can be degraded due to various factors that arise during the layout, such as conductor resistance or the development of parasitic capacitance. Therefore, this study employs low-voltage 22 nm UMC CMOS technology for LSA design layout and analyzes the factors influencing design performance post-layout process. Layout design optimization techniques are applied to mitigate the impact of parasitic capacitance on important signal lines such as data line/data line bar (DLL/DLLB). Based on the performance analysis results, it is possible to achieve a reduction in power consumption of up to 15% and a 5% decrease in read delay time by implementing circuit layout LSA design optimization techniques.

Parallel Maximum Cardinality Matching for General Graphs on GPUs.

The matching problem formulated as Maximum Cardinality Matching in General Graphs (MCMGG) finds the largest matching on graphs without restrictions. The Micali-Vazirani algorithm has the best asymptotic complexity for solving MCMGG when the graphs are sparse. Parallelizing matching in general graphs on the GPU is difficult for multiple reasons. First, the augmenting path procedure is highly recursive, and NVIDIA GPUs use registers to store kernel arguments, which eventually spill into cached device memory, with a performance penalty. Second, extracting parallelism from the matching process requires partitioning the graph to avoid any overlapping augmenting paths. We propose an implementation of the Micali-Vazirani algorithm which identifies bridge edges using thread-parallel breadth-first search, followed by block-parallel path augmentation and blossom contraction. Augmenting path and Union-find methods were implemented as stack-based iterative methods, with a stack allocated in shared memory. Our experimentation shows that compared to the serial implementation, our approach results in up to 15-fold speed-up for very sparse regular graphs, up to 5-fold slowdown for denser regular graphs, and finally a 50-fold slowdown for power-law distributed Kronecker graphs. This implementation has been open-sourced for further research on developing combinatorial graph algorithms on GPUs.