On-chip Cache Research Articles

Cache Memory Design With Magnetic Skyrmions in a Long Nanotrack

Magnetic skyrmion (MS), a vortexlike region with reversed magnetization in nanomagnets, has recently emerged as an exciting development in the field of spintronics. It has a number of beneficial features, including remarkably high stability, ultralow depinning current density, and extremely compact size. Due to these benefits, skyrmions have generated great interest in the design of spintronic memory. In this paper, we evaluate the use of skyrmion-based memory as a last-level cache for general-purpose processors. In the skyrmion-based memory structure, data can be densely packed as multiple bits in a long magnetic nanotrack. Write operations are performed by injecting a spin-polarized current in the nanotrack. Since multiple skyrmions (each representing a bit) are packed into a single nanotrack, they need to be accessed by shifting them along the nanotrack with a charge current passing through a spin-Hall metal (SHM). We identify the following key challenges associated with MS-based cache design: 1) the high-current requirements for skyrmion nucleation limit the density benefits offered by these structures, since the transistor supplying write currents is the limiting factor that determines the bit-cell area; 2) the proposed nanotrack structure results in significant performance overheads due to the latency arising from the shift operations; 3) the skyrmions move toward the edge of the nanotrack during shift operations owing to the Magnus force. Hence, an additional idle operation time is required to relax skyrmions back through the repulsive force from the edge; and 4) to avoid annihilation of skyrmions from the edge, the duration and the current density of the shift operation have to be well controlled. To overcome these challenges, a multi-bit skyrmion cell with appropriate peripheral circuit is proposed, considering the heterogeneity in the read/write characteristics. The density benefits are explored by performing the layout of different multi-bit cells. We perform a systematic device-circuit-architecture co-design to evaluate the feasibility of our proposal. Our experiments demonstrate the potential of, and the challenges involved in, using skyrmion-based memory as last-level caches.

An Adaptive Thermal-Aware ECC Scheme for Reliable STT-MRAM LLC Design

Considering the insatiable demand for high-performance computing, on-chip cache capacity increases rapidly. Spin-transfer-torque magnetoresistive random-access memory (STT-MRAM) is a promising cache candidate due to ultralow standby power, high-access speed, and integration density. Unfortunately, when the feature size of magnetic tunnel junction (MTJ) scales down to 1 Xnm, read current approaches write current closely, which may result in read disturbance threatening the reliability of STT-MRAM. Furthermore, the elevating on-chip temperature reduces the thermal stability of STT-MRAM remarkably and aggravates the read disturbance. Error correction code (ECC) is an effective technique to enhance memory reliability. In this paper, we take advantage of the thermal dependence of STT-MRAM and propose a thermally adaptive ECC design, called “Chameleon,” that can adjust the ECC protection strength dynamically to reduce the ECC storage overhead and improve the cache access performance and energy efficiency. Experimental results show that compared to the conservative nonadaptive ECC scheme, our design can improve both cache performance and energy consumption effectively.

Guest Editorial: IEEE Transactions on Computers Special Section on Emerging Non-Volatile Memory Technologies: From Devices to Architectures and Systems

The papers in this special section focus on emerging non-volatile memory technologies (NVM). Emerging NVM technologies have attracted significant interest in recent years because of the fast-growing performance and capacity demands on memory and storage in the big data era. Well known examples include the 3D XPoint memory and various NVDIMM hybrid memory technologies. They have shown potential towards larger memory and storage capacities with nearly zero leakage power, while extending memory/ system architecture design approaches. The unique characteristics of NVM technologies not only introduce new opportunities, but simultaneously create challenges to the designs at multiple levels of abstraction in computer systems, including those of device management, CPU cache management, memory/storage architecture, and system design. Furthermore, emerging NVM technologies also drive the development of techniques which perform computing operations in memory, i.e., processing-in-memory (PIM), by taking advantage of crossbar-based accelerators using NVMs. Thus, for the emerging NVM technologies, there is an urgent need for technology innovation, modeling, analysis, design, and application, ranging from the device-level to the system-level.

Morphable DRAM Cache Design for Hybrid Memory Systems

DRAM caches have emerged as an efficient new layer in the memory hierarchy to address the increasing diversity of memory components. When a small amount of fast memory is combined with slow but large memory, the cache-based organization of the fast memory can provide a SW-transparent solution for the hybrid memory systems. In such DRAM cache designs, their effectiveness is affected by the bandwidth and latency of both fast and slow memory. To quantitatively assess the effect of memory configurations and application patterns on the DRAM cache designs, this article first investigates how three prior approaches perform with six hybrid memory scenarios. From the investigation, we observe no single DRAM cache organization always outperforms the other organizations across the diverse hybrid memory configurations and memory access patterns. Based on this observation, this article proposes a reconfigurable DRAM cache design that can adapt to different hybrid memory combinations and workload patterns. Unlike the fixed tag and data arrays of conventional on-chip SRAM caches, this study advocates to exploit the flexibility of DRAM caches, which can store tags and data to DRAM in any arbitrary way. Using a sample-based mechanism, the proposed DRAM cache controller dynamically finds the best organization from three candidates and applies the best one by reconfiguring the tags and data layout in the DRAM cache. Our evaluation shows that the proposed morphable DRAM cache can outperform the fixed DRAM configurations across six hybrid memory configurations.

QoS-aware secure transaction framework for internet of things using blockchain mechanism

The Internet of Things (IoT) paradigm enables an enormous network with millions of connected smart things to control almost all aspect of services. In an IoT scenario, self-configured things are dynamically connected in a universal network. The small things are broadly distributed in a real world model with some essential processing capacity and finite storage. Security, privacy and reliability are the most important pillars of the IoT infrastructure. It is challenging to share personal data, highly confidential professional data over a centralized system. The infrastructure itself is capable to provide the guarantee to transmit data safely. However, a special attention for the data along with data consistency is required. Cyber attackers and hackers mostly target the centralized systems. To triumph over these types of situations we propose a security based blockchain service along with some encryption approaches for a secure communication. The encryption and decryption is performed by a hash based secret key. Risk modelling and relative risk reduction also done to prevent attacks under a virtual private network in IoT domain. Before and after attack users also wish to know quality of service (QoS). We have developed a mobile application framework which provides desire output of a user from different aspect of QoS for better understanding. The MQTT broker performs as a negotiator to convey data between cloud consumer and cloud provider. In QoS monitoring section user get chance to reconsider availability, reliability and responsibility like QoS factors while recovering resources and actual resources before attacking system based on CPU utilization, cache status and storage. As per experiment result we able to recover 100% RAM, 97.64% CPU utilization and 78.7% storage using proposed encryption and algorithm on blockchain infrastructure.

Cache attack detection in virtualized environments

Cache Side Channel (CSC) attacks are prominent security threats in modern day processors. CSC attacks have caused a large impact in security and widespread concern in information security community. The objective of CSC attacks is to crack the confidentiality of a computer system and extract sensitive secrets through CPU caches. CSC attacks do not require any specific permission and just need access to a shared cache. CSC attacks are subtle, powerful and much more feasible in virtualized environments than in non-virtualized environments. To help protect security in virtualization environments, CSC attacks detection methods have become critical and important. The Flush+Flush cache attack technique, as the most recent CSC attack variant demonstrated that the state-of-the-art discovery methods can be simply sidestepped. In this paper, we introduce novel approach that make the detection of Flush+Flush cache attack in virtualized environments.

A-CACHE: Alternating Cache Allocation to Conduct Higher Endurance in NVM-Based Caches

Recent developments in non-volatile memories (NVMs) have introduced them as an alternative for SRAMs in on-chip caches. Besides the promising features of NVMs, e.g., near-zero leakage power, immunity to radiation-induced particle strike, and higher density, a major drawback of NVM-based caches is their short lifetime due to limited write endurance. This brief first reveals that in L1 caches, the lifetime of data-cache is about 472 $\times$ shorter than that of instruction-cache (I-cache) due to extreme imbalance write stress between the two. Then, we propose a technique, so-called alternating cache allocation to conduct higher endurance (A-CACHE), to improve the lifetime of frequently written D-cache by exploiting rarely written I-cache. The key idea in A-CACHE is to alternate the locations of storing instructions and data between I-cache and D-cache. The evaluation results show that A-CACHE improves the lifetime of the cache by 83% with negligible overheads.

RAW-Tag: Replicating in Altered Cache Ways for Correcting Multiple-Bit Errors in Tag Array

Tag array in on-chip caches is one of the most vulnerable components to radiation-induced soft errors. Protecting the tag array in some processors is limited to error detection using the parity check, since the overheads of error correcting codes are not affordable in this component. State-of-the-art tag protection schemes combine the parity check with replication to provide error correction capability. Classifying these replication-based schemes into partial-replication and full-replication, the former offers a low overhead protection in which a large fraction of detectable errors remain uncorrectable, whereas the latter imposes a significant overhead to correct all of the errors. This paper proposes a low overhead full-replication scheme, so called Replicating in Altered Ways of Tag (RAW-Tag), to correct all detectable errors. RAW-Tag manipulates the cache replacement algorithm and keeps track of the incoming/evicting cache lines to not only provide a replica for all tags, but also eliminate the simultaneous susceptibility of both a tag and its replica to a single Multiple-Bit Upset (MBU). The simulation results show that RAW-Tag imposes no performance overhead and increases the energy consumption of L1 and L2 caches by only 6.6 and 0.3 percent, respectively, as compared with the baseline.

AWARE: Adaptive Way Allocation for Reconfigurable ECCs to Protect Write Errors in STT-RAM Caches

Spin-Transfer Torque Random Access Memories (STT-RAMs) are a promising alternative to SRAMs in on-chip caches. STT-RAMs face with a high error rate in write operations due to stochastic switching. To alleviate this problem, Error-Correcting Codes (ECCs) are commonly used, which results in a significant area and energy consumption overhead. This paper proposes an efficient technique, so-called Adaptive Way Allocation for Reconfigurable ECCs (AWARE), to correct write errors in STT-RAM caches. AWARE exploits the asymmetric error rate in cell switching directions, which leads to data-dependent write error rates, to reduce the ECC overheads without compromising the reliability of the cache. To this end, instead of protecting all cache lines using strong ECCs, AWARE employs a simple ECC that guarantees a given reliability level for the majority of writes. Meanwhile, when a data block with high error rate is written, one way in the target set is adaptively configured to store the check bits of a strong ECC for this block. The evaluation results show that, compared with conventional ECCs, AWARE reduces the ECC area by about 81.2 percent and the cache energy consumption by about 9.5 percent. These reductions are achieved by imposing less than 1 percent performance overhead and without compromising the reliability.

Modeling Emerging Memory-Divergent GPU Applications

Analytical performance models yield valuable architectural insight without incurring the excessive runtime overheads of simulation. In this work, we study contemporary GPU applications and find that the key performance-related behavior of such applications is distinct from traditional GPU applications. The key issue is that these GPU applications are memory-intensive and have poor spatial locality, which implies that the loads of different threads commonly access different cache blocks. Such memory-divergent applications quickly exhaust the number of misses the L1 cache can process concurrently, and thereby cripple the GPU's ability to use Memory-Level Parallelism (MLP) and Thread-Level Parallelism (TLP) to hide memory latencies. Our Memory Divergence Model (MDM) is able to accurately represent this behavior and thereby reduces average performance prediction error by 14× compared to the state-of-the-art GPUMech approach across our memory-divergent applications.

Power Reduction and BTI Mitigation of Data-Cache Memory Based on the Storage Management of Narrow-Width Values

Power dissipation of on-chip cache memories contributes a large portion of a processor’s power consumption. Therefore, power management of cache memories is crucial in modern processors. On the other hand, bias temperature instability (BTI) is one of the most serious reliability concerns in SRAM-based on-chip memories. The effect of BTI on SRAM-cell transistors is manifested as their threshold-voltage shift over stress duration, which decreases the robustness of these structures. This paper presents a power consumption-reduction technique for data-cache memories, based on a storage management considering narrow-width values (NWVs), which additionally mitigates the BTI rate on the most aging susceptible SRAM cells of data-cache memory, as well. In the proposed technique, the most significant bits of data-cache memory words are stored in SRAM blocks that operate with lower $V_{\mathrm {DD}}$ , to achieve the decrease of the related power consumption and aging rate. This is shown to reduce leakage-current and dynamic current of data-cache memory by 37.6% and 22.1%, respectively, besides improving its lifetime by up to 2.25x, all with negligible performance and area overheads.

Exploring the Role of Large Centralised Caches in Thermal Efficient Chip Design

In the era of short channel length, Dynamic Thermal Management (DTM) has become a challenging task for the architects and designers engineering modern Chip Multi-Processors (CMPs). Ever-increasing demand of processing power along with the developed integration technology produces CMPs with high power density, which in turn increases effective chip temperature. This increased temperature leads to increase in the reliability issues for the chip-circuitry with significant increment in leakage power consumption. Recent DTM techniques apply DVFS or Task Migration to reduce temperature at the cores, the hottest on-chip components, but often ignore the on-chip hot caches. To commensurate the high data demand of these cores, most of the modern CMPs are equipped with large multi-level on-chip caches, out of which on-chip Last Level Caches (LLCs) occupy the largest on-chip area. These LLCs are accounted for their significantly high leakage power consumption that can also potentially generate on-chip hotspots at the LLCs similar to the cores. As power consumption constructs the backbone of heat dissipation, hence, this work dynamically shrinks cache size while maintaining performance constraint to reduce LLC leakage, primarily. These turned-off cache portions further work as on-chip thermal buffers for reducing average and peak temperature of the CMP without affecting the computation. Simulation results claim that, at a minimal penalty on the performance, proposed cache-based thermal management having 8MB centralised multi-banked shared LLC gives around 5°C reduction in peak and average chip temperature, which are comparable with a Greedy DVFS policy.

Evaluation of Hardware Data Prefetchers on Server Processors

Data prefetching, i.e., the act of predicting an application’s future memory accesses and fetching those that are not in the on-chip caches, is a well-known and widely used approach to hide the long latency of memory accesses. The fruitfulness of data prefetching is evident to both industry and academy: Nowadays, almost every high-performance processor incorporates a few data prefetchers for capturing various access patterns of applications; besides, there is a myriad of proposals for data prefetching in the research literature, where each proposal enhances the efficiency of prefetching in a specific way. In this survey, we evaluate the effectiveness of data prefetching in the context of server applications and shed light on its design trade-offs. To do so, we choose a target architecture based on a contemporary server processor and stack various state-of-the-art data prefetchers on top of it. We analyze the prefetchers in terms of their ability to predict memory accesses and enhance overall system performance, as well as their imposed overheads. Finally, by comparing the state-of-the-art prefetchers with impractical ideal prefetchers, we motivate further work on improving data prefetching techniques.

An Efficient GPU Cache Architecture for Applications with Irregular Memory Access Patterns

GPUs provide high-bandwidth/low-latency on-chip shared memory and L1 cache to efficiently service a large number of concurrent memory requests. Specifically, concurrent memory requests accessing contiguous memory space are coalesced into warp-wide accesses. To support such large accesses to L1 cache with low latency, the size of L1 cache line is no smaller than that of warp-wide accesses. However, such L1 cache architecture cannot always be efficiently utilized when applications generate many memory requests with irregular access patterns especially due to branch and memory divergences that make requests uncoalesced and small. Furthermore, unlike L1 cache, the shared memory of GPUs is not often used in many applications, which essentially depends on programmers. In this article, we propose Elastic-Cache, which can efficiently support both fine- and coarse-grained L1 cache line management for applications with both regular and irregular memory access patterns to improve the L1 cache efficiency. Specifically, it can store 32- or 64-byte words in non-contiguous memory space to a single 128-byte cache line. Furthermore, it neither requires an extra memory structure nor reduces the capacity of L1 cache for tag storage, since it stores auxiliary tags for fine-grained L1 cache line managements in the shared memory space that is not fully used in many applications. To improve the bandwidth utilization of L1 cache with Elastic-Cache for fine-grained accesses, we further propose Elastic-Plus to issue 32-byte memory requests in parallel, which can reduce the processing latency of memory instructions and improve the throughput of GPUs. Our experiment result shows that Elastic-Cache improves the geometric-mean performance of applications with irregular memory access patterns by 104% without degrading the performance of applications with regular memory access patterns. Elastic-Plus outperforms Elastic-Cache and improves the performance of applications with irregular memory access patterns by 131%.

Effect of number of processor on the cache hit rate in symmetric multiprocessor environment

Processor performs operations on operands faster than the access time of large capacity main memory causing bottleneck in a multiprocessing systems. A possible design alternative for improving the performance of a multiprocessor system is to insert a private cache between each processor and the shared memory.Caches in multiprocessing environment introduce the cache coherence problem. Different cache coherence protocols are developed to solve this problem by maintaining a uniform state for each cache block of data.The purpose of this paper is to investigate the influence of number of processor on the cache hit rate for the Snoopy Cache Coherence Protocols (MESI and Dragon) in Symmetric Multiprocessor environment.

Time-sensitivity-aware shared cache architecture for multi-core embedded systems

In embedded systems such as automotive systems, multi-core processors are expected to improve performance and reduce manufacturing cost by integrating multiple functions on a single chip. However, inter-core interference in shared last-level cache (LLC) results in increased and unpredictable execution times for time-sensitive tasks (TSTs), which have (soft) timing constraints, thereby increasing the deadline miss rates of such systems. In this paper, we propose a time-sensitivity-aware dead block-based shared LLC architecture to mitigate these problems. First, a time-sensitivity indication bit is added to each cache block, which allows the proposed LLC architecture to be aware of instructions/data belonging to TSTs. Second, portions of the LLC space are allocated to general tasks without interfering with TSTs by developing a time-sensitivity-aware dead block-based cache partitioning technique. Third, to reduce the deadline miss rate of TSTs further, we propose a task matching in shared caches and a cache partitioning scheme that considers the memory access characteristics and the time-sensitivity of tasks (TATS). The TATS is combined with our proposed dead block-based scheme. Our evaluation shows that the proposed schemes reduce deadline miss rates of TSTs compared to conventional shared caches. On a dual-core system, compared to a baseline, equal partitioning, and state-of-the-art quality-of-service-aware cache partitioning, our proposed dead block-based cache partitioning provides 9.3%, 30.5%, and 2.6% lower average deadline miss rates, respectively. On a quad-core system, compared to the baseline, equal partitioning, and state-of-the-art quality-of-service-aware cache partitioning, the combination of our proposed schemes provides 21.2%, 17.7%, and 4.1% lower average deadline miss rates, respectively.

Physical model of sensitive data leakage from PC-based cryptographic systems

In this paper, we present an examination of several side-channel attack scenarios on PC-based cryptosystems. Our goal was the development of a unified physical model for sensitive information leakage. The main focus of our work was electromagnetic side channels since signals with high signal-to-noise ratio (SNR) can be more conveniently captured. Moreover, the attacker can make correlations of the EM signal with other types of side-channel signals (such as voltage fluctuations and acoustic emanations). It shows that there may be a common source for a vulnerable signal that passes through several sides channels. We have simulated several attacks on targeted cryptosystems with distinct instruction sets. Trace analysis reveals empirical evidence. which corresponds to the theoretical principles of the mechanisms x86 and x64 processors. Hardware reasons for leakage come from the instructions and data in the processor cache, to be specific, from the fluctuations of power consumption, leading to changes in the voltage regulator of the processor. Thus, the fluctuations in LC circuits result in leakage on multiple side channels. In general, the obtained data together with the principles of signal formation can be used in vulnerability testing, which can examine side-channel robustness of cryptographic software on the first steps of development.

REACT: Read/Write Error Rate Aware Coding Technique for Emerging STT-MRAM Caches

Spin-transfer torque magnetic RAMs (STT-MRAMs) are the most promising alternative for static random-access memories in large last-level on-chip caches due to their higher density and near-zero leakage power. However, the reliability of STT-MRAMs is threatened by high probability of read disturbance and write failure . Both read disturbance and write failure, which cause a soft error in the cache cells, have an asymmetric behavior. Read disturbance occurs only in STT-MRAM cells storing “1” value, and write failure error rate in a $0\rightarrow 1$ transition is much higher than that in a $1\rightarrow 0$ transition. In this paper, we propose Read/write Error-rate Aware Coding Technique (REACT) to improve the reliability of the emerging STT-MRAM caches. REACT decreases the read disturbance and write failure rates by reducing the total number of “1”s and $0\rightarrow 1$ transitions on a cache block update. Our simulation results show that REACT reduces the probability of read disturbance and write failure up to 58% and 71%, respectively. These improvements are achieved by imposing negligible area, power, and performance overheads (less than 1%).

Energy-Efficient Runtime Adaptable L1 STT-RAM Cache Design

Much research has shown that applications have variable runtime cache requirements. In the context of the increasingly popular Spin-Transfer Torque RAM (STT-RAM) cache, the retention time, which defines how long the cache can retain a cache block in the absence of power, is one of the most important cache requirements that may vary for different applications. In this paper, we propose a Logically Adaptable Retention Time STT-RAM (LARS) cache that allows the retention time to be dynamically adapted to applications' runtime requirements. LARS cache comprises of multiple STT-RAM units with different retention times, with only one unit being used at a given time. LARS dynamically determines which STT-RAM unit to use during runtime, based on executing applications' needs. As an integral part of LARS, we also explore different algorithms to dynamically determine the best retention time based on different cache design tradeoffs. Our experiments show that by adapting the retention time to different applications' requirements, LARS cache can reduce the average cache energy by 25.31%, compared to prior work, with minimal overheads.

Cache lifetime enhancement technique using hybrid cache-replacement-policy

Driven by the trends of emerging technologies in memories, a non-volatile memory (NVM) has been considered as an alternate technology to replace SRAM in an on-chip cache. Spin-transfer torque random access memory (STT-RAM), a new type of NVM technology has low leakage power and huge density of cells. Besides having advantages, the emerging NVM technologies have limited writes and huge error rates. The write endurance and error rates are influenced by the cache replacement algorithm, which ultimately dictates the lifetime of a cache. Hence, it becomes necessary to develop or modify the cache replacement algorithms for improving the lifetime of the STT-RAM caches. The proposed Hybrid-Cache-Replacement (HCR) policy reduces the impact of the replacement algorithm in such a way that its write endurance improves with reduction in error rate. The incoming data are appropriately placed in the existing block, which have minimum error rates and less number of writes. It has been implemented by comparing the incoming bits with the existing bits in a cache set. The simulation results show that the lifespan of the STT-RAM caches improves by 135%, 165% and 27% along with 2%, 2% and 1% performance overhead when compared to the existing methods.