On-chip Cache Research Articles

IMPLEMENTED RECONFIGURABLE CACHE MEMORY ARCHITECTURE BASED ON 32-BITS MIPS PROCESSOR

This work presents the design of a reconfigurable cache memory utilizing a 32-bit MIPS processor, implemented through the utilization of VHDL (Very high-speed IC Hardware Description Language). A comprehensive implementation of a 32-bit, single-cycle MIPS processor is presented in VHDL. This processor is capable of supporting 50 instructions, comprising 25-R-type, 16-I-type, and 9-J-type instructions. The processor incorporates a three-dimensional reconfigurable cache memory architecture, enabling the modification of cache memory size, cache organization (in terms of associativity), and cache block size. The reconfigurable cache memory concept proposes the use of multi-cache controller units to execute reconfiguration operations and ensure that all permitted cache memory sets can be utilized with varying levels of associativity. The design is built on an FPGA using the Xilinx ISE design suite, which is based on the hardware description language. To ensure that assess functionality of the proposed reconfigurable cache memory design for various reconfiguration scenarios, numerous assembly language programs have been developed and performed on the MIPS processor. The results validated the efficacy of the suggested reconfigurable cache memory concept and demonstrated its simulation using the Xilinx ISim simulator.

Towards Analysing Cache-Related Preemption Delay in Non-Inclusive Cache Hierarchies

The impact of preemptions has to be considered when determining the schedulability of a task set in a preemptively scheduled system. In particular, the contents of caches can be disturbed by a preemption, thus creating context-switching costs. These context-switching costs occur when a preempted task needs to reload data from memory after a preemption. The additional delay created by this effect is termed cache-related preemption delay (CRPD). The analysis of CRPD has been extensively studied for single-level caches in the past. However, for two-level caches, the analysis of CRPD is still an emerging area of research. In contrast to a single-level cache, which is only affected by direct preemption effects, the second-level cache in a two-level hierarchy can be subject to indirect interference after a preemption. Accesses that could be served from the L1 cache in the absence of preemptions, may be forwarded to the L2 cache, as the relevant data was evicted by a preemption. These accesses create the indirect interference in the L2 cache and can cause further evictions. Recently, a CRPD analysis for two-level non-inclusive cache hierarchies was proposed. In this article, we show that this state-of-the-art analysis is unsafe as it potentially underestimates the CRPD. Furthermore, we show that the analysis is pessimistic and can overestimate the indirect preemption effects. To address these issues, we propose a novel analysis approach for the CRPD in a two-level non-inclusive cache hierarchy. We prove the correctness of the presented approach based on the set of feasible program execution traces. We implemented the presented approach in a worst-case execution time (WCET) analysis tool and compared the performance to existing analysis methods. Our evaluation shows that the presented analysis increases task set schedulability by up to 14 percentage points compared with the state-of-the-art analysis.

Enhancing the Lifetime of STT-RAM by IFTRP

Abstract Spin Transfer Torque Random Access memory (STT-RAM) is a better alternative to overcome the shortcomings of the existing memory technologies. But, the implementation of STT-RAM memory technologies on the existing memory are limited due to its write restriction. The disadvantages of STT-RAM have been overcome by many of the existing techniques. The existing techniques either reduce the writes or variation in the writes on the cache block in inter-set(IeS) or intra-set(IrS) by writing the data from hot block to cold block, which may lead to a Repeated Address Attack. A IFTRP (Inter-secure Fault Tolerant Replacement Policy) is proposed to reduce the attacks and have lifetime enhancement on the STT-RAM. The variation in writes is reduced by IFTRP in both IeS and IrS by RSS (Randomized Swap Shift policy), RICL(Randomized Invalidation Cache Line) method with a variation in the threshold values, so that the location of data blocks is not predictable by the intruder. The proposed method also identifies the partial cell failures by flagging them as invalid, so that the future write is not performed on the invalid line. The proposed method reduces the attacks and improves the lifetime of the memory by 15% in cache and has negligible area overhead.

GraNDe: Efficient Near-Data Processing Architecture for Graph Neural Networks

Graph Neural Network (GNN) models have attracted attention, given their high accuracy in interpreting graph data. One of the primary building blocks of a GNN model is aggregation, which gathers and averages the feature vectors corresponding to the nodes adjacent to each node. Aggregation works by multiplying the adjacency and feature matrices. The size of both matrices exceeds the on-chip cache capacity for many realistic datasets, and the adjacency matrix is highly sparse. These characteristics lead to little data reuse, causing intensive main-memory accesses during the aggregation process. Thus, aggregation exhibits memory-intensive characteristics and dominates most of the total execution time. In this paper, we propose GraNDe, an NDP architecture that accelerates memory-intensive aggregation operations by locating NDP modules near DRAM datapath to exploit rank-level parallelism. GraNDe maximizes bandwidth utilization by separating the memory channel path with the buffer chip in between so that pre-/post-processing in the host processor and reduction in NDP modules operate simultaneously. By exploring the preferred data mappings of the operand matrices to DRAM ranks, we architect GraNDe to support adaptive matrix mapping that applies the optimal mapping for each layer depending on the dimension of the layer and the configuration of a memory system. We also propose adj-bundle broadcasting and re-tiling optimizations to reduce the transfer time for adjacency matrix data and to improve feature vector data reusability by exploiting tiling with consideration of adjacency between nodes. GraNDe achieves 3.01× and 1.69× on average, and up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$4.00\times$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1.98\times$</tex-math></inline-formula> speedups of GCN aggregation over the baseline system and the state-of-the-art NDP architecture for GCN, respectively.

Hercules: Enabling Atomic Durability for Persistent Memory with Transient Persistence Domain

Persistent memory (pmem) products bring the persistence domain up to the memory level. Intel recently introduced the eADR feature that guarantees to flush data buffered in CPU cache to pmem on a power outage, thereby making the CPU cache a transient persistence domain . Researchers have explored how to enable the atomic durability for applications’ in-pmem data. In this article, we exploit the eADR-supported CPU cache to do so. A modified cache line, until written back to pmem, is a natural redo log copy of the in-pmem data. However, a write-back due to cache replacement or eADR on a crash overwrites the original copy. We accordingly developed Hercules, a hardware logging design for the transaction-level atomic durability, with supportive components installed in CPU cache, memory controller (MC), and pmem. When a transaction commits, Hercules commits on-chip its data staying in cache lines. For cache lines evicted before the commit, Hercules asks the MC to redirect and persist them into in-pmem log entries and commits them off-chip upon committing the transaction. Hercules lazily conducts pmem writes only for cache replacements at runtime. On a crash, Hercules saves metadata and data for active transactions into pmem for recovery. Experiments show that, by using CPU cache for both buffering and logging, Hercules yields much higher throughput and incurs significantly fewer pmem writes than state-of-the-art designs.

Using Hardware Events to Detect Side-Channel Attacks

The article discusses the monitoring system being developed by authors for the detection of Side-Channel Attacks, SCA. The theoretical justification of the chosen method of detecting SCA attacks is given: analysis of hardware events of the target system. The architecture of the monitoring system is described, including low-level data collection processes and an expert system that analyzes the collected data. The results of the proof-of-concept on various classes of SCA-attacks are presented. The paper considers a class of attacks on side-channels that use the processor cache to obtain secret information. A method of countering attacks of this class based on the use of a hardware event mechanism is proposed. As a result of the analysis of existing hardware events, predicates have been built for the Intel architecture, allowing one to identify the suspicious behavior of programs in the Linux OS environment. To test the proposed method, a prototype monitoring system was implemented that successfully coped with the detection of simulated SCA-attacks. The advantages and disadvantages of this method are considered, and the direction of further research is indicated.

Design of real-time & historical database for photovoltaic power plant

Abstract The real-time &amp; historical database of photovoltaic power plant monitoring system is used to store and manage the real-time and historical data, including power plant parameters, environmental data, and photovoltaic unit data and so on. Analyzing of this data is helpful for monitoring the status of the plant equipment, performing maintenance, and enhancing the operational efficiency of the power plant. The data models and database function of the database is designed. The data models include tags, data processing, and data storage models. Database function include the modules of the collection, storage, compression, and index query. The data collection module utilizes dynamic link libraries to access database data and configures parameters through a comprehensive data collection framework. The storage strategies involve in a combination of in-memory databases and disk-based historical databases, incorporating secondary caching technology and cloud server storage techniques. The data compression module is achieved through secondary compression methods. The index query module encompass time index files and the secondary index files to improve query speed and real-time system performance. The designed elements collectively establish an efficient monitoring system database.

To Cache or Not to Cache

Unlike conventional CPU caches, non-datapath caches, such as host-side flash caches which are extensively used as storage caches, have distinct requirements. While every cache miss results in a cache update in a conventional cache, non-datapath caches allow for the flexibility of selective caching, i.e., the option of not having to update the cache on each miss. We propose a new, generalized, bimodal caching algorithm, Fear Of Missing Out (FOMO), for managing non-datapath caches. Being generalized has the benefit of allowing any datapath cache replacement policy, such as LRU, ARC, or LIRS, to be augmented by FOMO to make these datapath caching algorithms better suited for non-datapath caches. Operating in two states, FOMO is selective—it selectively disables cache insertion and replacement depending on the learned behavior of the workload. FOMO is lightweight and tracks inexpensive metrics in order to identify these workload behaviors effectively. FOMO is evaluated using three different cache replacement policies against the current state-of-the-art non-datapath caching algorithms, using five different storage system workload repositories (totaling 176 workloads) for six different cache size configurations, each sized as a percentage of each workload’s footprint. Our extensive experimental analysis reveals that FOMO can improve upon other non-datapath caching algorithms across a range of production storage workloads, while also reducing the write rate.

Cross-Core Data Sharing for Energy-Efficient GPUs

Graphics Processing Units (GPUs) are the accelerator of choice in a variety of application domains because they can accelerate massively parallel workloads and can be easily programmed using general-purpose programming frameworks such as CUDA and OpenCL. Each Streaming Multiprocessor (SM) contains an L1 data cache (L1D) to exploit the locality in data accesses. L1D misses are costly for GPUs due to two reasons. First, L1D misses consume a lot of energy as they need to access the L2 cache (L2) via an on-chip network and the off-chip DRAM in case of L2 misses. Second, L1D misses impose performance overhead if the GPU does not have enough active warps to hide the long memory access latency. We observe that threads running on different SMs share 55% of the data they read from the memory. Unfortunately, as the L1Ds are in the non-coherent memory domain, each SM independently fetches data from the L2 or the off-chip memory into its L1D, even though the data may be currently available in the L1D of another SM. Our goal is to service L1D read misses via other SMs, as much as possible, to cut down costly accesses to the L2 or the off-chip DRAM. To this end, we propose a new data sharing mechanism, called Cross-Core Data Sharing (CCDS) . CCDS employs a predictor to estimate whether or not the required cache block exists in another SM. If the block is predicted to exist in another SM’s L1D, CCDS fetches the data from the L1D that contains the block. Our experiments on a suite of 26 workloads show that CCDS improves average energy and performance by 1.30 × and 1.20 ×, respectively, compared to the baseline GPU. Compared to the state-of-the-art data-sharing mechanism, CCDS improves average energy and performance by 1.37 × and 1.11 ×, respectively.

Optimizing CNN Hardware Acceleration with Configurable Vector Units and Feature Layout Strategies

Convolutional neural network (CNN) hardware acceleration is critical to improve the performance and facilitate the deployment of CNNs in edge applications. Due to its efficiency and simplicity, channel group parallelism has become a popular method for CNN hardware acceleration. However, when processing data involving small channels, there will be a mismatch between feature data and computing units, resulting in a low utilization of the computing units. When processing the middle layer of the convolutional neural network, the mismatch between the feature-usage order and the feature-loading order leads to a low input feature cache hit rate. To address these challenges, this paper proposes an innovative method inspired by data reordering technology, aiming to achieve CNN hardware acceleration that reuses the same multiplier resources. This method focuses on transforming the hardware acceleration process into feature organization, feature block scheduling and allocation, and feature calculation subtasks to ensure the efficient mapping of continuous loading and the calculation of feature data. Specifically, this paper introduces a convolutional algorithm mapping strategy and a configurable vector operation unit to enhance multiplier utilization for different feature map sizes and channel numbers. In addition, an off-chip address mapping and on-chip cache management mechanism is proposed to effectively improve the feature access efficiency and on-chip feature cache hit rate. Furthermore, a configurable feature block scheduling policy is proposed to strike a balance between weight reuse and feature writeback pressure. Experimental results demonstrate the effectiveness of this method. When using 512 multipliers and accelerating VGG16 at 100 MHz, the actual computing performance reaches 102.3 giga operations per second (GOPS). Compared with other CNN hardware acceleration methods, the average computing array utilization is as high as 99.88% and the computing density is higher.

Design of a Convolutional Neural Network Accelerator Based on On-Chip Data Reordering

Convolutional neural networks have been widely applied in the field of computer vision. In convolutional neural networks, convolution operations account for more than 90% of the total computational workload. The current mainstream approach to achieving high energy-efficient convolution operations is through dedicated hardware accelerators. Convolution operations involve a significant amount of weights and input feature data. Due to limited on-chip cache space in accelerators, there is a significant amount of off-chip DRAM memory access involved in the computation process. The latency of DRAM access is 20 times higher than that of SRAM, and the energy consumption of DRAM access is 100 times higher than that of multiply–accumulate (MAC) units. It is evident that the “memory wall” and “power wall” issues in neural network computation remain challenging. This paper presents the design of a hardware accelerator for convolutional neural networks. It employs a dataflow optimization strategy based on on-chip data reordering. This strategy improves on-chip data utilization and reduces the frequency of data exchanges between on-chip cache and off-chip DRAM. The experimental results indicate that compared to the accelerator without this strategy, it can reduce data exchange frequency by up to 82.9%.

A novel approximate cache block compressor for error-resilient image data

In this research, we introduce the Image Approximate Block Compressor (IABC), a fast (single cycle), simple and high-performance cache block compressor targeting domain-specific image data. Our work presents a high-quality cache block compression technique by applying approximation to image pixels used in selected error-resilient applications. IABC not only works seamlessly alongside mainstream block compression approaches including zero, frequent and partial patterns detection but also, due to introducing the approximation, improves their performance by increasing the probability of detecting the patterns. Having examined multiple variants of IABC, the proposed block compression with one-cycle decompression and two-cycle compression latency, we have considered a state-of-the-art algorithm, namely Base-Delta-Immediate (BΔI), and its modified approximate version that we call approximate BΔI, as our baselines. The evaluation reveals that IABC brings about a block compression ratio of 25.7 on average (up to 106) against BΔI, with an average ratio of 2.69 (up to 45.0) and the Approximate BΔI with an average ratio of 2.7 (up to 45.2). The evaluation results also show that the compression benefits of IABC come at only a 2.73% average error in the quality of a deep learning object recognition application. In addition, IABC generates high-quality outputs for stand-alone images with a 39.49 dB average Peak Signal to Noise Ratio (PSNR). The mentioned qualities come at only 13% storage overhead.

Advanced hybrid MRAM based novel GPU cache system for graphic processing with high efficiency

With the rapid development of portable computing devices and users’ demand for high-quality graphics rendering, embedded Graphics Processing Units (GPU) systems for graphics processing are increasingly turning into a key component of computer architecture to enhance computability. The cache system based on traditional static random access memory (SRAM) plays a crucial role in GPUs. But high leakage, low lifetime and poor integration problems deeply plague the science and engineering field. In the paper, a novel magnetic random access memory (MRAM) based cache architecture of GPU systems is proposed for highly efficient graphics processing and computing accelerating, with the merits of high speed, long endurance, strong interference resistance, and ultra-low power consumption. Spin transfer torque-MRAM and spin orbit torque-MRAM are utilized in off-chip and on-chip caches, respectively. A controller design scheme with prefetching modules and optimized cache coherency protocols are adopted. After testing and evaluating with multiple loads, neural network models and datasets, the simulation results show that the proposed system can achieve up to 28%, 56%, and 66.45% optimizations mostly in terms of speed, energy and leakage power, respectively.

Improving the Performance and Endurance of Persistent Memory with Loose-Ordering Consistency

Persistent memory provides high-performance data persistence at main memory. Memory writes need to be performed in strict order to satisfy storage consistency requirements and enable correct recovery from system crashes. Unfortunately, adhering to such a strict order significantly degrades system performance and persistent memory endurance. This paper introduces a new mechanism, Loose-Ordering Consistency (LOC), that satisfies the ordering requirements at significantly lower performance and endurance loss. LOC consists of two key techniques. First, Eager Commit eliminates the need to perform a persistent commit record write within a transaction. We do so by ensuring that we can determine the status of all committed transactions during recovery by storing necessary metadata information statically with blocks of data written to memory. Second, Speculative Persistence relaxes the write ordering between transactions by allowing writes to be speculatively written to persistent memory. A speculative write is made visible to software only after its associated transaction commits. To enable this, our mechanism supports the tracking of committed transaction ID and multi-versioning in the CPU cache. Our evaluations show that LOC reduces the average performance overhead of memory persistence from 66.9% to 34.9% and the memory write traffic overhead from 17.1% to 3.4% on a variety of workloads.

Novel CPU cache architecture based on two-dimensional MTJ device with ferromagnetic Fe3GeTe2

With the development of Artificial Intelligence (AI) in recent years, the fields of computer, biology, medicine, and aerospace have demanded higher requirements for the processing and storage of information. In this paper, a novel Magnetic Tunnel Junction (MTJ) based Spin-Orbital Torque Magnetic Random Access Memory (SOT-MRAM) composed of Fe3GeTe2 (FGT) is employed as a storage medium in the computer architecture. On the basis of the analysis of the fundamentals, model configuration, characteristics and performance advantages of the FGT based SOT device, a hybrid storage (L1, L2, Last Level Cache) is constructed, with FGT-SOT-MRAM, conventional SOT-MRAM and STT-MRAM replacing the original static random access memory (SRAM) in the novel triple-level CPU cache architecture. This can override the increasing leakage problem of SRAM, while opening up the application of two-dimensional van der Waals ferromagnets in computer systems at the L1 cache level. Meanwhile, an innovative cache optimization scheme is put forward for magnetic memory to better match the performance of FGT-SOT-MRAM to CPU. The simulation results demonstrate that the FGT-based MRAM can achieve up to 38.03% IPC optimization and 53.41% power optimization in the CPU cache system in contrast to the conventional ones.

Diagonally‐Addressed Matrix Nicknack: How to improve SpMV performance

AbstractWe suggest a technique to reduce the storage size of sparse matrices at no loss of information. We call this technique Diagonally‐Addressed (DA) storage. It exploits the typically low matrix bandwidth of matrices arising in applications. For memory‐bound algorithms, this traffic reduction has direct benefits for both uni‐precision and multi‐precision algorithms. In particular, we demonstrate how to apply DA storage to the Compressed Sparse Rows (CSR) format and compare the performance in computing the Sparse Matrix Vector (SpMV) product, which is a basic building block of many iterative algorithms. We investigate 1367 matrices from the SuiteSparse Matrix Collection fitting into the CSR format using signed 32 bit indices. More than 95% of these matrices fit into the DA‐CSR format using 16 bit column indices, potentially after Reverse Cuthill‐McKee (RCM) reordering. Using IEEE 754 precision scalars, we observe a performance uplift of 11% (single‐threaded) or 17.5% (multithreaded) on average when the traffic exceeds the size of the last‐level CPU cache. The predicted uplift in this scenario is 20%. For traffic within the CPU's combined level 2 and level 3 caches, the multithreaded performance uplift is over 40% for a few test matrices.

Improving Cache Hits On Replacment Blocks Using Weighted LRU-LFU Combinations

Block replacement refers to the process of selecting a block of data or a cache line to be evicted or replaced when a new block needs to be brought into a cache or a memory hierarchy. In computer systems, block replacement policies are used in caching mechanisms, such as in CPU caches or disk caches, to determine which blocks are evicted when the cache is full and new data needs to be fetched. The combination of LRU (Least Recently Used) and LFU (Least Frequently Used) in a weighted manner is known as the "LFU2" algorithm. LFU2 is an enhanced caching algorithm that aims to leverage the benefits of both LRU and LFU by considering both recency and frequency of item access. In LFU2, each item in the cache is associated with two counters: the usage counter and the recency counter. The usage counter tracks the frequency of item access, while the recency counter tracks the recency of item access. These counters are used to calculate a combined weight for each item in the cache. Based on the experimental results, the LRU-LFU combination method succeeded in increasing cache hits from 94.8% on LFU and 95.5% on LFU to 96.6%.

The Art of Latency Hiding in Modern Database Engines

Modern database engines must well use multicore CPUs, large main memory and fast storage devices to achieve high performance. A common theme is hiding latencies such that more CPU cycles can be dedicated to "real" work, improving overall throughput. Yet existing systems are only able to mitigate the impact of individual latencies, e.g., by interleaving memory accesses with computation to hide CPU cache misses. They still lack the joint optimization of hiding the impact of multiple latency sources. This paper presents MosaicDB, a set of latency-hiding techniques to solve this problem. With stackless coroutines and carefully crafted scheduling policies, we explore how I/O and synchronization latencies can be hidden in a well-crafted OLTP engine that already hides memory access latency, without hurting the performance of memory-resident workloads. MosaicDB also avoids oversubscription and reduces contention using the coroutine-to-transaction paradigm. Our evaluation shows MosaicDB can achieve these goals and up to 33x speedup over prior state-of-the-art.

On-Chip Cache Procedure and Device for Efficient CPU and GPU Access Request Arbitration

On-Chip Cache Procedure and Device for Efficient CPU and GPU Access Request Arbitration

ZPP: A Dynamic Technique to Eliminate Cache Pollution in NoC based MPSoCs

Data prefetching efficiently reduces the memory access latency in NUCA architectures as the Last Level Cache (LLC) is shared and distributed across multiple cores. But cache pollution generated by prefetcher reduces its efficiency by causing contention for shared resources such as LLC and the underlying network. The paper proposes Zero Pollution Prefetcher (ZPP) that eliminates cache pollution for NUCA architecture. For this purpose, ZPP uses L1 prefetcher and places the prefetched blocks in the data locations of LLC where modified blocks are stored. Since modified blocks in LLC are stale and request for such blocks are served from the exclusively owned private cache, their space unnecessary consumes power to maintain such stale data in the cache. The benefits of ZPP are (a) Eliminates cache pollution in L1 and LLC by storing prefetched blocks in LLC locations where stale blocks are stored. (b) Insufficient cache space is solved by placing prefetched blocks in LLC as LLCs are larger in size than L1 cache. This helps in prefetching more cache blocks, thereby increasing prefetch aggressiveness. (c) Increasing prefetch aggressiveness increases its coverage. (d) It also maintains an equivalent lookup latency to L1 cache for prefetched blocks. Experimentally it has been found that ZPP increases weighted speedup by 2.19x as compared to a system with no prefetching while prefetch coverage and prefetch accuracy increases by 50%, and 12%, respectively compared to the baseline. 1