On-chip Cache Research Articles

The Yellow Active Queue Management Algorithm in ICN Routers Based on the Monitoring of Bandwidth Competition

Deploying the active queue management (AQM) algorithm on a router is an effective way to avoid packet loss caused by congestion. In an information-centric network (ICN), routers not only play a role of packets forwarding but are also content service providers. Congestion in ICN routers can be further summarized as the competition between the external forwarding traffic and the internal cache response traffic for limited bandwidth resources. This indicates that the traditional AQM needs to be redesigned to adapt to ICN. In this paper, we first demonstrated mathematically that allocating more bandwidth for the upstream forwarding flow could improve the quality of service (QoS) of the whole network. Secondly, we propose a novel AQM algorithm, YELLOW, which predicts the bandwidth competition event and adjusts the input rate of request and the marking probability adaptively. Afterwards, we model YELLOW through the totally asymmetric simple exclusion process (TASEP) and deduce the approximate solution of the existence condition for each stationary phase. Finally, we evaluated the performance of YELLOW by NS-3 simulator, and verified the accuracy of modeling results by Monte Carlo. The simulation results showed that the queue of YELLOW could converge to the expected value, and the significant gains of the router with low packet loss rate, robustness and high throughput.

New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs

On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance by reducing long memory access latency. However, cache efficiency remains low despite the facts that recent GPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is a reasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, we introduce a second key determinant to overcome the problem of predicting the performance gains from L1 data cache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefits of the cache by measuring the balance between cache efficiency and throughput. The throughput of the cache is predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used for a second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 data cache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing technique can be applied to recent GPU models and improves the performance by 6% on average compared to the architecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassing techniques.

Revising NoC in Future Multicore-Based Consumer Electronics for Performance

With the increasing demand for efficient data processing, the latest consumer electronics are powered by multi-core processors. Many applications that run on these devices expect minimum data access latency, where Network-on-Chip (NoC) plays a crucial role. However, NoC is found to be responsible for 60-75% of miss penalty. The objective of this article is to present a revised NoC communication framework that can possibly reduce the miss penalty. The proposed architecture orchestrates buffers of NoC routers as a victim cache to store evicted cache blocks. Future references to such stored blocks are replied from NoC routers which significantly reduces miss penalty and improves system performance. Minimising miss penalty by maximising NoC resource utilisation in these devices is a win-win situation.

Agglomerative Memory and Thread Scheduling for High-Performance Ray-Tracing on GPUs

Ray-tracing rendering has long been considered as a promising technology to enable a higher level of visual experience. The democratization of the ray-tracing rendering to consumer platforms, however, poses significant challenges to rendering hardware and software due to its highly irregular computing patterns. In fact, modern ray-tracing techniques typically depend on a tree-based acceleration structure to reduce the computing complexity of intersection testing of rays and graphics primitives. The traversal by a massive number of rays on a graphics processing unit (GPU) incurs a significant amount of irregular memory traffic, which turns out to be a major stumbling block for real-time performance. In this work, a scheduling mechanism, so-called Agglomerative Memory and Thread Scheduling, is proposed to unleash the inherence parallelism in the ray-tracing process on GPUs. It is associated with a tile-based ray-tracing framework in which the acceleration structure (i.e., KD-tree in this work) is partitioned into subtrees that can be completely loaded into the on-chip L1 cache inside a streaming multiprocessor. An effective scheduling mechanism collects threads with regard to the subtrees hit by their respective rays and regroup threads into warps for dispatching. In addition, subtrees are dynamically preloaded into the L1 cache of multiprocessors in an on-demand fashion. The proposed scheduler can be integrated on today’s high-end GPUs with only minor overhead. Microarchitecture simulation results prove that the proposed framework significantly improves memory efficiency and outperforms a traditional GPU microarchitecture by 47.4% for average.

Cache-efficient sweeping-based interval joins for extended Allen relation predicates

We develop a family of efficient plane-sweeping interval join algorithms for evaluating a wide range of interval predicates such as Allen’s relationships and parameterized relationships. Our technique is based on a framework, components of which can be flexibly combined in different manners to support the required interval relation. In temporal databases, our algorithms can exploit a well-known and flexible access method, the Timeline Index, thus expanding the set of operations it supports even further. Additionally, employing a compact data structure, the gapless hash map, we utilize the CPU cache efficiently. In an experimental evaluation, we show that our approach is several times faster and scales better than state-of-the-art techniques, while being much better suited for real-time event processing.

Observing the Invisible: Live Cache Inspection for High-Performance Embedded Systems

The vast majority of high-performance embedded systems implement multi-level CPU cache hierarchies. But the exact behavior of these CPU caches has historically been opaque to system designers. Absent expensive hardware debuggers, an understanding of cache makeup remains tenuous at best. This enduring opacity further obscures the complex interplay among applications and OS-level components, particularly as they compete for the allocation of cache resources. Notwithstanding the relegation of cache comprehension to proxies such as static cache analysis, performance counter-based profiling, and cache hierarchy simulations, the underpinnings of cache structure and evolution continue to elude software-centric solutions. In this paper, we explore a novel method of studying cache contents and their evolution via snapshotting. Our method complements extant approaches for cache profiling to better formulate, validate, and refine hypotheses on the behavior of modern caches. We leverage cache introspection interfaces provided by vendors to perform live cache inspections without the need for external hardware. We present CacheFlow, a proof-of-concept Linux kernel module which snapshots cache contents on an NVIDIA Tegra TX1 SoC (system on chip).

Working Set Analytics

The working set model for program behavior was invented in 1965. It has stood the test of time in virtual memory management for over 50 years. It is considered the ideal for managing memory in operating systems and caches. Its superior performance was based on the principle of locality, which was discovered at the same time; locality is the observed tendency of programs to use distinct subsets of their pages over extended periods of time. This tutorial traces the development of working set theory from its origins to the present day. We will discuss the principle of locality and its experimental verification. We will show why working set memory management resists thrashing and generates near-optimal system throughput. We will present the powerful, linear-time algorithms for computing working set statistics and applying them to the design of memory systems. We will debunk several myths about locality and the performance of memory systems. We will conclude with a discussion of the application of the working set model in parallel systems, modern shared CPU caches, network edge caches, and inventory and logistics management.

MemCAM: A Hybrid Memristor-CMOS CAM Cell for On-Chip Caches

Non-volatile nanoscale memory devices (such as memristors) have promised to overcome the challenges of scalability and leakage currents of CMOS based memory devices. These novel memories can be fabricated in back-end-of-the-line of any CMOS process. Currently, a lot of research is focused on investigating the benefits of memristors for associative memories. These are Content-Addressable Memories (CAM) in which search based data access takes place. Searching for a particular bit in memristor is time consuming while search in CMOS CAM zone is efficient. To combine the speed and ease of search of CMOS memory and the scalability of memristor memory, we present a novel multibit hybrid CMOS-Memristor Associative Memory Cell. The benefits of such memory cells manifest in on-chip caches - the instruction and data cache, Branch Target Buffer, and Translation Lookaside Buffer. To exemplify the benefit of the cell further, we also simulate the MemCAM as the TLB of an ARM processor and obtained upto 50% decrease in miss rates of Data TLB and upto 93% in that of Instruction TLB. Average speedup of 1.16 was also achieved on various benchmark applications of PARCSEC and MiBench suites.

DAM: Deadblock Aware Migration Techniques for STT-RAM-Based Hybrid Caches

Last Level Caches (LLCs) play a critical role in reducing the number of costly off-chip memory accesses. Hence, there is a demand to make the LLCs larger to meet the requirements of growing dataset size of emerging data-intensive applications. Non-volatile STT-RAM-based caches with high memory density are a promising alternative to satisfy such a demand. However, STT-RAM-based caches have high write-latency and write energy. Therefore, researchers proposed a unified cache consisting of both SRAM and STT-RAM portions to handle expensive writes in these large hybrid LLCs. However, these large LRU-managed LLCs still remain underutilized due to the presence of a significant fraction of deadblocks (zero-reuse blocks) in them. By choosing the deadblock as an eviction candidate, we show a significant reduction in the LLC misses. Our work considers the entire set of the hybrid cache to select the deadblock as an eviction candidate. Therefore, we propose Deadblock-based Victim Selection (DVS), a deadblock-aware replacement policy. We observe that in up to 40 percent of cases, evicting a deadblock on an LLC miss requires migration of a cache block within the cache set. To incorporate this opportunity into DVS, we augment it with a cache block migration strategy and propose Deadblock-based Migration (DM) policy. The detailed evaluation on the SPEC CPU2006 workloads shows that DVS and DM enhance the performance of the system by up to 13 and 52 percent, respectively. Our results demonstrate that a cache replacement policy that is both deadblock-aware and migration-aware is essential in improving the performance of large hybrid LLCs.

Energy-Efficient Shared Cache Using Way Prediction Based on Way Access Dominance Detection

To meet the performance demands of chip multi-processors, chip designers have increased the capacity and hierarchy of cache memories. Accordingly, a shared lower-level cache reduces conflict misses by adopting a multi-way set-associative structure with high associativity. This structure allows fast access because it allows access to all the ways in the cache set in parallel. However, it consumes a large amount of dynamic energy. Therefore, various schemes have been proposed to increase the cache memory energy efficiency. These schemes use way prediction or partial comparison to reduce unnecessary way accesses. This paper proposes a way prediction algorithm suitable to a shared second-level cache with high associativity. This algorithm is based on the real-time way access dominance detection (WADD). Through this detection, the proposed algorithm can determine the number and location of way candidates suitable for each partial access pattern among the fragmented access patterns due to the first-level cache replacement policy and intermingled accesses by multiple cores. Through this process, the proposed algorithm can implement efficient way prediction. Simulation results show that the WADD exhibits the highest energy efficiency among the comparison group, thus reducing the energy-delay product by 13.5% compared with the conventional cache without way prediction. This result is achieved by reducing the way prediction penalty through fast detection and high prediction accuracy.

3RSeT: Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison

Recent development in memory technologies has introduced Spin-Transfer Torque Magnetic RAM (STT-MRAM) as the most promising replacement for SRAMs in on-chip cache memories. Besides its lower leakage power, higher density, immunity to radiation-induced particles, and non-volatility, an unintentional bit flip during read operation, referred to as read disturbance error, is a severe reliability challenge in STT-MRAM caches. One major source of read disturbance error in STT-MRAM caches is simultaneous accesses to all tags for parallel comparison operation in a cache set, which has not been addressed in previous work. This paper first demonstrates that high read accesses to tag arrays extremely increase the read disturbance rate and then proposes a low-cost scheme, so-called Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison (3RSeT), to reduce the error rate by eliminating a significant portion of tag reads. 3RSeT proactively disables the tags that have no chance for hit, using low significant bits of the tags on each access request. Our evaluations using gem5 full-system cycle-accurate simulator show that 3RSeT reduces the read disturbance rate in the tag array by 71.8%, which results in 3.6x improvement in Mean Time To Failure. In addition, the energy consumption is reduced by 62.1% without compromising performance and with less than 0.4% area overhead.

Evaluation of Cache Attacks on Arm Processors and Secure Caches

Timing-based side and covert channels in processor caches continue to be a threat to modern computers. This work shows for the first time, a systematic, large-scale analysis of Arm devices and the detailed results of attacks the processors are vulnerable to. Compared to x86, Arm uses different architectures, microarchitectural implementations, cache replacement policies, etc., which affects how attacks can be launched, and how security testing for the vulnerabilities should be done. To evaluate security, this paper presents security benchmarks specifically developed for testing Arm processors and their caches. The benchmarks are evaluated with sensitivity tests, which examine how sensitive the benchmarks are to having a correct configuration in the testing phase. Further, to evaluate a large number of devices, this work leverages a novel approach of using a cloud-based Arm device testbed for architectural and security research on timing channels and runs the benchmarks on 34 different physical devices. In parallel, there has been much interest in secure caches to defend the various attacks. Consequently, this paper also investigates secure cache architectures using proposed benchmarks. Especially, this paper implements and evaluates secure PL and RF caches, showing the security of PL and RF caches, but also uncovers new weaknesses.

Alleviating I/O Interference in Virtualized Systems With VM-Aware Persistency Control

Consolidating multiple servers into a physical machine is now a commonplace in cloud infrastructures. The virtualized systems often arrange virtual disks of multiple virtual machines (VMs) on the same underlying storage device while striving to guarantee the service level objective (SLO) of the performance of each VM. Unfortunately, sync operations called by a VM may make it hard to satisfy the performance SLO by disturbing I/O activities of other VMs. In this paper, we experimentally uncover that the disk cache flush operation incurs significant I/O interference among VMs, and revisit the internal architecture and flush mechanism of the flash memory-based SSD. Then, we present vFLUSH, a novel VM-aware flush mechanism, that supports VM-based persistency control for the disk cache flush operation. We also discuss the long-tail latency issue in vFLUSH and an efficient scheme for mitigating the problem. Our evaluation with various micro- and macro-benchmarks shows that vFLUSH reduces the average latency of disk cache flush operations by up to 58.5%, thereby producing improvements in throughput by up to 1.93×. The method for alleviating the long-tail latency problem, which is applied to vFLUSH, achieves a significant reduction in tail latency by up to 75.9%, with a modest throughput degradation by 2.9-7.2%.

Analysis and Optimization of Persistent Memory Index Structures’ Write Amplification

Using persistent memory (PM) for an in-memory database (IMDB) system significantly extends data processing capacity and reduces service downtime for the recovery process in case of system failure. However, it requires an index structure tailored for the characteristics of PM to recover the consistent state from system failure. The processor flushes data with the cache line size to the memory controller, whereas the PM media writes with the media-write unit size, which is four times larger than the cache line size. This mismatch in write sizes amplifies the quantity of data written by an index structure at the physical media level. Additionally, the PM-aware index structures explicitly and frequently flush cache lines to ensure memory write orders, which worsen the write amplification factor. The write amplification not only degrades the performance but also shortens the lifespan of PM media. This study analyzes the write amplification and performance of various PM-aware index structures at the memory controller and physical media levels. Furthermore, this paper proposes four optimization techniques based on our observation: removal of unintended bloat, cache flush coalescing, early eviction of volatile data, and XPLine-aware unit sizing. Our analysis demonstrated that up to 3.9 times as many writes could occur depending on the choice of the index structure. We also revealed that the proposed optimization techniques could save up to 51% of physical media write operations and improve throughput by up to 63%.

Improving the Performance of Deduplication-Based Storage Cache via Content-Driven Cache Management Methods

Data deduplication, as a proven technology for effective data reduction in backup and archiving storage systems, is also showing promises in increasing the logical space capacity for storage caches by removing redundant data. However, our in-depth evaluation of the existing deduplication-aware caching algorithms reveals that they only work well when the cached block size is set to 4 KB. Unfortunately, modern storage systems often set the block size to be much larger than 4 KB, and in this scenario, the overall performance of these caching schemes drops below that of the conventional replacement algorithms without any deduplication. There are several reasons for this performance degradation. The first reason is the deduplication overhead, which is the time spent on generating the data fingerprints and their use to identify duplicate data. Such overhead offsets the benefits of deduplication. The second reason is the extremely low cache space utilization caused by read and write alignment. The third reason is that existing algorithms only exploit access locality to identify block replacement. There is a lost opportunity to effectively leverage the content usage patterns such as intensity of content redundancy and sharing in deduplication-based storage caches to further improve performance. We propose CDAC, a Content-driven Deduplication-Aware Cache, to address this problem. CDAC focuses on exploiting the content redundancy in blocks and intensity of content sharing among source addresses in cache management strategies. We have implemented CDAC based on LRU and ARC algorithms, called CDAC-LRU and CDAC-ARC respectively. Our extensive experimental results show that CDAC-LRU and CDAC-ARC outperform the state-of-the-art deduplication-aware caching algorithms, D-LRU, and D-ARC, by up to 23.83X in read cache hit ratio, with an average of 3.23X, and up to 53.3 percent in IOPS, with an average of 49.8 percent, under a real-world mixed workload when the cache size ranges from 20 to 50 percent of the workload size and the block size ranges from 4KB to 32 KB.

COPE

Prefetching helps in reducing the memory access latency in multi-banked NUCA architecture, where the Last Level Cache (LLC) is shared. In such systems, an application running on core generates significant traffic on the shared resources, the underlying network and LLC. While prefetching helps to increase application performance, but an inaccurate prefetcher can cause harm by generating unwanted traffic that additionally increases network and LLC contention. Increased network contention results in untimely prefetching of cache blocks, thereby reducing the effectiveness of a prefetcher. Prefetch accuracy is extensively used to reduce unwanted prefetches that can mitigate the prefetcher caused contention. However, the conventional prefetch accuracy parameter has major limitations in NUCA architectures. The article exposes that prefetch accuracy can create two major false-positive cases of prefetching, Under-estimation and Over-estimation problems, and false feedback loop that can mislead a prefetcher in generating more unwanted traffic. We propose a novel technique, Coordinated Prefetching for Efficient (COPE), which addresses these issues by redefining prefetch accuracy for such architectures and identifies additional parameters that can avoid generating unwanted prefetch requests. Experiment conducted using PARSEC benchmark on a 64-core system shows that COPE achieve 3% reduction in L1 cache miss rate, 12.64% improvement in IPC, 23.2% reduction in average packet latency and 18.56% reduction in dynamic power consumption of the underlying network.

Efficient Nearest-Neighbor Data Sharing in GPUs

Stencil codes (a.k.a. nearest-neighbor computations) are widely used in image processing, machine learning, and scientific applications. Stencil codes incur nearest-neighbor data exchange because the value of each point in the structured grid is calculated as a function of its value and the values of a subset of its nearest-neighbor points. When running on Graphics Processing Unit (GPUs), stencil codes exhibit a high degree of data sharing between nearest-neighbor threads. Sharing is typically implemented through shared memories, shuffle instructions, and on-chip caches and often incurs performance overheads due to the redundancy in memory accesses. In this article, we propose Neighbor Data (NeDa), a direct nearest-neighbor data sharing mechanism that uses two registers embedded in each streaming processor (SP) that can be accessed by nearest-neighbor SP cores. The registers are compiler-allocated and serve as a data exchange mechanism to eliminate nearest-neighbor shared accesses. NeDa is embedded carefully with local wires between SP cores so as to minimize the impact on density. We place and route NeDa in an open-source GPU and show a small area overhead of 1.3%. The cycle-accurate simulation indicates an average performance improvement of 21.8% and power reduction of up to 18.3% for stencil codes in General-Purpose Graphics Processing Unit (GPGPU) standard benchmark suites. We show that NeDa’s performance is within 13.2% of an ideal GPU with no overhead for nearest-neighbor data exchange.

WITHDRAWN: On-chip cache memory protection with tag overflow buffers and VLSI implementation

A increasing barrier in the development of the future generation of reliable microprocessors has secure memory on chip against soft bugs.Present efforts primarily focused on improving cache data set performance.A tag array also needs massive production efficiency for soft factual errors for its vital significance for caching entry accuracy.Using the memory access address location, it is recommended to repeat the most recent access to tag documents in the tiny tag duplication buffer (TRB) in order to maintain the tag list skill integrity in the cache memory.

Design Space Exploration of High-Performance Parallel Architectures

High-performance single-threaded processors achieve their performance goal partly by relying, among other architectural techniques, on speculation and large on-chip caches. The hardware to support these techniques is usually a large portion of the overall processor real state area, and therefore it consumes a significant amount of power that sometimes is not optimally used toward doing useful work. In this work, we study the intuitive fact that architectures with hardware support for threads are more power efficient than a more traditional single-threaded superscalar architecture. Toward this goal, we have created a model of the power, performance and area of several parallel architectures. This model shows that a parallel architecture can be designed so that (a) it requires less area and power (to reach the same performance), or (b) it achieves better power efficiency and less area (for the same power budget), or (c) it has higher performance and better power efficiency (for the same area constraint), when compared to a single-threaded superscalar architecture.

Comparative Analysis of Spintronic Memories for Low Power on-chip Caches

The continuous downscaling in CMOS devices has increased leakage power and limited the performance to a few GHz. The research goal has diverted from operating at high frequencies to deliver higher performance in essence with lower power. CMOS based on-chip memories consumes significant fraction of power in modern processors. This paper aims to explore the suitability of beyond CMOS, emerging magnetic memories for the use in memory hierarchy, attributing to their remarkable features like nonvolatility, high-density, ultra-low leakage and scalability. NVSim, a circuit-level tool, is used to explore different design layouts and memory organizations and then estimate the energy, area and latency performance numbers. A detailed system-level performance analysis of STT-MRAM and SOT-MRAM technologies and comparison with 22[Formula: see text]nm SRAM technology are presented. Analysis infers that in comparison to the existing 22[Formula: see text]nm SRAM technology, SOT-MRAM is more efficient in area for memory size [Formula: see text][Formula: see text]KB, speed and energy consumption for cache size [Formula: see text][Formula: see text]KB. A typical 256[Formula: see text]KB SOT-MRAM cache design is 27.74% area efficient, 2.97 times faster and consumes 76.05% lesser leakage than SRAM counterpart and these numbers improve for larger cache sizes. The article deduces that SOT-MRAM technology has a promising potential to replace SRAM in lower levels of computer memory hierarchy.