Level Cache Research Articles

A Power-Aware Hybrid Cache for Chip-Multi Processors Based on Neural Network Prediction Technique

The increasing need to run applications for significant data analytics, and the augmented demand of useful tools for big data computing systems has resulted in a cumulative necessity for efficient platforms with high performance and realizable power consumption, for example, chip multiprocessors. Correspondingly, due to the demand for features like shrinkable sizes, and the concurrent need to pack increasing numbers of transistors into a single chip, has led to serious design challenges, consuming a significant of power within high area densities. We present a reconfigurable hybrid cache system for last level cache by the integration of emerging designs, such as STT-RAM with SRAM memories. This approach consists of two phases: off-time and on-time. In off time, training NN is implemented while in the on-time phase, a reconfiguration cache uses a neural network learning approach to predict demanded latency of the running application. Experimental results of a three-dimensional chip with 64 cores show that the suggested design under PARSEC benchmarks provides a speedup in terms of the performance at 25% and improves energy consumption by 78.4% in comparison to non-reconfigurable pure SRAM cache architectures.

Improving in-memory file system reading performance by fine-grained user-space cache mechanisms

Nowadays, as the memory capacity of servers become larger and larger, distributed in-memory file systems, which enable applications to interact with data at fast speed, have been widely used. However, the existing distributed in-memory file systems still face the problem of low data access performance in small data reading, which seriously reduce their usefulness in many important big data scenarios. In this paper, we analyze the factors that affect the performance of reading in-memory files and propose a two-layer user space cache management mechanism: in the first layer, we cache data packet references to reduce frequent page fault interruptions (packet-level cache); in the second layer, we cache and manage small file data units to avoid redundant inter-process communications (object-level cache). We further design a fine-grained caching model based on the submodular function optimization theory, for efficiently managing the variable-length cache units with partially overlapping fragments on the client side. Experimental results on synthetic and real-world workloads show that compared with the existing cutting-edge systems, the first level cache can double the reading performance on average, and the second level cache can improve random reading performance by more than 4 times. Our caching strategies also outperform the cutting-edge cache algorithms over 20% on hit ratio. Furthermore, the proposed client-side caching framework idea has been adopted by the Alluxio open source community, which shows the practical benefits of this work. • A packet-level cache policy for reducing page faults during sequential data reading. • An in-memory cache layer to cache the variable-length hot file fragments. • Proved that the variable-length cache problem is a submodular optimization problem theoretically. • Comprehensive performance evaluation of the client-side cache strategies.

DAM: Deadblock Aware Migration Techniques for STT-RAM-Based Hybrid Caches

Last Level Caches (LLCs) play a critical role in reducing the number of costly off-chip memory accesses. Hence, there is a demand to make the LLCs larger to meet the requirements of growing dataset size of emerging data-intensive applications. Non-volatile STT-RAM-based caches with high memory density are a promising alternative to satisfy such a demand. However, STT-RAM-based caches have high write-latency and write energy. Therefore, researchers proposed a unified cache consisting of both SRAM and STT-RAM portions to handle expensive writes in these large hybrid LLCs. However, these large LRU-managed LLCs still remain underutilized due to the presence of a significant fraction of deadblocks (zero-reuse blocks) in them. By choosing the deadblock as an eviction candidate, we show a significant reduction in the LLC misses. Our work considers the entire set of the hybrid cache to select the deadblock as an eviction candidate. Therefore, we propose Deadblock-based Victim Selection (DVS), a deadblock-aware replacement policy. We observe that in up to 40 percent of cases, evicting a deadblock on an LLC miss requires migration of a cache block within the cache set. To incorporate this opportunity into DVS, we augment it with a cache block migration strategy and propose Deadblock-based Migration (DM) policy. The detailed evaluation on the SPEC CPU2006 workloads shows that DVS and DM enhance the performance of the system by up to 13 and 52 percent, respectively. Our results demonstrate that a cache replacement policy that is both deadblock-aware and migration-aware is essential in improving the performance of large hybrid LLCs.

COPE

Prefetching helps in reducing the memory access latency in multi-banked NUCA architecture, where the Last Level Cache (LLC) is shared. In such systems, an application running on core generates significant traffic on the shared resources, the underlying network and LLC. While prefetching helps to increase application performance, but an inaccurate prefetcher can cause harm by generating unwanted traffic that additionally increases network and LLC contention. Increased network contention results in untimely prefetching of cache blocks, thereby reducing the effectiveness of a prefetcher. Prefetch accuracy is extensively used to reduce unwanted prefetches that can mitigate the prefetcher caused contention. However, the conventional prefetch accuracy parameter has major limitations in NUCA architectures. The article exposes that prefetch accuracy can create two major false-positive cases of prefetching, Under-estimation and Over-estimation problems, and false feedback loop that can mislead a prefetcher in generating more unwanted traffic. We propose a novel technique, Coordinated Prefetching for Efficient (COPE), which addresses these issues by redefining prefetch accuracy for such architectures and identifies additional parameters that can avoid generating unwanted prefetch requests. Experiment conducted using PARSEC benchmark on a 64-core system shows that COPE achieve 3% reduction in L1 cache miss rate, 12.64% improvement in IPC, 23.2% reduction in average packet latency and 18.56% reduction in dynamic power consumption of the underlying network.

Online Thread and Data Mapping Using a Sharing-Aware Memory Management Unit

Current and future architectures rely on thread-level parallelism to sustain performance growth. These architectures have introduced a complex memory hierarchy, consisting of several cores organized hierarchically with multiple cache levels and NUMA nodes. These memory hierarchies can have an impact on the performance and energy efficiency of parallel applications as the importance of memory access locality is increased. In order to improve locality, the analysis of the memory access behavior of parallel applications is critical for mapping threads and data. Nevertheless, most previous work relies on indirect information about the memory accesses, or does not combine thread and data mapping, resulting in less accurate mappings. In this paper, we propose the Sharing-Aware Memory Management Unit (SAMMU), an extension to the memory management unit that allows it to detect the memory access behavior in hardware. With this information, the operating system can perform online mapping without any previous knowledge about the behavior of the application. In the evaluation with a wide range of parallel applications (NAS Parallel Benchmarks and PARSEC Benchmark Suite), performance was improved by up to 35.7% (10.0% on average) and energy efficiency was improved by up to 11.9% (4.1% on average). These improvements happened due to a substantial reduction of cache misses and interconnection traffic.

Performance-Energy Trade-off in Modern CMPs

Chip multiprocessors (CMPs) are ubiquitous in all computing systems ranging from high-end servers to mobile devices. In these systems, energy consumption is a critical design constraint as it constitutes the most significant operating cost for computing clouds. Analogous to this, longer battery life continues to be an essential user concern in mobile devices. To optimize on power consumption, modern processors are designed with Dynamic Voltage and Frequency Scaling (DVFS) support at the individual core as well as the uncore level. This allows fine-grained control of performance and energy. For an n core processor with m core and uncore frequency choices, the total DVFS configuration space is now m (n+1) (with the uncore accounting for the + 1). In addition to that, in CMPs, the performance-energy trade-off due to core/uncore frequency scaling concerning a single application cannot be determined independently as cores share critical resources like the last level cache (LLC) and the memory. Thus, unlike the uni-processor environment, the energy consumption of an application running on a CMP depends not only on its characteristics but also on those of its co-runners (applications running on other cores). The key objective of our work is to select a suitable core and uncore frequency that minimizes power consumption while limiting application performance degradation within certain pre-defined limits (can be termed as QoS requirements). The key contribution of our work is a learning-based model that is able to capture the interference due to shared cache, bus bandwidth, and memory bandwidth between applications running on multiple cores and predict near-optimal frequencies for core and uncore.

STFL-DDR: Improving the Energy-Efficiency of Memory Interface

Power dissipation is a significant problem limiting the performance of today's computer systems. One of the main contributors to power consumption in microprocessors is data movement in cache and memory interface. Several solutions such as low power interconnects, energy-aware data encoding, and low power signaling have been proposed to mitigate this problem. Almost all of these techniques result in a significant system performance degradation. This article examines the application of a novel technique, called STFL-DDR, for hybrid signaling on low-power DRAM interface. To keep the power consumption low, STFL-DDR employs a high-performance clock rate for transferring data on low power wires. To avoid any signal deterioration, STFL-DDR employs data encoding/decoding to prevent each wire from switching in any two consecutive cycles. STFL-DDR creates new opportunities for optimizing the energy-efficiency of DRAM systems. We compare the efficiency of STFL-DDR with the state-of-the-art methods by simulating a mix of 12 parallel benchmark applications on a muticore system. Our simulation results indicate that STFL can reduce the energy consumption of a contemporary DRAM interface by 17 percent as compared to an LPDDR baseline while achieving the throughput of a high-performance DRAM. Applying STFL to both last level cache and DRAM interface results in improving the system energy, energy-delay product, and performance by 8, 15, and 9 percent respectively. Compared with a high-performance memory interface, STFL improves the system energy and energy-delay product by 25 and 75 percent, while reaching 98 percent of the average performance of the high-performance system.

DTaPO: Dynamic Thermal-Aware Performance Optimization for Dark Silicon Many-Core Systems

Future many-core systems need to handle high power density and chip temperature effectively. Some cores in many-core systems need to be turned off or ‘dark’ to manage chip power and thermal density. This phenomenon is also known as the dark silicon problem. This problem prevents many-core systems from utilizing and gaining improved performance from a large number of processing cores. This paper presents a dynamic thermal-aware performance optimization of dark silicon many-core systems (DTaPO) technique for optimizing dark silicon a many-core system performance under temperature constraint. The proposed technique utilizes both task migration and dynamic voltage frequency scaling (DVFS) for optimizing the performance of a many-core system while keeping system temperature in a safe operating limit. Task migration puts hot cores in low-power states and moves tasks to cooler dark cores to aggressively reduce chip temperature while maintaining high overall system performance. To reduce task migration overhead due to cold start, the source core (i.e., active core) keeps its L2 cache content during the initial migration phase. The destination core (i.e., dark core) can access it to reduce the impact of cold start misses. Moreover, the proposed technique limits tasks migration among cores that share the last level cache (LLC). In the case of major thermal violation and no cooler cores being available, DVFS is used to reduce the hot cores temperature gradually by reducing their frequency. Experimental results for different threshold temperatures show that DTaPO can keep the average system temperature below the thermal limit. Affirmatively, the execution time penalty is reduced by up to 18% compared with using only DVFS for all thermal thresholds. Moreover, the average peak temperature is reduced by up to 10.8°C. In addition, the experimental results show that DTaPO improves the system’s performance by up to 80% compared to optimal sprinting patterns (OSP) and reduces the temperature by up to 13.6°C.

An Adaptive Delay-Tolerant Routing Algorithm for Data Transmission in Opportunistic Social Networks

In opportunistic networks, the requirement of QoS (quality of service) poses several major challenges to wireless mobile devices with limited cache and energy. This implies that energy and cache space are two significant cornerstones for the structure of a routing algorithm. However, most routing algorithms tackle the issue of limited network resources from the perspective of a deterministic approach, which lacks an adaptive data transmission mechanism. Meanwhile, these methods show a relatively low scalability because they are probably built up based on some special scenarios rather than general ones. To alleviate the problems, this paper proposes an adaptive delay-tolerant routing algorithm (DTCM) utilizing curve-trapezoid Mamdani fuzzy inference system (CMFI) for opportunistic social networks. DTCM evaluates both the remaining energy level and the remaining cache level of relay nodes (two-factor) in opportunistic networks and makes reasonable decisions on data transmission through CMFI. Different from the traditional fuzzy inference system, CMFI determines three levels of membership functions through the trichotomy law and evaluates the fuzzy mapping from two-factor fuzzy input to data transmission by curve-trapezoid membership functions. Our experimental results show that within the error interval of 0.05~0.1, DTCM improves delivery ratio by about 20% and decreases end-to-end delay by approximate 25% as compared with Epidemic, and the network overhead from DTCM is in the middle horizon.

Phase-Aware Cache Partitioning to Target Both Turnaround Time and System Performance

The Last Level Cache (LLC) plays a key role in the system performance of current multi-cores by reducing the number of long latency main memory accesses. The inter-application interference at this shared resource, however, can lead the system to undesired situations regarding performance and fairness. Recent approaches have successfully addressed fairness and turnaround time (TT) in commercial processors. Nevertheless, these approaches must face sustaining system performance, which is challenging. This work makes two main contributions. LLC behaviors regarding cache performance, data reuse and cache occupancy, that adversely impact on the final performance are identified. Second, based on these behaviors, we propose the Critical-Phase Aware Partitioning Approach (CPA), which reduces TT while sustaining (and even improving) IPC by making an effective use of the LLC space. Experimental results show that CPA outperforms CA, Dunn and KPart state-of-the-art approaches, and improves TT (over 40 percent in some workloads) over Linux default behavior while sustaining or even improving IPC by more than 3 percent in several mixes.

Improving power-performance via hybrid cache for chip many cores based on neural network prediction technique

Recently, the increasing need to run applications for significant data analytics, and the augmented demand of useful tools for big data computing systems has resulted in a cumulative necessity for efficient platforms with high performance and realizable power consumption, for example, chip multiprocessors (CMPs). Correspondingly, due to the demand for features like shrinkable sizes, and the concurrent need to pack increasing numbers of transistors into a single chip, has led to serious design challenges, consuming a significant of power within high area densities. We present a reconfigurable hybrid cache system for last level cache (LLC) by the integration of emerging designs, such as STT-RAM with SRAM memories. This approach consists of two phases: off- time and on-time. In off time, training NN is implemented while in the on-time phase, a reconfiguration cache uses a neural network (NN) learning approach to predict demanded latency of the running application. Experimental results of a three-dimensional chip with 64 cores show that the suggested design under PARSEC benchmarks provides a speedup in terms of the performance at 25% and improves energy consumption by 78.4% in comparison to non-reconfigurable pure SRAM cache architectures.

Efficient Cache Resizing policy for DRAM-based LLCs in ChipMultiprocessors

In today’s ChipMultiprocessors (CMPs), multiple cores share the common Last Level Cache (LLC), divided into multiple banks. As the data requirement is increasing the demand for larger LLC sizes is also increasing. The traditional SRAM technology is not area efficient to design such larger LLCs as demanded by the modern CMPs. From the last few years, DRAM technologies have been used to propose LLC. DRAM technology has almost 8 times density over the SRAM and hence larger cache size can be designed. Though DRAM is already considered as an alternative to design low cost, area-efficient larger size LLC, it must be used efficiently to get the benefits. Due to its overheads like access latency and refresh operations efficient techniques must be used to get better performance from DRAM LLC. In the existing works, it has been observed that though the larger LLC is required for the current as well as future data-intensive applications, the entire LLC may not be required while executing other applications. In such situations, some banks can be almost idle during a particular period of execution. These idle banks can be powered-off and restart later whenever required. The mechanism is called Cache Resizing as it resizes the cache (LLC) according to the current requirements. Cache resizing techniques are already proposed for SRAM based LLCs but due to the larger size of DRAM LLC, the same mechanisms cannot be used for DRAM LLCs. In this paper, we have proposed an efficient cache resizing policy for large sized LLC, especially for DRAM-based LLCs. We call our proposed cache resizing technique as Efficient Cache Resizing (ECR) which is implemented on top of a 3D Tiled CMP. Experimental analysis shows that ECR can reduce up to 44% more energy consumption as compared to the existing technique.

Compacted CPU/GPU Data Compression via Modified Virtual Address Translation

We propose a method to reduce the footprint of compressed data by using modified virtual address translation to permit random access to the data. This extends our prior work on using page translation to perform automatic decompression and deswizzling upon accesses to fixed rate lossy or lossless compressed data. Our compaction method allows a virtual address space the size of the uncompressed data to be used to efficiently access variable-size blocks of compressed data. Compression and decompression take place between the first and second level caches, which allows fast access to uncompressed data in the first level cache and provides data compaction at all other levels of the memory hierarchy. This improves performance and reduces power relative to compressed but uncompacted data. An important property of our method is that compression, decompression, and reallocation are automatically managed by the new hardware without operating system intervention and without storing compression data in the page tables. As a result, although some changes are required in the page manager, it does not need to know the specific compression algorithm and can use a single memory allocation unit size. We tested our method with two sample CPU algorithms. When performing depth buffer occlusion tests, our method reduces the memory footprint by 3.1x. When rendering into textures, our method reduces the footprint by 1.69x before rendering and 1.63x after. In both cases, the power and cycle time are better than for uncompacted compressed data, and significantly better than for accessing uncompressed data.

J SWof 5.5 MA/cm2 and RA of 5.2-Ω · μm2 STT-MRAM Technology for LLC Application

Due to the complexity of device processing, the trade-off between yield and area has resulted in diminishing rate of scaling for the high-density static random access memory (SRAM) cell at advanced CMOS nodes. An introduction of extreme ultraviolet (EUV) and multipatterning has added additional cost to technology in order to realize 3-D device structure and ultrascaled metal routing. In this era, spin-transfer torque (STT)-MRAM technology can provide an alternative to high-density SRAM and for the last level cache (LLC) applications. In this article, we discuss the memory design and technology tradeoff to enable the STT-MRAM as a viable option. We have realized the technology over 300-mm wafer, measuring 1 million samples to build a SPICE model for circuit simulation. Occupying up to 83.3% of an area that of SRAM macro has been designed and simulated for the scaled 5-nm CMOS node. The simulated MRAM macro shows the best read and write access time of 3.1 and 6.2 ns, respectively. Magnetic tunneling junction (MTJ) pillar of 38-nm diameter is realized at 90-nm pitch, measuring resistance area (RA) of 5.2- $\Omega \cdot \mu \text{m}^{2}$ , ${J}_{sw}$ of 5.5 MA/cm2 with improved $\Delta $ avg of 70, and breakdown voltage of 0.99 V. The energy comparison shows increasing gains versus SRAM for the increasing cache sizes crossing over at of 0.3 and 4 MB for the single-cycle read and write operations, respectively.

PFCA: A Programmable FIB Caching Architecture

Ternary Content-Addressable Memory (TCAM) chips are used to store Forwarding Information Bases (FIB) in modern routers. TCAM provides next-hop lookup for IP packets at the line-rate. However, TCAM is expensive and energy-consuming; in addition, the constant FIB growth may lead to TCAM overflow problem. Yet only a small portion of FIB entries carries the most of network traffic. Thus, FIB caching, namely, installing the most popular entries in a fast memory, e.g., TCAM, may significantly minimize TCAM usage. To date, FIB caching architecture has not been widely deployed in backbone routers due to high cache-miss latency and the lack of an efficient cache replacement strategy. In this work, we leverage the concept of the programmable data plane to design a Programmable FIB Caching Architecture (PFCA) with two levels of cache. We present a pipeline-based algorithm to detect the least popular prefixes in a cache for a victim selection. We tested the prototype of PFCA using real traffic traces and an FIB with more than 599K entries, and showed that PFCA can be implemented using P4 programmable data plane language. Our results show that PFCA achieves 99.8% hit ratio for Level-1 cache with 20K entries and nearly 99.9% hit ratio for Level-2 cache with 40K entries. We also demonstrate that PFCA significantly reduces the number of BGP updates in the cache and thus makes the cache more stable.

Rainbow: A composable coherence protocol for multi‐chip servers

SummaryThe use of multi‐chip modules (MCM) and/or multi‐socket boards is the most suitable approach to increase the computation density of servers while keep chip yield attained. This article introduces a new coherence protocol suitable, in terms of complexity and scalability, for this class of systems. The proposal uses two complementary ideas: (1) A mechanism that dissociates complexity from performance by means of colored‐token counting, (2) A construct that optimizes performance and cost by means of two functionally symmetrical structures working in the last level cache of each chip and each memory controller. The coordinated work of both structures minimizes the coherence‐related effects on the average memory latency perceived by the processor. Our proposal is able to improve on the performance of a HyperTransport‐like coherence protocol by from 25% to 60%.

Mitigating Process Variability for Non-Volatile Cache Resilience and Yield

While inclusion of emerging technology-based Non-Volatile Memory (NVM) devices in on-chip memory subsystems offers excellent potential for energy savings and scalability, their sensing vulnerability creates Process Variation (PV) challenges. This paper presents a circuit-architecture cross-layer solution to realize a radically-different approach to leveraging as-built variations via specific Sense Amplifier (SA) design and use. This novel approach, referred to as a Self-Organized Sub-bank (SOS) design, assigns the preferred SA to each Sub-Bank (SB) based on a PV assessment, resulting in energy consumption reduction and increased read access reliability. To improve the PV immunity of SAs, two reliable and power efficient SAs, called the Merged SA (MSA) and the Adaptive SA (ASA) are introduced herein for use in the SOS scheme. Furthermore, we propose a dynamic PV and energy-aware cache block migration policy that utilizes mixed SRAM and STT-MRAM banks in Last Level Cache (LLC) to maximize the SOS bandwidth. Our experimental results indicate that SOS can alleviate the sensing vulnerability by 89 percent on average, which significantly reduces the risk of application contamination by fault propagation. Furthermore, in the light of the proposed block migration policy, write performance is improved by 12.4 percent on average compared to the STT-MRAM-only design.

Page Reusability-Based Cache Partitioning for Multi-Core Systems

Most modern multi-core processors provide a shared last level cache (LLC) where data from all cores are placed to improve performance. However, this opens a new challenge for cache management, owing to cache pollution. With cache pollution, data with weak temporal locality can evict other data with strong temporal locality when both are mapped into the same cache set. In this article, we propose page reusability-based cache partitioning (PRCP) for multi-core systems to maximize cache utilization by minimizing cache pollution. To achieve this, PRCP divides pages into two groups: (1) highly-reused pages and (2) lowly-reused pages. The reusability of each page is collected online via periodic page table scans. PRCP then dynamically partitions the shared cache into two corresponding areas using page coloring technique. We have implemented PRCP in Linux kernel and evaluated it using SPEC CPU2006 benchmarks. The results show that our scheme can achieve comparable performance to the optimal offline MRC-guided process-based cache partitioning scheme without a priori knowledge of workloads.

Cache efficient Value Iteration using clustering and annealing

Value Iteration (VI) is a powerful, though time consuming, approach to solve Markov Decision Processes (MDPs). Existing algorithms for VI incur a large number of cache misses. Motivated by the observation that, on modern computers, the cost of a cache miss is two to three orders of magnitude more than that of an arithmetic operation, we explore the possibility of improving the performance of VI by reducing the number of cache misses, possibly at the expense of increasing the number of arithmetic operations. Cache efficiency is obtained by performing VI on partitions of the MDP state space that fit in the lowest level cache of the computational platform on which the code is to run. Further performance improvement, motivated by the use of MDP partitions, is obtained using a clustering scheme to construct the partitions and an annealing schedule to converge to the target accuracy. We demonstrate experimentally that incorporating partitioning, clustering, and annealing into state-of-the-art VI software result in speedups of up to a factor of 8.1 on our computational platforms.

On the parallelization and performance analysis of Barnes–Hut algorithm using Java parallel platforms

Multi-core processors provide time-efficient and cost-effective solutions to execute the algorithms for complex physical systems. However, to efficiently exploit the processing capabilities of the underlying architectures, the applications executing on multi-core processors must be parallelized. Conventionally, the applications for high-performance computing (HPC) are written in native (programming) languages. However, several researchers have argued that Java can be an excellent alternative for writing HPC applications. To gauge the comparative performance of Java parallel platforms, a benchmark study is required considering some real HPC applications. In this study, as a pioneer application, Java-based parallel platforms, namely Java thread API, JavaSymphony, and MPJ Express, are used to parallelize and benchmark the Barnes–Hut algorithm to simulate the N-body physical system of celestial objects. The parallel implementations of the Barnes–Hut algorithm are tested for performance on shared memory multi-core architectures. The hardware-level performance analysis involving the parameters like the number of instructions executed by the CPU, level-1 and level-3 cache misses, and the number of main memory accesses is conducted to investigate the insights of the attained performances. The celestial objects up to one million are simulated, and the results revealed that the JavaSymphony-based implementation of the Barnes–Hut algorithm produces better results as compared to the other employed parallel frameworks.