Non-uniform Cache Access Research Articles

A Novel High Performance and Energy Efficient NUCA Architecture for STT-MRAM LLCs With Thermal Consideration

As the speed gap of the modern processor and the off-chip main memory enlarges, on-chip cache capacity increases to sustain the performance scaling. As a result, the cache power occupies a large portion of the total power budget. Spin transfer torque magnetic memory (STT-MRAM) is proposed as a promising solution for the low power cache design due to its high integration density and ultralow leakage power. Nevertheless, the high write power and latency of STT-MRAM become new barriers for the commercialization of this emerging technology. In this paper, we investigate the thermal effect on the access performance of STT-MRAM, and observe that the temperature can affect the write delay and energy significantly. Then, we explore the nonuniform cache access (NUCA) design of the chip-multiprocessors with STT-MRAM-based last level cache (LLC). A thermal aware data migration policy, called “Thermosiphon,” which takes advantage of the thermal property of STT-MRAM, is proposed to reduce the LLC write energy. This policy splits the LLC into different regions dynamically based on the thermal distribution monitored by thermal sensors available on-chip, and adaptively migrates write intensive data among different thermal regions considering the thermal gradient. Compared to the conventional NUCA design, our proposed design can save 41.2% write energy at most and 13.01% on average with negligible hardware overhead.

A Low Overhead In-Network Data Compressor for the Memory Hierarchy of Chip Multiprocessors

Data compression has been intensively studied to increase the utility of cache, network-on-chip (NoC), and main memory in energy-efficient processors. However, prior solutions to data compression often add remarkable compression and decompression delay to the critical path of memory access, which is thought as the major factor limiting its application to commodity processors. Unlike prior work that deals with memory compression or network compression separately, this paper proposes a unified on-chip distributed data compressor (DISCO), to enable near-zero-latency cache and memory block compression for chip multiprocessors adopting nonuniform cache access. DISCO integrates a multimode cache compressor into the NoC routers and overlaps the de/compression latency with the queuing delay in the network. In addition, cache block evicted to or fetched from the main memory can also be compressed or decompressed during the network queuing time in this unified DISCO compressor. With the support of congestion-awareness, it is shown in the evaluation that DISCO, which unifies the compression solution of the memory hierarchy, dramatically decreases the compression overhead of isolated techniques, and significantly boosts the efficiency of data moving and store.

VANUCA: Enabling Near-Threshold Voltage Operation in Large-Capacity Cache

In this paper, we investigate the feasibility of voltage adjustment in a large capacity cache, and propose the architecture of voltage-adaptable nonuniform cache access (VANUCA) that exploits near-threshold computing and multivoltage domain to approach the limit of $V_{\rm dd}$ in a low-power cache. However, the adoption of near-threshold voltage (NTV) leads to a rocketing error probability in SRAM arrays, which has to be addressed by effective fault-tolerant techniques. Instead of using error correction code or data duplication, the VANUCA exploits the natural data redundancy across the whole memory hierarchy to enable fast fault recovery in the NTV cache. Based on the discovered data resilience and the multi- $V_{\rm dd}$ architecture, the VANUCA is able to match vulnerable/invulnerable data clusters to available high-/low-voltage domains by utilizing the data migration mechanism in dynamic NUCA. The proposed VANUCA includes two important architectural techniques: 1) static assignment that assumes a fixed voltage domain partitioning and 2) DataMotion that dynamically fits the working set into heterogeneous cache banks through $V_{\rm dd}$ switching. Experimental results show that the VANUCA achieves considerable improvements in energy efficiency over the conventional single-voltage domain NUCA cache.

Efficient Energy and Power Consumption of 3-D Chip Multiprocessor with NUCA Architecture

Objective: Technology trend are one of the most important to use the bulky on-chip memories such as level-2 (L2) and level-3 (L3) cache. In this paper, Non-Uniform Cache access Architectures (NUCA) has been used in order to decrease the effects of the high access latencies. For the systems which operate at high frequency, the latencies of such caches are conquered by wire delays. Methods: In this architecture, several banks are partitioned and which is connected to the controller through switches and links. This larger cache memory increases overall wire delays across the chip due to very bulky latencies. The majority of the access time will not absorb them self and that occupy in the routing to and from the several banks. The predictable lower-level caches have the best ever access time among the entire the sub arrays, although such uniform cache access don’t make the use of differences in latencies of the entire sub arrays. Findings: Hence there is a necessitate for non-uniform cache architectures where the cache memories is broken down into several banks that can be analyzed at dissimilar latencies and resulting in diminish of energy and power consumption. Applications/Improvements: An experimental result shows the comparison of NUCA and UCA using SRAM/MRAM in terms of power and energy.

Design space exploration of non-uniform cache access for soft-error vulnerability mitigation

In this paper, the design space exploration problem is concerned with finding the best composition of different Non-Uniform Cache Access (NUCA) specifications in many-core processors. The single-objective and multi-objective exploration problems are intended to meet the desired level of reliability without violating the performance and energy constraints. The main objective is to find the best choice for each cache specification which can minimize the vulnerability of L1 and L2 caches in NUCA architectures. The design space consists of 72 implementations, made up of combinations of different structures in the current NUCA specifications (cache organization, write policy, coherence protocol, inclusiveness, replacement policy, and network topology). Moreover, the effects of design implementations on reliability (as the main objective), performance, cache energy consumption, and interconnection traffic (as the constraints) have been investigated.

Hybrid Shared-aware Cache Coherence Transition Strategy

Chip-multiprocessors have played a significant role in real parallel computer architecture design. For integrating tens of cores into a chip, designs tend towards with physically distributed last level caches. This naturally results in a Non-Uniform Cache Access design, where on-chip access latencies depend on the physical distances between requesting cores and home cores where the data is cached. Therefore, data movement and management impact access latency and power consumption. Remote misses limit the performance of multi-threaded applications, so using data locality is fundamental importance in Chip-multiprocessors. In this work we observed that a shared data writing behavior dramatically wastes precious on-chip cache resource and seriously affects the whole system performance. Therefore, we emphasis on improving the performance of applications that exhibit high data sharing, and propose a new prediction mechanism to predict accurately the impact of shared data and a scalable, efficient hybrid shared-aware cache coherence transition strategy which collaborate with directory-based MESI cache coherence protocol. In order to evaluate our proposal transition strategy, we experiment with the NAS Parallel Benchmarks and a modern Intel Harpertown multi-core machine. Results show the whole performance gains of up to 21% opposed to the traditional write-invalidate cache coherence transition strategy.

NUCA-L1

Research has shown that operating in the near-threshold region is expected to provide up to 10× energy efficiency for future processors. However, reliable operation below a minimum voltage (Vccmin) cannot be guaranteed due to process variations. Because SRAM margins can easily be violated at near-threshold voltages, their bit-cell failure rates are expected to rise steeply. Multicore processors rely on fast private L1 caches to exploit data locality and achieve high performance. In the presence of high bit-cell fault rates, traditionally an L1 cache either sacrifices capacity or incurs additional latency to correct the faults. We observe that L1 cache sensitivity to hit latency offers a design trade-off between capacity and latency. When fault rate is high at extreme Vccmin, it is beneficial to recover L1 cache capacity, even if it comes at the cost of additional latency. However, at low fault rates, the additional constant latency to recover cache capacity degrades performance. With this trade-off in mind, we propose a Non-Uniform Cache Access L1 architecture (NUCA-L1) that avoids additional latency on accesses to fault-free cache lines. To mitigate the capacity bottleneck, it deploys a correction mechanism to recover capacity at the cost of additional latency. Using extensive simulations of a 64-core multicore, we demonstrate that at various bit-cell fault rates, our proposed private NUCA-L1 cache architecture performs better than state-of-the-art schemes, along with a significant reduction in energy consumption.

Thread Migration Prediction for Distributed Shared Caches

Chip-multiprocessors (CMPs) have become the mainstream parallel architecture in recent years; for scalability reasons, designs with high core counts tend towards tiled CMPs with physically distributed shared caches. This naturally leads to a Non-Uniform Cache Access (NUCA) design, where on-chip access latencies depend on the physical distances between requesting cores and home cores where the data is cached. Improving data locality is thus key to performance, and several studies have addressed this problem using data replication and data migration. In this paper, we consider another mechanism, hardware-level thread migration. This approach, we argue, can better exploit shared data locality for NUCA designs by effectively replacing multiple round-trip remote cache accesses with a smaller number of migrations. High migration costs, however, make it crucial to use thread migrations judiciously; we therefore propose a novel, on-line prediction scheme which decides whether to perform a remote access (as in traditional NUCA designs) or to perform a thread migration at the instruction level. For a set of parallel benchmarks, our thread migration predictor improves the performance by 24% on average over the shared-NUCA design that only uses remote accesses.

CMP CACHE ARCHITECTURES - A SURVEY

As the number of cores on Chip Multi-Processor (CMP) increases, the need for effective utilization (management) of the cache increases. Cache Management plays an important role in improving the performance and miss latency by reducing the number of misses. In most of the cases, CMP with shared Last Level Cache (LLC) is a winner over the private LLC. Non-Uniform Cache Access (NUCA) represent two emerging trends in computer architecture. In NUCA the LLC is divided into multiple banks which lead to different banks being accessed with different latencies. Hence the heavily used blocks can be mapped or migrated towards the closer bank of the requesting core. Though NUCA is the best architecture for single core systems, implementing NUCA in CMP has many challenges. Researchers proposed many innovative ideas to implement NUCA in CMP but still there exists lot more complexities. Thus CMP cache architecture is a widely open research area. In this paper we did a survey on different CMP cache architectures based on NUCA. We have only given a basic overview and there are lot more advanced innovations which are not been covered. The performance evaluation of CMP architecture is a challenging task and must have to do for proving the correctness of any proposed architecture. Therefore, we also discussed about how the performance of CMP cache architectures can be evaluated

A study of 3D Network-on-Chip design for data parallel H.264 coding

In this paper, we implement, analyze and compare different Network-on-Chip (NoC) architectures aiming at higher efficiencies for MPEG-4/H.264 coding. Two-dimensional (2D) and three-dimensional (3D) NoCs based on Non-Uniform Cache Access (NUCA) are analyzed. We present results using a full system simulator with realistic workloads. Experiments show the average network latencies in two 3D NoCs are reduced by 28% and 34% respectively, comparing with 2D design. It is also shown that heat dissipation is a trade-off in improving performance of 3D chips. Our analysis and experiment results provide a guideline to design efficient 3D NoCs for data parallel H.264 coding applications.

Process-Variation-Aware Adaptive Cache Architecture and Management

Fabricating circuits that employ ever-smaller transistors leads to dramatic variations in critical process parameters. This in turn results in large variations in execution/access latencies of different hardware components. This situation is even more severe for memory components due to minimum-sized transistors used in their design. Current design methodologies that are tuned for the worst case scenarios are becoming increasingly pessimistic from the performance angle, and thus, may not be a viable option at all for future designs. This paper makes two contributions targeting on-chip data caches. First, it presents an adaptive cache management policy based on nonuniform cache access. Second, it proposes a latency compensation approach that employs several circuit-level techniques to change the access latency of select cache lines based on the criticalities of the load instructions that access them. Our experiments reveal that both these techniques can recover significant amount of the lost performance due to worst case designs.

BENoC: A Bus-Enhanced Network on-Chip for a Power Efficient CMP

Network-on-chips (NoCs) outperform buses in terms of scalability, parallelism and system modularity and therefore are considered as the main interconnect infrastructure in future chip multi-processor (CMP). However, while NoCs are very efficient for delivering high throughput point-to-point data from sources to destinations, their multi-hop operation is too slow for latency sensitive signals. In addition, current NoCS are inefficient for broadcast operations and centralized control of CMP resources. Consequently, state-of-the-art NoCs may not facilitate the needs of future CMP systems. In this paper, the benefit of adding a low latency, customized shared bus as an internal part of the NoC architecture is explored. BENoC (bus-enhanced network on-chip) possesses two main advantages: First, the bus is inherently capable of performing broadcast transmission in an efficient manner. Second, the bus has lower and more predictable propagation latency. In order to demonstrate the potential benefit of the proposed architecture, an analytical comparison of the power saving in BENoC versus a standard NoC providing similar services is presented. Then, simulation is used to evaluate BENoC in a dynamic non-uniform cache access (DNUCA) multiprocessor system.

Nonuniform cache architectures for wire-delay dominated on-chip caches

Nonuniform cache access designs solve the on-chip wire delay problem for future large integrated caches. By embedding a network in the cache, NUCA designs let data migrate within the cache, clustering the working set nearest the processor. The authors propose several designs that treat the cache as a network of banks and facilitate nonuniform accesses to different physical regions. NUCA architectures offer low-latency access, increased scalability, and greater performance stability than conventional uniform access cache architectures.