Cache Memory Hierarchy Research Articles

Memory Space Recycling

Many program codes from different application domains process very large amounts of data, making their cache memory behavior critical for high performance. Most of the existing work targeting cache memory hierarchies focus on improving data access patterns, e.g., maximizing sequential accesses to program data structures via code and/or data layout restructuring strategies. Prior work has addressed this data locality optimization problem in the context of both single-core and multi-core systems. Another dimension of optimization, which can be as equally important/beneficial as improving data access pattern is to reduce the data volume (total number of addresses) accessed by the program code. Compared to data access pattern restructuring, this volume minimization problem has relatively taken much less attention. In this work, we focus on this volume minimization problem and address it in both single-core and multi-core execution scenarios. Specifically, we explore the idea of rewriting an application program code to reduce its "memory space footprint". The main idea behind this approach is to reuse/recycle, for a given data element, a memory location that has originally been assigned to another data element, provided that the lifetimes of these two data elements do not overlap with each other. A unique aspect is that it is "distance aware", i.e., in identifying the memory/cache locations to recycle it takes into account the physical distance between the location of the core and the memory/cache location to be recycled. We present a detailed experimental evaluation of our proposed memory space recycling strategy, using five different metrics: memory space consumption, network footprint, data access distance, cache miss rate, and execution time. The experimental results show that our proposed approach brings, respectively, 33.2%, 48.6%, 46.5%, 31.8%, and 27.9% average improvements in these metrics, in the case of single-threaded applications. With the multi-threaded versions of the same applications, the achieved improvements are 39.5%, 55.5%, 53.4%, 26.2%, and 22.2%, in the same order.

Методика формирования тестовых последовательностей на основе графовой модели иерархии кеш-памяти

The verification of the cache memory hierarchy in modern SoC due to the large state space requires a huge number of complex tests. This becomes the main problem for functional verification. To cover the entire state space, a graph model of the cache memory hierarchy as well as the methods of generating the formation of the test sequences based on this model have been proposed. The graph model vertices are a set of states (tags, values, etc.) of each hierarchy level, and the edges are a set of transitions between states (instructions for reading, records). The graph model, describing all states of the cache-memory hierarchy states, has been developed. Each edge in the graph is a separate check sequence. In case of the non-deterministic situations, such as the choice of a channel (port) for multichannel cache memory, it will not be possible to resolve them at the level of the graph model, since the choice of the channel depends on many factors not considered within the model framework. It has been proposed to create a separate instance of a subgraph for each channel. The described approach has revealed, in verification of the multiport cache-memory hierarchy of the developed core with the new vector architecture VLIW DSP, a few architectural and functional errors. This approach can be used to test other processor cores and their blocks

ROCKY: A Robust Hybrid On-chip Memory Kit For The Processors With STT-MRAM Cache Technology

STT-MRAM is regarded as an extremely promising NVM technology for replacing SRAM-based on-chip memories. While STT-MRAM memories benefit from ultra-low leakage power and high density, they suffer from some reliability challenges, namely, read disturbance, write failure, and retention failure. The write failure; storing a wrong value in an STT-MRAM cell during a write operation, is the most crucial reliability challenge. In this article, we propose ROCKY; a robust architecture equipped with efficient replacement policies for STT-MRAM-based cache memory hierarchy to improve the robustness of STT-MRAM part against the write failures. ROCKY reduces susceptible transitions in STT-MRAM cache memories leading to more reliable STT-MRAM write operations. The simulation results through comparison with traditional cache memory hierarchy demonstrate ROCKY decreases the WER of STT-MRAM cache memories by up to 35.4 percent while imposing less than 1 percent performance overhead to the system.

NUCA-2A: A New Adaptive and Behavior Aware Block Placement Process

Background: The last three decades were marked by a spectacular evolution of CPUs. Both cores number on chip and shared Low Level Cache (LLC) size are increasing what makes LLC the bottleneck's system. One major weakness of future cache memory hierarchies will be to carry out memory blocks availability for vertical requests, with no consideration to horizontal proximity to cores. Simulations show that some LLC accesses cost more latency cycles than off-chip accesses. Objective: This paper presents a new adaptive and blocks behavior aware process, called NUCA-2A. It manages blocks in LLC in a purpose of reducing it's latency, and it's inner bandwidth, by studying each block's behavior, and by placing it in the most suitable location among LLC banks. Methods: LLC accesses are classified basing on each one's specific behavior. Authors establish also a two levels horizontal hierarchy in LLC. This work consists to place blocks in the zones that matches the best their behaviors. Results: In contrast to the classic S-NUCA scheme, NUCA-2A makes a reduction of up to 60,39% of global LLC latency as well as 40,74% of average inner traffic. It makes also an average speedup of 17,89 % in term of number of instructions executed by cycle. Conclusion: Behaviors study gives encouraging results. Several methods are in use in different fields to forecast a behavior basing on previous observations. We are working on a prefetching model that permits blocks migration to and from privileged banks.

Read-Tuned STT-RAM and eDRAM Cache Hierarchies for Throughput and Energy Optimization

As capacity and complexity of on-chip cache memory hierarchy increases, the service cost to the critical loads from last level cache (LLC), which are frequently repeated, has become a major concern. The processor may stall for a considerable interval while waiting to access the data stored in the cache blocks in LLC, if there are no independent instructions to execute. To provide accelerated service to the critical loads requests from LLC, this paper concentrates on leveraging the additional capacity offered by replacing SRAM-based L2 with spin-transfer torque random access memory (STT-MRAM) to accommodate frequently accessed cache blocks in exclusive read mode in favor of reducing the overall read service time. Our proposed technique improves the temporal locality while preventing cache thrashing via sufficient accommodation of the frequently read reused fraction of working set that may exhibit distant re-reference interval in L2. Our experimental results show that the proposed technique can reduce the L2 read miss ratio by 51.7% on average compared to conventional STT-MRAM L2 design across PARSEC and SPEC2006 workloads while significantly decreasing the L2 dynamic energy consumption.

Multi-objective optimization applied to unified second level cache memory hierarchy tuning aiming at energy and performance optimization

Cache memory optimization has an important impact on the energy consumption of the embedded system. However, optimization is a hard task due to the large exploration space and conflicting objectives. In this work five multiobjective optimization techniques are applied to cache memory optimization. The PESA-II, NSGAII, SPEA2, PAES and NPGA approaches were applied to 18 different applications from MiBench and PowerStone benchmark suites. Results compared the quality of results in terms of the metrics of general distance, diversity, hypervolume and precision. All techniques had good performance to cache optimization, but PESA-II showed a better performance for all metrics analyzed, having better results in 83% and 88% of cases, compared with the metrics of generational distance and hypervolume, respectively. Additionally, PESA-II needs to explore only 1.47% of exploration space, finding solutions near to Pareto Optimal.

Modeling Cache Memory Utilization on Multicore Using Common Pool Resource Game on Cellular Automata

Recent computing architectures are implemented by shared memory technologies to alleviate the high latency experienced by off-chip memory transfers, but the high architectural complexity of modern multicore processors has presented many questions. To tackle the design of efficient algorithms scheduling workloads over available cores, this article presents a parallel bioinspired model that simulates the utilization of shared memory on multicore systems. The proposed model is based on cellular automata (CA) and coupled with game theory principles. CA are selected due to their inherent parallelism and especially their ability to incorporate inhomogeneities. Furthermore, the novelty of the model is realized on the fact that multilevel CA are used to simulate the different levels of cache memory usually found in multicore processors. These characteristics make the model able to cope with the increasing diversity of cache memory hierarchies on modern and future processors. Nonetheless, by acquiring data from hardware performance counters and processing them with the proposed model online, the performance of the system can be calculated and a better scheduling strategy can be adopted in real time. The CA-based model was verified on the behavior of a real multicore system running a multithreaded application, and it successfully simulated the acceleration achieved by an increased number of cores available for the execution of the workload. More specifically, the example of common pool resource from game theory was used with two variations: a static and a variable initial endowment. The static variation of the model approximates slightly better the acceleration of a workload when the number of available processor cores increases, whereas the dynamic variation simulates better the moderate differences due to operation system’s scheduler alternations on the same amount of cores.

A Survey on Cache Management Mechanisms for Real-Time Embedded Systems

Multicore processors are being extensively used by real-time systems, mainly because of their demand for increased computing power. However, multicore processors have shared resources that affect the predictability of real-time systems, which is the key to correctly estimate the worst-case execution time of tasks. One of the main factors for unpredictability in a multicore processor is the cache memory hierarchy. Recently, many research works have proposed different techniques to deal with caches in multicore processors in the context of real-time systems. Nevertheless, a review and categorization of these techniques is still an open topic and would be very useful for the real-time community. In this article, we present a survey of cache management techniques for real-time embedded systems, from the first studies of the field in 1990 up to the latest research published in 2014. We categorize the main research works and provide a detailed comparison in terms of similarities and differences. We also identify key challenges and discuss future research directions.

Modification of Cache Inclusion Property for Multicore Systems

The cache coherency protocol guarantees the validity of the cache block by keeping it with the latest updated contents. In multi-level cache memory hierarchy, the inclusion property enforces the lower layer cache to maintain the superset of its higher layer cache contents. The inclusion property has its benefits for cache coherence, but it may waste valuable cache blocks and bandwidth by invalidating the duplicated contents in the higher level cache. In this paper, a new cache contention management scheme is presented with modified inclusion property to avoid unnecessary cache block invalidation.

Cache-Aware Asymptotically-Optimal Sampling-Based Motion Planning.

We present CARRT* (Cache-Aware Rapidly Exploring Random Tree*), an asymptotically optimal sampling-based motion planner that significantly reduces motion planning computation time by effectively utilizing the cache memory hierarchy of modern central processing units (CPUs). CARRT* can account for the CPU's cache size in a manner that keeps its working dataset in the cache. The motion planner progressively subdivides the robot's configuration space into smaller regions as the number of configuration samples rises. By focusing configuration exploration in a region for periods of time, nearest neighbor searching is accelerated since the working dataset is small enough to fit in the cache. CARRT* also rewires the motion planning graph in a manner that complements the cache-aware subdivision strategy to more quickly refine the motion planning graph toward optimality. We demonstrate the performance benefit of our cache-aware motion planning approach for scenarios involving a point robot as well as the Rethink Robotics Baxter robot.

Measurement of the latency parameters of the Multi-BSP model: a multicore benchmarking approach

Computer benchmarking is a common method for measuring the parameters of a computational model. It helps to measure the parameters of any computer. With the emergence of multicore computers, the evaluation of computers was brought under consideration. Since these types of computers can be viewed and considered as parallel computers, the evaluation methods for parallel computers may be appropriate for multicore computers. However, because multicore architectures seriously focus on cache hierarchy, there is a need for new and different benchmarks to evaluate them correctly. To this end, this paper presents a method for measuring the parameters of one of the most famous multicore computational models, namely Multi-Bulk Synchronous Parallel (Multi-BSP). This method measures the hardware latency parameters of multicore computers, namely communication latency (g i ) and synchronization latency (L i ) for all levels of the cache memory hierarchy in a bottom-up manner. By determining the parameters, the performance of algorithms on multicore architectures can be evaluated as a sequence.

Novel Nonvolatile L1/L2/L3 Cache Memory Hierarchy Using Nonvolatile-SRAM With Voltage-Induced Magnetization Switching and Ultra Low-Write-Energy MTJ

To reduce power consumption of CPU, nonvolatile cache memory has been expected by replacing conventional volatile cache memory based on SRAM. This paper describes nonvolatile cache memory hierarchy design using fast and low-power perpendicular (FL-p-) STT-MRAM. For L3, L2 and L1 cache, 1T-1MTJ with FL-p-STT-MRAM, 6T-2MTJ, and short write pulse based 6T-2MTJ having voltage-induced magnetization switching has been presented for the most suitable combination for the cache memory.

Two-level caches tuning technique for energy consumption in reconfigurable embedded MPSoC

In order to meet the ever-increasing computing requirement in the embedded market, multiprocessor chips were proposed as the best way out. In this work we investigate the energy consumption in these embedded MPSoC systems. One of the efficient solutions to reduce the energy consumption is to reconfigure the cache memories. This approach was applied for one cache level/one processor architecture, but has not yet been investigated for multiprocessor architecture with two level caches. The main contribution of this paper is to explore two level caches (L1/L2) multiprocessor architecture by estimating the energy consumption. Using a simulation platform, we first built a multiprocessor architecture, and then we propose a new algorithm that tunes the two-level cache memory hierarchy (L1 and L2). The tuning caches approach is based on three parameters: cache size, line size, and associativity. To find the best cache configuration, the application is divided into several execution intervals. And then, for each interval, we generate the best cache configuration. Finally, the approach is validated using a set of open source benchmarks; Spec 2006, Splash-2, MediaBench and we discuss the performance in terms of speedup and energy reduction.

Power and performance analysis of multimedia applications running on low-power devices by cache modeling

High processing speed is required to support computation intensive applications. Cache memory is used to improve processing speed by reducing the speed gap between the fast processing core and slow main memory. However, the problem of adopting cache into computing systems is twofold: cache is power hungry (that challenges energy constraints) and cache introduces execution time unpredictability (that challenges supporting real-time multimedia applications). Recently published articles suggest that using cache locking improves predictability. However, increased cache activities due to aggressive cache locking make the system consume more energy and become less efficient. In this paper, we investigate the impact of cache parameters and cache locking on power consumption and performance for real-time multimedia applications running on low-power devices. In this work, we consider Intel Pentium-like single-processor and Xeon-like multicore architectures, both with two-level cache memory hierarchy, using three popular multimedia applications: MPEG-4 (the global video coding standard), H.264/AVC (the network friendly video coding standard), and recently introduced H.265/HEVC (for improved video quality and data compression ratio). Experimental results show that cache locking mechanism added to an optimized cache memory structure is very promising to increase the performance/power ratio of low-power systems running multimedia applications. According to the simulation results, performance can be improved by decreasing cache miss rate down to 36 % and the total power consumption can be saved up to 33 %. It is also observed that H.265/HEVC has significant performance advantage over H.264/AVC (and MPEG-4) for smaller caches.

Decoupled Processors Architecture for Accelerating Data Intensive Applications using Scratch-Pad Memory Hierarchy

We present an architecture of decoupled processors with a memory hierarchy consisting only of scratch-pad memories, and a main memory. This architecture exploits the more efficient pre-fetching of Decoupled processors, that make use of the parallelism between address computation and application data processing, which mainly exists in streaming applications. This benefit combined with the ability of scratch-pad memories to store data with no conflict misses and low energy per access contributes significantly for increasing the system's performance. The application code is split in two parallel programs the first runs on the Access processor and computes the addresses of the data in the memory hierarchy. The second processes the application data and runs on the Execute processor, a processor with a limited address space--just the register file addresses. Each transfer of any block in the memory hierarchy up to the Execute processor's register file is controlled by the Access processor and the DMA units. This strongly differentiates this architecture from traditional uniprocessors and existing decoupled processors with cache memory hierarchies. The architecture is compared in performance with uniprocessor architectures with (a) scratch-pad and (b) cache memory hierarchies and (c) the existing decoupled architectures, showing its higher normalized performance. The reason for this gain is the efficiency of data transferring that the scratch-pad memory hierarchy provides combined with the ability of the Decoupled processors to eliminate memory latency using memory management techniques for transferring data instead of fixed prefetching methods. Experimental results show that the performance is increased up to almost 2 times compared to uniprocessor architectures with scratch-pad and up to 3.7 times compared to the ones with cache. The proposed architecture achieves the above performance without having penalties in energy delay product costs.

Impact of level-2 cache sharing on the performance and power requirements of homogeneous multicore embedded systems

In order to satisfy the needs for increasing computer processing power, there are significant changes in the design process of modern computing systems. Major chip-vendors are deploying multicore or manycore processors to their product lines. Multicore architectures offer a tremendous amount of processing speed. At the same time, they bring challenges for embedded systems which suffer from limited resources. Various cache memory hierarchies have been proposed to satisfy the requirements for different embedded systems. Normally, a level-1 cache (CL1) memory is dedicated to each core. However, the level-2 cache (CL2) can be shared (like Intel Xeon and IBM Cell) or distributed (like AMD Athlon). In this paper, we investigate the impact of the CL2 organization type (shared Vs distributed) on the performance and power consumption of homogeneous multicore embedded systems. We use VisualSim and Heptane tools to model and simulate the target architectures running FFT, MI, and DFT applications. Experimental results show that by replacing a single-core system with an 8-core system, reductions in mean delay per core of 64% for distributed CL2 and 53% for shared CL2 are possible with little additional power (15% for distributed CL2 and 18% for shared CL2) for FFT. Results also reveal that the distributed CL2 hierarchy outperforms the shared CL2 hierarchy for all three applications considered and for other applications with similar code characteristics.

An Enhancement of Futures Runtime in Presence of Cache Memory Hierarchy

A future is a simple abstraction mechanism for exposing potential concurrency in programs. In this paper, we propose an enhancement of our previously developed runtime for scheduling and executing futures based on the lazy task creation technique that aims to reflect the cache memory hierarchy present in modern multi-core and multiprocessor systems.

Cache coherency design in pentium III SMP system

This paper analyzes cache coherency mechanism from the view of system. It firstly discusses cache-memory hierarchy of Pentium III SMP system, including memory area distribution, cache attributes control and bus transaction. Secondly it analyzes hardware snoopy mechanism of P6 bus and MESI state transitions adopted by Pentium III. Based on these, it focuses on how multiprocessors and the P6 bus cooperate to ensure cache coherency of the whole system, and gives the key of cache coherency design.

On cache memory hierarchy for Chip-Multiprocessor

Advances in integrated circuit technology have led the microprocessor design to more challenging areas by providing a high budget of transistors. The one billion microprocessor is not that far from now. This leads microarchitects to look for efficient ways to make use of this feature. Chip-Multiprocessor (CMP) is one of the strong candidates in this area. Memory system for any multiprocessor is a pivotal part and can boost the performance or decrease it dramatically. The purpose of this study is to investigate the most appropriate cache memory hierarchy for CMP. We provide comparisons between several candidates and present a conclusion that can lead to better performance for CMP as well as the configurations that must be avoided.

Cache memory replacement policy for a uniprocessor system

Cache memory hierarchies are used to buffer those portions of main memory with the most frequent use by the CPU. As cache memory is very costly, good design techniques must consider small cache sizes maintaining high levels of use (hit ratio) and ease of implementation. The memory replacement policy is important in maintaining a high hit ratio. Most replacement policies used are easily implemented when the main memory has fixed page locations. A new cache algorithm using a variable page configuration is explained in terms of program behaviour.