Level Cache Memory Research Articles

Split Ways : An Efficient Replacement Policy for Larger Sized Cache Memory

As the data requirement of today’s application is increasing the size of the cache memory in a multicore processor is also increasing. A multicore processor has many levels of cache memory. The Last Level Cache (LLC) is normally shared by all the cores and its size must be large enough to handle today’s data intensive applications. Such larger sized set-associative LLC facing major challenges to efficiently implement the replacement policy of its sets. As the number of ways are increasing in the LLC, the replacement policy becomes a bottleneck of the system. To improve the performance of the system and to reduce the hardware overhead, in this paper we propose a simple but hardware efficient replacement policy for the larger sized LLCs. We call the techniques as SplitWays as it divides (splits) the ways of a set into multiple groups called wayGroups. Each wayGroup maintains its own replacement policy. Experimental analysis using full-system simulator found that the proposed technique reduces the hardware overhead by up to 66% without any performance degradation..

Performance and Fairness Improvement on CMPs Considering Bandwidth and Cache Utilization

Chip multiprocessors (CMPs) have become dominant both in server and embedded domain as they accommodate an increasing amount of cores in order to satisfy the workload demands. However, when applications run concurrently, they compete for shared resources, such as Last Level Cache (LLC) and main memory bandwidth. Applications are affected in various ways by contention, and uneven degradation makes the system unreliable and the overall performance unpredictable. The goal of this work is to improve performance by sophisticated grouping that balances bandwidth and LLC requirements, while at the same time providing a fair execution environment by prioritizing applications that experience the least accumulated progress. The proposed scheduler achieves an average performance gain of 16 percent over the Linux scheduler and 6.3 percent over another performance-oriented scheduler. Additionally, it keeps unfairness very close to two fairness-oriented schedulers.

PR-LRU: partial random LRU technique for performance improvement of last level cache

As chip multiprocessors (CMP) have become eminent in all areas of computing, it is inevitable for the operating system to schedule processes efficiently on different cores. These multi-cores pose different challenges of which shared resource contention is the dominant one, as cores share resources like last level cache (LLC) and main memory. This can lead to poor and unpredictable performance of the threads running on the system. The cache replacement policy of LLC becomes critical in managing the cache data in an efficient way. Though prominent, least recently used (LRU) algorithm has some issues with applications which do not follow the temporal locality pattern. This study proposes a modification to the LRU algorithm where a random selection of the victim from 'N' LRU blocks yields better results than the conventional method. The evaluation of the algorithm is carried out using Multi2sim simulator using Parsec and Splash2 benchmarks. The results show an overall performance improvement in hit ratio up to 6% and 2% over LRU for PARSEC and SPLASH2 benchmarks respectively.

PR-LRU: partial random LRU technique for performance improvement of last level cache

As chip multiprocessors (CMP) have become eminent in all areas of computing, it is inevitable for the operating system to schedule processes efficiently on different cores. These multi-cores pose different challenges of which shared resource contention is the dominant one, as cores share resources like last level cache (LLC) and main memory. This can lead to poor and unpredictable performance of the threads running on the system. The cache replacement policy of LLC becomes critical in managing the cache data in an efficient way. Though prominent, least recently used (LRU) algorithm has some issues with applications which do not follow the temporal locality pattern. This study proposes a modification to the LRU algorithm where a random selection of the victim from 'N' LRU blocks yields better results than the conventional method. The evaluation of the algorithm is carried out using Multi2sim simulator using Parsec and Splash2 benchmarks. The results show an overall performance improvement in hit ratio up to 6% and 2% over LRU for PARSEC and SPLASH2 benchmarks respectively.

Fast modeling DRAM access latency based on the LLC memory stride distribution without detailed simulations

The last level cache (LLC) memory accessing patterns influence the DRAM performance significantly due to their determination on the DRAM Row buffer hit rates (RBH), which are highly related to the DRAM access latency. However, it is too difficult to accurately quantify these effects without the LLC memory accessing traces from detailed simulations.This paper proposes an analytical model of DRAM access latency based on the RBH analysis. By exploring the relationship between the LLC spatial localities (described with the LLC stride distribution in this paper) and different RBH cases, the unknown parameters of the model can be quantified. Instead of detailed simulations, the LLC stride distributions are deduced from the stride distributions extracted from the software trace and the LLC miss rate estimated by the multi-level cache behavior model in our prior works, which significantly decreases the time overhead of our model.Fifteen benchmarks, chosen from Mobybench 2.0, Mibench 1.0 and Mediabench II, are utilized to evaluate the model's accuracy. Compared to the results of full simulations by gem5 and DRAMSim2, the average error of our model is lower than 7%, while the prediction process can be sped up by 50 times on average.

Reducing Competitive Cache Misses in Modern Processor Architectures

The increasing number of threads inside the cores of a multicore processor, and competitive access to the shared cache memory, become the main reasons for an increased number of competitive cache misses and performance decline. Inevitably, the development of modern processor architectures leads to an increased number of cache misses. In this paper, we make an attempt to implement a technique for decreasing the number of competitive cache misses in the first level of cache memory. This technique enables competitive access to the entire cache memory when there is a hit - but, if there are cache misses, memory data (by using replacement techniques) is put in a virtual part given to threads, so that competitive cache misses are avoided. By using a simulator tool, the results show a decrease in the number of cache misses and performance increase for up to 15%. The conclusion that comes out of this research is that cache misses are a real challenge for future processor designers, in order to hide memory latency.

Multi-objective optimization applied to unified second level cache memory hierarchy tuning aiming at energy and performance optimization

Cache memory optimization has an important impact on the energy consumption of the embedded system. However, optimization is a hard task due to the large exploration space and conflicting objectives. In this work five multiobjective optimization techniques are applied to cache memory optimization. The PESA-II, NSGAII, SPEA2, PAES and NPGA approaches were applied to 18 different applications from MiBench and PowerStone benchmark suites. Results compared the quality of results in terms of the metrics of general distance, diversity, hypervolume and precision. All techniques had good performance to cache optimization, but PESA-II showed a better performance for all metrics analyzed, having better results in 83% and 88% of cases, compared with the metrics of generational distance and hypervolume, respectively. Additionally, PESA-II needs to explore only 1.47% of exploration space, finding solutions near to Pareto Optimal.

A shared cache-aware Task scheduling strategy for multi-core systems

In the past few years, multi-core processors incorporating four, six, eight, or more cores on a single die have become ubiquitous. Those cores, having their own private caches, often share a higher level cache memory, which leads to compete among different tasks. This can seriously affect the avera ge performance of multi-core systems as the probability of cache hit could be lowered. In realizing this, we study the problem of scheduling bag-of-tasks (BoT) applications with shared cache constraint on multi-core systems. We first use cache space isolation techniques to divide shared caches into partitions. Then, we give a motivational example and outline the shared cache aware scheduling problem of multi-core systems. Finally, to provide an optimum solution for this problem, we propose a heuristic shared cache contention aware scheduling (SCAS) algorithm on multi-core systems. Our extensive simulation performance evaluation study clearly demonstrate that our proposed SCAS algorithm outperforms the existing traditional scheduling algorithm Min-min and the modified algorithm MSCAS in terms of schedule length and average response time.

Modeling Cache Memory Utilization on Multicore Using Common Pool Resource Game on Cellular Automata

Recent computing architectures are implemented by shared memory technologies to alleviate the high latency experienced by off-chip memory transfers, but the high architectural complexity of modern multicore processors has presented many questions. To tackle the design of efficient algorithms scheduling workloads over available cores, this article presents a parallel bioinspired model that simulates the utilization of shared memory on multicore systems. The proposed model is based on cellular automata (CA) and coupled with game theory principles. CA are selected due to their inherent parallelism and especially their ability to incorporate inhomogeneities. Furthermore, the novelty of the model is realized on the fact that multilevel CA are used to simulate the different levels of cache memory usually found in multicore processors. These characteristics make the model able to cope with the increasing diversity of cache memory hierarchies on modern and future processors. Nonetheless, by acquiring data from hardware performance counters and processing them with the proposed model online, the performance of the system can be calculated and a better scheduling strategy can be adopted in real time. The CA-based model was verified on the behavior of a real multicore system running a multithreaded application, and it successfully simulated the acceleration achieved by an increased number of cores available for the execution of the workload. More specifically, the example of common pool resource from game theory was used with two variations: a static and a variable initial endowment. The static variation of the model approximates slightly better the acceleration of a workload when the number of available processor cores increases, whereas the dynamic variation simulates better the moderate differences due to operation system’s scheduler alternations on the same amount of cores.

Multi-objective optimization of energy consumption and execution time in a single level cache memory for embedded systems

Current embedded systems are specifically designed to run multimedia applications. These applications have a big impact on both performance and energy consumption. Both metrics can be optimized selecting the best cache configuration for a target set of applications. Multi-objective optimization may help to minimize both conflicting metrics in an independent manner. In this work, we propose an optimization method that based on Multi-Objective Evolutionary Algorithms, is able to find the best cache configuration for a given set of applications. To evaluate the goodness of candidate solutions, the execution of the optimization algorithm is combined with a static profiling methodology using several well-known simulation tools. Results show that our optimization framework is able to obtain an optimized cache for Mediabench applications. Compared to a baseline cache memory, our design method reaches an average improvement of 64.43 and 91.69% in execution time and energy consumption, respectively.

Tuning the cache memory usage in tomographic reconstruction on standard computers with Advanced Vector eXtensions (AVX).

Cache blocking is a technique widely used in scientific computing to minimize the exchange of information with main memory by reusing the data kept in cache memory. In tomographic reconstruction on standard computers using vector instructions, cache blocking turns out to be central to optimize performance. To this end, sinograms of the tilt-series and slices of the volumes to be reconstructed have to be divided into small blocks that fit into the different levels of cache memory. The code is then reorganized so as to operate with a block as much as possible before proceeding with another one. This data article is related to the research article titled Tomo3D 2.0 – Exploitation of Advanced Vector eXtensions (AVX) for 3D reconstruction (Agulleiro and Fernandez, 2015) [1]. Here we present data of a thorough study of the performance of tomographic reconstruction by varying cache block sizes, which allows derivation of expressions for their automatic quasi-optimal tuning.

A Performance Study of a Dual Xeon-Phi Cluster for the Forward Modelling of Gravitational Fields

With at least 60 processing cores, the Xeon-Phi coprocessor is a truly multicore architecture, which consists of an interconnection speed among cores of 240 GB/s, two levels of cache memory, a theoretical performance of 1.01 Tflops, and programming flexibility, all making the Xeon-Phi an excellent coprocessor for parallelizing applications that seek to reduce computational times. The objective of this work is to migrate a geophysical application designed to directly calculate the gravimetric tensor components and their derivatives and in this way research the performance of one and two Xeon-Phi coprocessors integrated on the same node and distributed in various nodes. This application allows the analysis of the design factors that drive good performance and compare the results against a conventional multicore CPU. This research shows an efficient strategy based on nested parallelism using OpenMP, a design that in its outer structure acts as a controller of interconnected Xeon-Phi coprocessors while its interior is used for parallelyzing the loops. MPI is subsequently used to reduce the information among the nodes of the cluster.

Design of BiCMOS SRAMs for high‐speed SiGe applications

This study documents the speeds of various SRAM buffer memories that are possible in a contemporary fast SiGe heterojunction bipolar transistor (HBT) BiCMOS process. An SRAM in a 0.13 µm HBT BiCMOS technology using current mode logic (CML)-style circuits serves as a basis for the discussion. This basic SRAM design features a CML decoder, CML word line driver, bipolar sense amplifier for achieving high speed and CMOS 6T memory cells for high density. The BiCMOS technology is especially useful for realising ultra-high-speed SRAMs for low level cache memory in high-clock rate computer systems, but when reorganised can also be utilised in analogue-to-digital converter (ADC) systems to store digitalised data. Speed and power tradeoffs can be made using different bias strategies, CML logic levels and different generations of SiGe HBTs. A demonstrated 128 kb SRAM macro consumes 2.7 W at 4 GHz using a −3.4 and −1.5 V supply voltage for the bipolar and CMOS circuits, respectively, and has dimensions of 3.5 mm × 3.6 mm by using IBM 8HP SiGe technology, which provides an HBT with a fT of 210 GHz. This macro can be integrated into large scale, ultra-wide bus SRAMs using heterogeneous silicon and 3D technology. Simulation indicates that with the next generation of SiGe HBTs, this SRAM macro can operate at 5 GHz, while consuming the same amount of power or alternatively consume 0.73 W, which is 73% less power consumption compared to 8HP, while operating with the same frequency of 4 GHz. Reorganising the memory for a 4 way-interleaved ADC, it can accept data written at 9.5 GS/s for 8HP designs, and 11.9 GS/s for 8XP designs.

HIT RATE MAXIMIZATION BY LOGICAL CACHE PARTITIONING IN A MULTI-CORE ENVIRONMENT

It is imperative for any level of cache memory in a multi-core architecture to have a well defined, dynamic replacement algorithm in place to ensure consistent superlative performance. The most prevalently used LRU replacement policy does not acquaint itself dynamically to the changes in the workload. As a result, it can lead to sub-optimal performance for certain applications whose workloads exhibit frequently fluctuating patterns. To overcome the limitation of this conventional LRU approach, our paper proposes a novel counter-based replacement technique which logically partitions the cache elements into four zones based on their ‘likeliness’ to be referenced by the processor in the near future. Categorizing the elements into different zones is achieved with the help of a 3-bit counter that is associated with every cache line. On a cache hit, the corresponding element is promoted from one zone to another zone. Replacement candidates are chosen from the zones in the ascending order of their ‘likeliness factor’ (i.e.,) the first search space for the victim would be the never likely to be referenced zone, followed by the subsequent zones till the most likely to be referenced zone is reached. Periodic zone demotion of elements also occurs to make sure that stale data does not pollute the cache. Experimental results obtained by using the PARSEC benchmarks have shown almost 7% improvement in the overall number of hits and 3% improvement in the average cache occupancy percentage when compared to LRU algorithm.

A dynamic adaptive converter and management for PRAM-based main memory

As DRAM-based main memory becomes a dominant factor in the energy consumption and cost of any computer system, new non-volatile memory technologies have been proposed to replace DRAMs. For example, PRAM is emerged as a leading alternative for main memory technology. However, the access latency of PRAM is significantly slower than that of DRAM and an interfacing converter is required to at least partly alleviate this latency difference. The interfacing converter sits between PRAM-based main memory and the last level of cache memory. In this paper, we present a proposed dynamic adaptive converter and its management scheme for PRAM-based main memory. In addition to overcoming long access latency, it provides enhanced endurance. The adaptive converter is composed of an aggressive streaming buffer to make better use of spatial locality by dynamically varying fetch size, a write buffer to improve endurance limit, and an adaptive filtering buffer to better utilize temporal locality. Our experimental results show that we can reduce buffer miss rate by about 59%, compared with using a single buffer structure with same space. Our approach also hides PRAM access latency more effectively. It improves the number of superblocks pre-fetched from main memory by 25%. Therefore, the converter shows its effectiveness comparable to a case with larger buffer space, without expending the extra power.

Model-based cache-aware dispatching of object-oriented software for multicore systems

In recent years, processor technology has evolved towards multicore processors, which include multiple processing units (cores) in a single package. Those cores, having their own private caches, often share a higher level cache memory dedicated to each processor die. This multi-level cache hierarchy in multicore processors raises the importance of cache utilization problem. Assigning parallel-running software components with common data to processor cores that do not share a common cache increases the number of cache misses. In this paper we present a novel approach that uses model-based information to guide the OS scheduler in assigning appropriate core affinities to software objects at run-time. We build graph models of software and cache hierarchies of processors and devise a graph matcher algorithm that provides mapping between these two graphs. Using this mapping we obtain candidate core sets that each software object can be affiliated with at run-time. These affiliations are determined based on the idea that software components that have the potential to share common data at run-time should run on cores that share a common cache. We also develop an object dispatcher algorithm that keeps track of object affiliations at run-time and dispatches objects by using the information from the compile-time graph matcher. We apply our approach on design pattern implementations and two different application program running on servers using CFS scheduling. Our results show that cache-aware dispatching based on information obtained from software model, decreases number of cache misses significantly and improves CFS’ scheduling performance.

Dynamic QoS management for chip multiprocessors

With the continuing scaling of semiconductor technologies, chip multiprocessor (CMP) has become the de facto design for modern high performance computer architectures. It is expected that more and more applications with diverse requirements will run simultaneously on the CMP platform. However, this will exert contention on shared resources such as the last level cache, network-on-chip bandwidth and off-chip memory bandwidth, thus affecting the performance and quality-of-service (QoS) significantly. In this environment, efficient resource sharing and a guarantee of a certain level of performance is highly desirable. Researchers have proposed different frameworks for providing QoS. Most of these frameworks focus on individual resource for QoS management. Coordinated management of multiple QoS-aware shared resources at runtime remains an open problem. Recently, there has been work that proposed a class-of-serviced based framework to jointly managing cache, NoC and memory resources simultaneously. However, the work allocates shared resources statically at the beginning of application runtime, and do not dynamically track, manage and share shared resources across applications. In this article, we address this limitation by proposing dynamic resource management policies that monitor the resource usage of applications at runtime, then steals resources from the high-priority applications for lower-priority ones. The goal is to maintain the targeted level of performance for high-priority applications while improving the performance of lower-priority applications. We use a PI (Proportional-Integral gain) feedback controller based technique to maintain stability in our framework. Our evaluation results show that our policy can improve performance for lower-priority applications significantly while maintaining the performance for high-priority application, thus demonstrating the effectiveness of our dynamic QoS resource management policy.

Exploring DRAM Last Level Cache for 3D Network-on-Chip Architecture

In this paper, we implement and analyze different Network-on-Chip (NoC) designs with Static Random Access Memory (SRAM) Last Level Cache (LLC) and Dynamic Random Access Memory (DRAM) LLC. Different 2D/3D NoCs with SRAM/DRAM are modeled based on state-of-the-art chips. The impact of integrating DRAM cache into a NoC platform is discussed. We explore the advantages and disadvantages of DRAM cache for NoC in terms of access latency, cache size, area and power consumption. We present benchmark results using a cycle accurate full system simulator based on realistic workloads. Experiments show that under different workloads, the average cache hit latencies in two DRAM based designs are increased by 12.53% (2D) and reduced by 27.97% (3D) respectively compared with the SRAM. It is also shown that the power consumption is a tradeoff consideration in improving the cache hit latency of DRAM LLC. Overall, the power consumption of 3D NoC design with DRAM LLC has reduced 25.78% compared with the SRAM design. Our analysis and experimental results provide a guideline to design efficient 3D NoCs with DRAM LLC.

Efficient AdHoc Ondemand Distance Vector Routing Protocol using Link State Algorithm

developed a new algorithm for route discovery, nodes management, and mobility handling for on-demand cache routing on mobile Ad-Hoc networks (MANET). We used Ad- Hoc On demand Distance Vector (AODV) protocol as the better known reactive protocol, as well as using Link State algorithm of the Optimize Link State Routing (OLSR) protocol together. We used two levels of caches memory L-1 and L-2 along with link state routing table for each node. Which maintaining by using the algorithm of OLSR, which working under the AODV protocol. For mobility handling, we used link state algorithm working under AODV to manage node addition, deletion and movement in the network efficiently. We used the Network simulator NS-2 version 2.29 to show the results comparing with the AODV used just cache memory, and comparing with AODV without cache memory. The results shows that our algorithms outperform comparing with AODV without cache memories, and AODV with two levels of cache memory on packet delivery rate, where the link state routing protocol is used to distribute and maintain routing information among various nodes within a domain by using two messages which are Hello messages and Topology Control messages (TC) .

Area-efficient floorplans and interconnects for homogeneous multi-core architectures

Homogeneous multi-core architectures come in a variety of floorplans. A large portion of the chip area is occupied by the second level cache memories and the interconnect. Usually, the floorplan is dictated by the interconnect type and the second level cache memory size and by whether large resources on the chip are shared or private. We consider floorplans for a 16-core architecture consisting of identical cores and estimate their areas based on two different processor cores. Then, we focus on an area-efficient and performance-scaling four-partition crossbar-interconnected architecture with third level cache memory and conduct performance modelling and evaluation of the multi-core architecture's memory system. With a database workload and a first level cache miss rate under 20%, the Average Memory Access Time (AMAT) is estimated to be under 20 processor cycles. With higher memory contention resulting into longer bridge queue wait times, the various cache miss rates rate make a more pronounced effect on the Chip Multiprocessors AMAT, necessitating design measures like proper cache sizing. When the size of shared resources becomes too large, recent work in sharing and partitioning large resources in CMP architectures becomes crucial.