Cache Simulation Research Articles

Study of an efficient simulation method

Trace-driven simulation has been widely used to evaluate the performance of cache memory. Prior to this investigation, a number of research projects were proposed for reducing the long execution time of trace-driven simulation. In the paper, a one-pass algorithm, called the Hash-RM algorithm, is proposed which is both simple and efficient. The Hash-RM algorithm facilitates simulation by reducing the average search depth and using the right-match comparison. Since the Hash-RM algorithm can be adapted to simulation of direct-mapped, set-associative and fully-associative caches simultaneously, it is compared to the all-associativity algorithm proposed by Hill and Smith. For instruction traces, the results indicated that the present algorithm is not making great progress because of the preferable locality. For data traces, however, the results showed that the Hash-RM algorithm is much faster than the all-associativity algorithm. Implications of the results are discussed in detail.

Randomized cache placement for eliminating conflicts

Applications with regular patterns of memory access can experience high levels of cache conflict misses. In shared-memory multiprocessors conflict misses can be increased significantly by the data transpositions required for parallelization. Techniques such as blocking which are introduced within a single thread to improve locality, can result in yet more conflict misses. The tension between minimizing cache conflicts and the other transformations needed for efficient parallelization leads to complex optimization problems for parallelizing compilers. This paper shows how the introduction of a pseudorandom element into the cache index function can effectively eliminate repetitive conflict misses and produce a cache where miss ratio depends solely on working set behavior. We examine the impact of pseudorandom cache indexing on processor cycle times and present practical solutions to some of the major implementation issues for this type of cache. Our conclusions are supported by simulations of a superscalar out-of-order processor executing the SPEC95 benchmarks, as well as from cache simulations of individual loop kernels to illustrate specific effects. We present measurements of instructions committed per cycle (IPC) when comparing the performance of different cache architectures on whole-program benchmarks such as the SPEC95 suite.

Analytical modeling of set-associative cache behavior

Cache behavior is complex and inherently unstable, yet it is a critical factor affecting program performance. A method of evaluating cache performance is required, both to give quantitative predictions of miss-ratio and information to guide optimization of cache use. Traditional cache simulation gives accurate predictions of miss-ratio, but little to direct optimization, Also, the simulation time is usually far greater than the program execution time. Several analytical models have been developed, but concentrate mainly on direct-mapped caches, often for specific types of algorithm, or to give qualitative predictions. Novel analytical models of cache phenomena are presented, applicable to numerical codes consisting mostly of array operations in looping constructs. Set-associative caches are considered, through an extensive hierarchy of cache reuse and interference effects, including numerous forms of temporal and spatial locality. Models of each effect are given which, when combined, predict the overall miss-ratio. An advantage is that the models also indicate sources of cache interference. The accuracy of the models is validated through example program fragments. The predicted miss-ratios are compared with simulations and shown typically to be within 15 percent. The evaluation time of the models is shown to be independent of the problem size, generally several orders of magnitude faster than simulation.

Combining trace sampling with single pass methods for efficient cache simulation

The design of the memory hierarchy is crucial to the performance of high performance computer systems. The incorporation of multiple levels of caches into the memory hierarchy is known to increase the performance of high end machines, but the development of architectural prototypes of various memory hierarchy designs is costly and time consuming. In this paper, we will describe a single pass method used in combination with trace sampling techniques to produce a fast and accurate approach for simulating multiple sizes of caches simultaneously.

Modelling Cache Pollution

Cache memories are commonly used to reduce the coat of accessing data and instructions in memory. Misses in the cache can severely reduce system performance. It is therefore beneficial to try to anticipate cache misses in an attempt to reduce their frequency. Prefetching can be used to prime a cache with the addresses that will most probably be needed in the future. One side-effect of prefetching is that cache lines that would had been used in the future will be supplanted by prefetdied data and instructions that may never be used. This is known as cache pollution. We model this effect using tracedriven cache simulation, and propose a new mechanism called the Prefetch Buffer Filter (PBF) that, combined with tagged prefetching, significantly reduces the cache pollution while also reducing the number of memory references issued for prefetching. Through the use of traced-driven simulation we demonstrate that using a 16-entry PBF for both the instruction and the data stream can consistently reduce data cache miss rates while effectively reducing memory traffic.

Compile-Time Cache Performance Prediction and Its Application to Tiling

Tiling has been used by parallelizing compilers to define fine-grain parallel tasks and to optimize cache performance. In this paper we present a novel compile-time technique, called miss-driven cache simulation, for determining loop tile sizes that achieve the highest cache hit-rate. The widening disparity between a processor's peak instruction rate and main memory access time in modern computer systems makes this kind of optimization increasingly important for overall program efficiency. Our simulation technique generates only those references of a loop nest that may generate a cache memory miss and processes them on an architecturally accurate cache model at compile-time. Processing only a small portion of the memory reference trace of a program yields simulation speeds in the millions of memory references per second on workstations, with statistics of misses per reference and inter-reference interference counts gathered at the same time. These simulation speeds and statistics allow for the accurate analysis of the impact of cache optimizations at compile-time. We discuss the results of applying this method to guide loop tiling for such commonly used computational kernels as matrix multiplication and Jacobi iteration for various cache parameters.

The energy efficiency of IRAM architectures

Portable systems demand energy efficiency in order to maximize battery life. IRAM architectures, which combine DRAM and a processor on the same chip in a DRAM process, are more energy efficient than conventional systems. The high density of DRAM permits a much larger amount of memory on-chip than a traditional SRAM cache design in a logic process. This allows most or all IRAM memory accesses to be satisfied on-chip. Thus there is much less need to drive high-capacitance off-chip buses, which contribute significantly to the energy consumption of a system. To quantify this advantage we apply models of energy consumption in DRAM and SRAM memories to results from cache simulations of applications reflective of personal productivity tasks on low power systems. We find that IRAM memory hierarchies consume as little as 22% of the energy consumed by a conventional memory hierarchy for memory-intensive applications, while delivering comparable performance. Furthermore, the energy consumed by a system consisting of an IRAM memory hierarchy combined with an energy efficient CPU core is as little as 40% of that of the same CPU core with a traditional memory hierarchy.

Analysis of caching algorithms for distributed file systems

When picking a cache replacement policy for file systems, LRU (Least Recently Used) has always been the obvious choice, because of the temporal locality found in programs and data. However, in the case of Sun NFS servers much of the locality is filtered out by the client cache. It has been conjectured that this filtering of locality by the client caches render LRU ineffective as a server cache replacement policy. This study disproves the conjecture by simulating a NFS server cache with real world traces. Traces were taken of NFS read and write requests sent to an NFS server. These traces were then run through a cache simulator using six different cache replacement policies: Least Recently Used (LRU), Least Frequently Used (LFU), Frequency Based Replacement (FBR), First In First Out (FIFO), Random (RAND), and Optimal (OPT). RAND and OPT were used to provide lower and upper bounds on performance. Results show that LRU is an effective NFS server cache replacement policy and frequency based tend to exhibit erratic behavior in the presence of temporal locality and sequentially accessed files.

PROGRAM OPTIMIZATION BASED ON COMPILE-TIME CACHE PERFORMANCE PREDICTION

We present a novel, compile-time method for determining the cache performance of the loop nests in a program. The cache hit-rates are produced by applying the reference string, determined during compilation, to an architecturally parameterized cache simulator. We also describe a heuristic that uses this method for compile-time optimization of loop ranges in iteration-space blocking. The results of the loop program optimizations are presented for different parallel program benchmarks and various processor architectures, such as IBM SP1 RS/6000, the SuperSPARC, and the Intel 1860.

Mathematical analysis of statistical sampling for estimating computer cache performance

Computer cache memory simulations consume a lot of resources, both in terms of disk space and CPU time. A statistical approach to this problem is proposed in which program traces are sampled at certain intervals, significantly reducing the required computer resources. Using, both the mathematical analysis and experimentation, we show that this approach can achieve a high accuracy in predicting the hit and miss ratios for cache simulations

Will parallel simulation come to an end?

In 1987, I worked on the performance issues of the multi-processor caches under the supervision of Professor Jean-Loup Baer. We used the trace-driven simulation approach and found that the process might be too time consuming. Then we investigated the possibility of utilizing multiple processors to speed up the simulation procedure. At this point, my thesis advisor Professor Edward Lazowska joined in the project. By exploiting the communication patterns of the cache protocols under study, we observed the number of synchronization operations among the processors (that execute the cache simulation) can be significantly reduced. Thus, good speedup could be achieved. We published this exciting results in the Proceedings of the SCS Multi-Conferences on Distributed Simulation. After we finished this project, I felt that "It is too expensive to develop a parallel simulation program just for one application. It is desirable to develop a general simulation system." Jean-Loup showed me two papers:1. Chandy, K. M., and Misra, J., "Distributed Simulation: A Case Study in Design and Verification of Distributed Programs." IEEE Trans. on Software Engineering, SE-5, 1979.2. Jefferson, D., "Virtual Time." ACM Transactions on Programming Languages and Systems, July, 1985.

Empirical study of parallel trace-driven LRU cache simulators

This paper reports on the performance of four parallel algorithms for simulating an associative cache operating under the LRU (Least-Recently-Used) replacement policy. Three of the algorithms are implemented on the MasPar MP-2. Another algorithm is a parallelization of an efficient serial algorithm on the Intel Paragon. We assess the strengths and weaknesses of these algorithms as a function of problem size and characteristics, and compare their performance on traces derived from execution of three SPEC92 benchmark programs.

On the effective use of a cache memory simulator in a computer architecture course

In this paper, we describe how a cache memory simulator can be effectively used to enhance the understanding of cache memories in a computer architecture course. Various types of course assignments that are described in this paper require the use of the simulator to understand effects of varying different cache parameters on the system performance, to analyze cache memory designs, to synthesize cache designs, and to incorporate advanced caching concepts like victim caching, stall caching, and multilevel caching by modifying the simulator. Dinero, a cache simulator available in the public domain, has been enhanced and used in the course. Our experience of using the simulator is described. >

Trace Driven Pipeline and Cache Simulation of Multithreaded Computers

A technique for applying trace-driven simulation to cached multithreaded ma chines with dynamic thread scheduling is presented. A small amount of constraint information is added to each trace record, permitting the correct order of execution in the pipeline to be constructed. Strategies like context switching on a cache miss, in which the behaviour of the cache affects the instruction dispatch order, can be modelled with this technique. The design of a multithreaded pipeline and cache trace-driven simulation system is described. This system allows thread- scheduled multithreaded processors, which are not simulatable by conventional trace- driven techniques, to be efficiently and accurately simulated. Sample simulation results illustrate the flexibility of this simulation technique.

Massively parallel algorithms for trace-driven cache simulations

Considers the use of massively parallel architectures to execute a trace-driven simulation of a single cache set. A method is presented for the least-recently-used (LRU) policy, which, regardless of the set size C, runs in time O(log N) using N processors on the EREW (exclusive read, exclusive write) parallel model. A simpler LRU simulation algorithm is given that runs in O(C log N) time using N/log N processors. We present timings of this algorithm's implementation on the MasPar MP-1, a machine with 16384 processors. A broad class of reference-based line replacement policies are considered, which includes LRU as well as the least-frequently-used (LFU) and random replacement policies. A simulation method is presented for any such policy that, on any trace of length N directed to a C line set, runs in O(C log N) time with high probability using N processors on the EREW model. The algorithms are simple, have very little space overhead, and are well suited for SIMD implementation.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Experience with Teaching Computer Architecture

Experience with teaching computer architecture A middle-level undergraduate course in computer architecture is described, in terms of both formal coursework and laboratory exercises. These exercises are undertaken using a PDP8 simulator for the Apple Macintosh, an MC68000-based microcomputer, and RISC and Cache simulators for Unix.

Efficient simulation of caches under optimal replacement with applications to miss characterization

Cache miss characterization models such as the three Cs model are useful in developing schemes to reduce cache misses and their penalty. In this paper we propose the OPT model that uses cache simulation under optimal (OPT) replacement to obtain a finer and more accurate characterization of misses than the three Cs model. However, current methods for optimal cache simulation are slow and difficult to use. We present three new techniques for optimal cache simulation. First, we propose a limited lookahead strategy with error fixing, which allows one pass simulation of multiple optimal caches. Second, we propose a scheme to group entries in the OPT stack, which allows efficient tree based fully-associative cache simulation under OPT. Third, we propose a scheme for exploiting partial inclusion in set-associative cache simulation under OPT. Simulators based on these algorithms were used to obtain cache miss characterizations using the OPT model for nine SPEC benchmarks. The results indicate that miss ratios under OPT are substantially lower than those under LRU replacement, by up to 70% in fully-associative caches, and up to 32% in two-way set-associative caches.

An analysis of cache performance for a hypercube multicomputer

Multicomputer cache simulation results derived from address traces collected from an Intel iPSC/2 hypercube multicomponent are presented. The primary emphasis is on examining how increasing the number of processor nodes executing a parallel application affects the overall multicomputer cache performance. The effects on multicomputer direct-mapped cache performance of application-specific data partitioning, data access patterns, communication distribution, and communication frequency are illustrated. The effects of system accesses on total cache performance are explored, as well as the reasons for application-specific differences in cache behavior for system and user accesses. Comparing user code results with full user and system code analysis reveals the significant effect of system accesses, and this effect increases with multicomputer size. The time distribution of an application's message-passing operations is found to more strongly affect cache performance than the total amount of time spent in message-passing code. >

Cache behavior of combinator graph reduction

The results of cache-simulation experiments with an abstract machine for reducing combinator graphs are presented. The abstract machine, called TIGRE, exhibits reduction rates that, for similar kinds of combinator graphs on similar kinds of hardware, compare favorably with previously reported techniques. Furthermore, TIGRE maps easily and efficiently onto standard computer architectures, particularly those that allow a restricted form of self-modifying code. This provides some indication that the conventional "stored program" organization of computer systems is not necessarily an inappropriate one for functional programming language implementations. This is not to say, however, that present day computer systems are well equipped to reduce combinator graphs. In particular, the behavior of the cache memory has a significant effect on performance. In order to study and quantify this effect, trace-driven cache simulations of a TIGRE graph reducer running on a reduced instruction-set computer are conducted. The results of these simulations are presented with the following hardware-cache parameters varied: cache size, block size, associativity, memory update policy, and write-allocation policy. To begin with, the cache organization of a commercially available system is used and then the performance sensitivity with respect to variations of each parameter are measured. From the results of the simulation study, a conclusion is made that combinator-graph reduction using TIGRE runs most efficiently when using a cache memory with an allocate-on-write-miss strategy, moderately large block size (preferably with subblock placement), and copy-back memory updates.

Address tracing of parallel systems via TRAPEDS

Trace-driven simulation is an important aid in performance analysis of computer systems. Capturing address traces to use in these simulations, however, is a difficult problem for parallel processor architectures. A technique termed TRAPEDS modifies executable code (at the assembly language level) to dynamically collect the address trace from executing code. TRAPEDS has recently been implemented on both a hypercube multicomputer and a shared-memory multiprocessor. Particular attention is focused on strategies for efficiently and accurately collecting traces from both classes of parallel machines. The iPSC/2 hypercube multicomputer implementation traces both user and system code, and performs simulation on-the-fly to avoid large storage costs. Strategies are detailed for mitigating address trace distortion when collecting operating system traces. The Encore Multimax multiprocessor implementation uses a timer-based approach to reflect the interleaving of the processor traces and stores the traces to disc. Time and space overhead results are presented for both TRAPEDS implementations. Experimental cache simulation results derived from iPSC/2 address traces are presented to illustrate the importance of tracing operating system references.