Cache Parameters Research Articles

Cache optimization for embedded processor cores

Embedded microprocessor cores are increasingly being used in embedded and mobile devices. The software running on these embedded microprocessor cores is often a priori known; thus, there is an opportunity for customizing the cache subsystem for improved performance. In this work, we propose an efficient algorithm to directly compute cache parameters satisfying desired performance criteria. Our approach avoids simulation and exhaustive exploration, and, instead, relies on an exact algorithmic approach. We demonstrate the feasibility of our algorithm by applying it to a large number of embedded system benchmarks.

Simulation study of memory performance of SMP multiprocessors running a TPC-W workload

The infrastructure to support electronic commerce is one of the areas where more processing power is needed. A multiprocessor system can offer advantages for running electronic commerce applications. The memory performance of an electronic commerce server, i.e. a system running electronic commerce applications, is evaluated in the case of shared-bus multiprocessor architecture. The software architecture of this server is based on a three-tier model and the workloads have been setup as specified by the TPC-W benchmark. The hardware configurations are: a single SMP running tiers two and three, and two SMPs each one running a single tier. The influence of memory subsystem on performance and scalability is analysed and several solutions aimed at reducing the latency of memory considered. After initial experiments, which validate the methodology, choices for cache, scheduling algorithm, and coherence protocol are explored to enhance performance and scalability. As in previous studies on shared-bus multiprocessors, it was found that the memory performance is highly influenced by cache parameters. While scaling the machine, the coherence overhead weighs more and more on the memory performance. False sharing in the kernel is among the main causes of this overhead. Unlike previous studies, passive sharing i.e. the useless sharing of the private data of the migrating processes, is shown to be an important factor that influences performance. This is especially true when multiprocessors with a higher number of processors are considered: an increase in the number of processors produces real benefits only if advanced techniques for reducing the coherence overhead are properly adopted. Scheduling techniques limiting process migration may reduce passive sharing, while restructuring techniques of the kernel data may reduce false sharing misses. However, even when process migration is reduced through cache-affinity techniques, standard coherence protocols like MESI protocol don't allow the best performance. Coherence protocols such as PSCR and AMSD produce performance benefits. PSCR, in particular, eliminates coherence overhead due to passive sharing and minimises the number of coherence misses. The adoption of PSCR and cache-affinity scheduling allows the multiprocessor scalability to be extended to 20 processors for a 128-bit shared bus and current values of main-memory-to-processor speed gap.

Towards a theory of cache-efficient algorithms

We present a model that enables us to analyze the running time of an algorithm on a computer with a memory hierarchy with limited associativity, in terms of various cache parameters. Our cache model, an extension of Aggarwal and Vitter's I/O model, enables us to establish useful relationships between the cache complexity and the I/O complexity of computations. As a corollary, we obtain cache-efficient algorithms in the single-level cache model for fundamental problems like sorting, FFT, and an important subclass of permutations. We also analyze the average-case cache behavior of mergesort, show that ignoring associativity concerns could lead to inferior performance, and present supporting experimental evidence.We further extend our model to multiple levels of cache with limited associativity and present optimal algorithms for matrix transpose and sorting. Our techniques may be used for systematic exploitation of the memory hierarchy starting from the algorithm design stage, and for dealing with the hitherto unresolved problem of limited associativity.

Teaching computer architecture with a new superscalar processor emulator

Current computers use several techniques to improve performance such as cache memories, pipeline and multiple instruction issue per cycle. Using a real computer to teach these concepts is actually impractical, because these computers are designed to be programmed in high-level languages.In order to solve this problem, we have implemented a superscalar processor emulator, where most of the processor and cache parameters can be defined by the student. Its objective is to create a set of laboratory works allowing the student to observe how the different components of the computer evolve while executing an assembler program. It allows detection of the different kinds of cache misses and hazards as well as their impact on performance. Then, the student can apply some software techniques to reduce cache misses and to avoid hazards.

Experimental evaluation of the Hewlett-Parkard exemplar file system

This article presents results from an experimental evaluation study of the HP Exemplar file system. Our experiments consist of simple micro-benchmarks that study the impact of various factors on the file system performance. These factors include I/O request/buffer sizes, vectored/non-vectored access patterns, read-ahead policies, multi-threaded (temporally irregular) requests, and architectural issues (cache parameters, NUMA behavior, etc.). Experimental results indicate that the Exemplar file system provides high I/O bandwidth, both for single- and multi-threaded applications. The buffer cache, with prioritized buffer management and large buffer sizes, is effective in exploiting temporal and spatial access localities. The performance of non-contiguous accesses can be improved by either using vectored I/O interfaces or tuning the read-ahead facilities. The file system performance depends on the relative locations of the computing threads and the file system, and also on various Exemplar design parameters such as the NUMA architecture, TLB/data cache management and paging policies.

Compile-Time Cache Performance Prediction and Its Application to Tiling

Tiling has been used by parallelizing compilers to define fine-grain parallel tasks and to optimize cache performance. In this paper we present a novel compile-time technique, called miss-driven cache simulation, for determining loop tile sizes that achieve the highest cache hit-rate. The widening disparity between a processor's peak instruction rate and main memory access time in modern computer systems makes this kind of optimization increasingly important for overall program efficiency. Our simulation technique generates only those references of a loop nest that may generate a cache memory miss and processes them on an architecturally accurate cache model at compile-time. Processing only a small portion of the memory reference trace of a program yields simulation speeds in the millions of memory references per second on workstations, with statistics of misses per reference and inter-reference interference counts gathered at the same time. These simulation speeds and statistics allow for the accurate analysis of the impact of cache optimizations at compile-time. We discuss the results of applying this method to guide loop tiling for such commonly used computational kernels as matrix multiplication and Jacobi iteration for various cache parameters.

A compiler optimization to reduce execution time of loop nest

In this paper, a compiler optimization to reduce the execution time of loop nest is proposed. Loop tiling is used to optimize loop nest. Loop tiling is the well-known optimization for improving locality. However, it has a count result that increases the number of instructions to loop control. These increased instructions disturb the effect of locality optimization. Therefore, the tiling of innermost loop is not perform in order to reduce the instructions for loop control in this paper. This optimization consists of two steps. In the first step, loop permutation is used to maximize the spatial reuse of innermost loop. Because innermost loop tiling is not performed, spatial reuse of innermost loop is exploited to reduce execution time of loop nest. In the second step, the tile size of outer loops is decided using the basic cache parameters. When this optimization is applied to the matrix multiplication, execution time is improved 3 to 4 fold. Also, this optimization technique is easily applied to make a parallelizing and/or optimizing compilers.

On the effective use of a cache memory simulator in a computer architecture course

In this paper, we describe how a cache memory simulator can be effectively used to enhance the understanding of cache memories in a computer architecture course. Various types of course assignments that are described in this paper require the use of the simulator to understand effects of varying different cache parameters on the system performance, to analyze cache memory designs, to synthesize cache designs, and to incorporate advanced caching concepts like victim caching, stall caching, and multilevel caching by modifying the simulator. Dinero, a cache simulator available in the public domain, has been enhanced and used in the course. Our experience of using the simulator is described. >

Unfavorable Strides in Cache Memory Systems (RNR Technical Report RNR‐92‐015)

An important issue in obtaining high performance on a scientific application running on a cache‐based computer system is the behavior of the cache when data are accessed at a constant stride. Others who have discussed this issue have noted an odd phenomenon in such situations: A few particular innocent‐looking strides result in sharply reduced cache efficiency. In this article, this problem is analyzed, and a simple formula is presented that accurately gives the cache efficiency for various cache parameters and data strides.

A SIMULATION STUDY ON THE INTERACTIONS BETWEEN MULTITHREADED ARCHITECTURES AND THE CACHE

As processors’ speeds increase, the problems of the memory and communication bottlenecks have resurfaced. Recently, there have been growing interest in the design of multithreaded architecture which can potentially overcome these problems. In such architectures, hardware is used to maintain a fixed number of instruction streams out of which instructions may be issued. The motivation of this study is to examine the effects of such architectures on the cache, which so far has been the mainstay against the memory bottleneck problem. Unlike previous studies of caches in multiprogramming systems, multithreaded architectures switch contexts much more rapidly. We will present and analyze results on the effects of the number of streams and the stream switch interval on the cache. Our results show significant differences from those previously obtained from multiprogramming experiments. We have also studied the interaction of the various cache parameters (cache size and associativity, in particular) with these two new parameters. Our findings confirm the intuition that multithreading will lead to a higher cache miss ratio. However, our findings give a more detailed picture of the interactions involved.

Expected I-cache miss rates via the gap model

To evaluate the performance of a memory system, computer architects must determine the miss rate m of a cache C when running program P. Typically, the measured miss rate depends on the specific address mapping M of P set arbitrarily by the compiler and linker. In this paper, we remove the effect of the address-mapping on the miss rate by analyzing a symbolic trace T of basic blocks. By assuming each basic block has an equal probability of ending up anywhere in the address map, we determine the expected miss rate averaged over all possible address mappings.Our resulting gap model gives the expected miss rate for instruction caches of varying cache size, line size, and set associativity. Our model is simple but robust, and turns out to be the familiar LRU stack model with a statistical viewpoint. Our model allows a trace of arbitrary length to be compactly summarized in a few thousand bytes of information. Our model also predicts how an intervening trace, such as an operating system call or a task switch, will affect the miss rate. Comparisons to measured miss rates from SPEC 92 instruction traces show that our model typically has relative differences of less than 20% for a variety of cache parameters.

Performance and reliability improvement by using asynchronous algorithms in disk buffer cache memory

This paper proposes performance and reliability improvement by using new algorithms for asynchronous operations in disk buffer cache memory. These algorithms allow for writing the files into the buffer cache by the processes and consider the number of active processes in the system and the length of the queue to the disk buffer cache. Writing the contents of the buffer cache to the disk depends on the system load and the write activity. Performance and reliability measures including the elapsed time of writing a file into the buffer cache, the waiting time to start writing a file, and the mean number of blocks written to the disk between system failures are used to show performance and reliability improvement by using the algorithms. Sensitivity analysis is used to influence the algorithms' design. Examples of real systems are used to show the numerical results of performance and reliability improvement in different systems with various disk cache parameters and file sizes.

An analytical access time model for on-chip cache memories

An analytical access time model for on-chip cache memories that shows the dependence of the cache access time on the cache parameters is described. The model includes general cache parameters, such as cache size (C), block size (B), and associativity (A), and array configuration parameters that are responsible for determining the subarray aspect ratio and the number of subarrays. With this model, a large cache design space can be covered, which cannot be done using only SPICE circuit simulation within a limited time. Using the model, it is shown that for given C, B, and A, optimum array configuration parameters can be used to minimize the access time; if the optimum array parameters are used, then the optimum access time is roughly proportional to the log (cache size), and when the optimum array parameters are used, larger block size gives smaller access time, but larger associativity does not give smaller access time because of the increase of the data-bus capacitances. >

An analytical cache model

Trace-driven simulation and hardware measurement are the techniques most often used to obtain accurate performance figures for caches. The former requires a large amount of simulation time to evaluate each cache configuration while the latter is restricted to measurements of existing caches. An analytical cache model that uses parameters extracted from address traces of programs can efficiently provide estimates of cache performance and show the effects of varying cache parameters. By representing the factors that affect cache performance, we develop an analytical model that gives miss rates for a given trace as a function of cache size, degree of associativity, block size, subblock size, multiprogramming level, task switch interval, and observation interval. The predicted values closely approximate the results of trace-driven simulations, while requiring only a small fraction of the computation cost.

Optimizing cache parameters using CAE workstations

The paper deals with the problem of designing cache subsystems for high performance microcomputers. First, three existing approaches and their disadvantages for optimizing cache parameters are discussed. A new strategy is presented, which is a compromise between the state of the art methods and eliminates their disadvantages. This new cache optimization strategy is based upon the tools available with CAE workstations and software development systems. The design concepts and a prototype implementation of this new Cache Optimization Tool Environment (COTE) are also described in the paper.

Cache performance of vector processors

An instruction-level simulator for IBM 3090 with VF (vector facility) has been developed for studying the performance of vector processors and their memory hierarchies. Initial use of the simulator is to understand the program locality of real vectorized applications. Observation on several large scientific applications indicates that the program locality of vector execution can be significantly different from that of the scalar execution of the same application. Although these large applications generally do not exhibit a locality as strong as that of the conventional mainframe applications, their cache hit ratios are high enough to take advantage of a cache. The cache performance of these applications with respect to various cache parameters is also presented.

Program locality of vectorized applications running on the IBM 3090 with Vector Facility

An instruction-level simulator is used to study the program locality of large scientific applications. The simulator, which models an IBM 3090 processor with Vector Facility and a cache, was developed to help a programmer improve the performance of an application through better understanding and use of the Vector Facility and the memory hierarchy of the IBM 3090 system. Our main observations on a set of scientific applications are as follows: (1) although the applications have different characteristics of memory accesses and vectorization , their program locality is high enough to take advantage of conventional cache structures; (2) the cache hit ratio of the vector execution can be quite different from (but not significantly lower than) that of the scalar execution of the same application; and (3) the application programs that are written to optimize the use of the memory hierarchy in the system generally result in higher cache hit ratios than the others. The cache performance of these applications with respect to various cache parameters is also presented. In particular, our study finds that the cache structure of the IBM 3090 is well suited for large scientific applications.

Cache memory design: An evolving art: Designers are looking to line size, degree of associativity, and virtual addresses as important parameters in speeding up the operation

The role of cache memories and the factors that decide the success of a particular design are examined. The operation of a cache memory is described. The specification of cache parameters is considered. Also discussed are the size of a cache, cache hierarchies, fetching and replacing, cache organization, updating the main memory, the use of two caches rather than one, virtual-address caches, and cache consistency.