Chapter 5 - Multi-Level Cache Hierarchies

Abstract

This chapter describes multi-level cache hierarchies. It presents the differences between cache design for a single level of caching and for a part of a larger hierarchy. Experiments based on a two-level simulation model and a multi-level analytical model indicated how the choice of optimal caches varies between the two environments. In particular, the speed–size tradeoff changes significantly when there is an upstream cache. All other parameters being equal, the presence of the 4KB first-level cache shifted the lines of constant performance to the right by about seven binary orders of magnitude. Changes in the L1 size have only a limited effect on the optimal cache size: four doublings in the L1 size are needed to shift the curves by one factor of two towards larger L2 sizes. In general, though as the miss rate of the upstream cache decreases, the amount of performance improvement that is achieved through the addition of more layers of caching decreases dramatically. The presence of the upstream caches significantly improves the viability of set associativity to the extent that large set associativity caches made of MSI TTL and static RAMs may be an improvement over their direct-mapped equivalents. Except for the peculiarities of the definition of latency and transfer rate between two cache layers, the choice of block size remains unaffected by the upstream hierarchy.