7 - Large and Fast: Exploiting Memory Hierarchy

Abstract

This chapter discusses memory hierarchy. Programs exhibit both temporal locality, that is, the tendency to re-use recently accessed data items, and spatial locality, that is, the tendency to reference data items that are close to other recently accessed items. Memory hierarchies take advantage of temporal locality by keeping more recently accessed data items closer to the processor. Memory hierarchies take advantage of spatial locality by moving blocks consisting of multiple contiguous words in memory to upper levels of the hierarchy. A memory hierarchy uses smaller and faster memory technologies close to the processor. Thus, accesses that hit in the highest level of the hierarchy can be processed quickly. Accesses that miss go to lower levels of the hierarchy, which are larger but slower. If the hit rate is high enough, the memory hierarchy has an access time close to that of the highest and fastest level and a size equal to that of the lowest and largest level. The chapter presents a simple analogy that illustrates the key principles and mechanisms that are used.