Introduction to data prefetching

Abstract

For many years, computer designers benefitted from Moore's law to significantly improve the speed of processors. Unlike processors, with the main memory, the focus of improvement has been capacity and not access time. Hence, there is a large gap between the speed of processors and main memory. To hide the gap, a hierarchy of caches has been used between a processor and main memory. While caches proved to be quite effective, their effectiveness directly depends on how many times a requested piece of data can be found in the cache. Due to the complexity of data access patterns, on many occasions, the requested piece of data cannot be found in the cache hierarchy, which exposes large delays to processors and significantly degrades their performance. Data prefetching is the science and art of predicting, in advance, what pieces of data a processor needs and bringing them into the cache before the processor requests them. In this chapter, we justify the importance of data prefetching, look at a taxonomy of data prefetching, and briefly discuss some simple ways to do prefetching.