The impact of cache inclusion policies on cache management techniques

Abstract

Caches were designed to mitigate the large disparity between processor and memory speeds. Many last-level cache (LLC) management techniques have been designed to further improve performance. A first observation is that most techniques in the literature have been designed and evaluated on non-inclusive caches. A second observation is that many modern processors implement either inclusive or exclusive policies in their LLCs. Exclusive caches are becoming increasingly popular as the number of cores increases because of their larger effective capacity. It is expected therefore that future multi-core caches will be exclusive. These two observations unveil a mismatch between the assumptions made for evaluation in terms of cache inclusion policy, and the reality in nowadays and potentially future processor implementations. This paper addresses the question of how sensitive existing cache management techniques are to the inclusion policy, quantifies how effective they are for multiple inclusivity options, and proposes a concept LLC design including the features that best suit exclusive caches. The results show that state-of-the-art prefetchers are fundamental when evaluating replacement policies due to their tight interplay, and that inclusive caches require a less aggressive prefetching mechanism to prevent excessive back-invalidation. The results also pinpoint the features required for the replacement policy of an exclusive cache. Due to the lack of recency information and the impracticality of knowing the memory access that allocated the data, LLCs need to receive reuse information from lower cache levels along with the data block. Moreover, the proposed LLC features include keeping global reuse information structures detached from data blocks to prevent losing that information when the data is evicted on hit.