MANA: Microarchitecting a Temporal Instruction Prefetcher

Abstract

L1 instruction(L1-l) cache misses are a source of performance bottleneck. While many instruction prefetchers have been proposed, most of them leave a considerable potential uncovered. In 2011, Proactive Instruction Fetch (PIF) showed that a hardware prefetcher could effectively eliminate all instruction-cache misses. However, its enormous storage cost makes it impractical. Consequently, reducing the storage cost was the main research focus in instruction prefetching in the past decade. Several instruction prefetchers, including RDIP and Shotgun, were proposed to offer PIF-level performance with significantly lower storage overhead. However, our findings show that there is a considerable performance gap between these proposals and PIF. While these proposals use different mechanisms for prefetching, the performance gap is mainly not because of the mechanism, and instead, is due to not having sufficient storage. We make the case that the key to designing a powerful and cost-effective instruction prefetcher is choosing a metadata record and microarchitecting the prefetcher to minimize the storage. Our proposal, MANA, offers PIF-level performance with 15.7x lower storage cost. MANA outperforms RDIP and Shotgun by 12.5 and 29%, respectively. We also evaluate a version of MANA with no storage overhead and show that it offers 98% of the peak performance benefits.