Improving the Instruction Fetch Throughput with Dynamically Configuring the Fetch Pipeline

Abstract

Instruction cache misses are the critical performance bottleneck in the execution of recent workloads such as Web applications written in JavaScript and server applications. Although various instruction prefetchers have been proposed to reduce the misses, the requirements for both high miss coverage and small hardware cost are not satisfied. In this article, we propose a novel method that improves the instruction fetch throughput not by instruction prefetching but by dynamically configuring the fetch pipeline structure. Our scheme switches between the normal pipeline and newly introduced miss-assuming pipeline, which does not degrade the fetch throughput even when L1 instruction cache misses occur. Our method achieves high instruction fetch throughput with simple hardware and small cost unlike previously proposed prefetchers. Our evaluation results using Web and database workloads show that our method improves the performance by 16.6 percent and 8.6 percent on average, compared to that with noprefetching and the state-of-the-art instruction prefetcher, PIF, respectively, and achieves as much as 79.0 percent of the performance of the processor with a perfect instruction cache.