Trade-off between Hit Rate and Hit Latency for Optimizing DRAM Cache

Abstract

Due to the large storage capacity, high bandwidth and low latency, 3D DRAM is proposed to be the last level cache, referred to as DRAM cache. The hit rate and hit latency are two conflicting optimization goals for DRAM cache. To address this issue, we design a new DRAM organization that trades the lower hit rate for shorter hit latency by way-locator cache and novel cache set layout. We have designed a novel DRAM cache organization to simultaneously achieve a good hit rate and shorter latency, referred to as SODA-cache. The SODA-cache adapts 2-way set associate cache motivated by the observation that 2-way set associative cache provides the most hit rate improvement from the direct-mapped cache to highly associative cache. The proposed way-locator cache and a novel set layout effectively reduce the cache-hit latency. We use SPEC 2006 CPU benchmark to evaluate our design and two stat-of-art DRAM cache designs. Experimental results show that SODA-cache can improve hit rate by 8.1 percent compared with Alloy-cache and reduce average access latency by 23.1, 13.2 and 8.6 percent compared with LH-cache, Alloy-cache and ATCache respectively on average. Accordingly, SODA-cache outperforms over LH-cache, Alloy-cache and ATCache by on average 17, 12.8 and 8.4 percent respectively in term of weighted speedup.