An Analytic Cache Performance Model

Abstract

An analytical expression is derived that predicts the miss ratios achievable with set-associative caches. It is assumed that the sequence of memory addresses generated by a computer program can be characterized by a random walk on a one-dimensional lattice with the probability that an address differs from its predecessor by an amount k defined by the distribution P(k ≥ n) = P(k ≤ −n) = l/2n α for all integers n > 0. The resulting miss ratio predictions are simple functions of the exponent a, the cache size, the set size, the number of bytes per cache line and the number of references made since the program began execution. It is shown that, to the extent that actual program behavior is represented by a random walk of this sort, cache efficiency increases at most in inverse proportion to the square root of its size. The predictions resulting from this model are compared with published miss ratios, estimated by analyzing address traces generated by various executing programs, and found to be in excellent agreement with them.