Performance Analysis Methods for List-Based Caches With Non-Uniform Access

Giuliano Casale,Nicolas Gast

doi:10.1109/tnet.2020.3042869

Abstract

List-based caches can offer lower miss rates than single-list caches, but their analysis is challenging due to state space explosion. In this setting, we propose novel methods to analyze performance for a general class of list-based caches with tree structure, non-uniform access to items and lists, and random or first-in first-out replacement policies. Even though the underlying Markov process is shown to admit a product-form solution, this is difficult to exploit for large caches. Thus, we develop novel approximations for cache performance metrics, in particular by means of a singular perturbation method and a refined mean field approximation. We compare the accuracy of these approaches to simulations, finding that our new methods rapidly converge to the equilibrium distribution as the number of items and the cache capacity grow in a fixed ratio. We find that they are much more accurate than fixed point methods similar to prior work, with mean average errors typically below 1.5% even for very small caches. Our models are also generalized to account for synchronous requests, fetch latency, and item sizes, extending the applicability of approximations for list-based caches.

Highlights

Replacement policies provide a strategy to select items to replace in a cache and are an important performance factor influencing the design of web systems [1], content delivery networks [2], edge networks [3], and peer-to-peer traffic [4], among others
We here focus in particular on a general class of list-based caches that admit a tree structure for the lists, rather than linear as in earlier work on these models [15], and where access to the cache is non-uniform, in the sense that items are promoted within the tree based on probabilities that depend on the item, its current list, and the stream requesting it
Since transient analysis is an important concern in caching systems, we propose a highlyaccurate refined mean field (RMF) approximation for random replacement (RR)-C(m) grounded in the general framework developed in [21]

Summary

Introduction

Replacement policies provide a strategy to select items to replace in a cache and are an important performance factor influencing the design of web systems [1], content delivery networks [2], edge networks [3], and peer-to-peer traffic [4], among others. The focus of this paper is on the theoretical analysis of randomized policies operating within a single cache, a problem that has attracted much attention over the years [5]– [13]. We consider list-based caches, which can deliver lower miss rates than single-list caches [14], [15], but due to statespace explosion issues are still not fully understood from a theoretical standpoint. We here focus in particular on a general class of list-based caches that admit a tree structure for the lists, rather than linear as in earlier work on these models [15], and where access to the cache is non-uniform, in the sense that items are promoted within the tree based on probabilities that depend on the item, its current list, and the stream requesting it. Items of one stream may be stored in a different list than items of another stream to avoid cache pollution and optimize hit ratios

Objectives

Results

Conclusion