The Cell Probe Complexity of Succinct Data Structures

Abstract

In the cell probe model with word size 1 (the bit probe model), a static data structure problem is given by a map f : {0,1}^n * {0,1}^m -> {0,1}, where {0,1}^n is a set of possible data to be stored, {0,1}^m is a set of possible queries (for natural problems, we have m << n) and f(x,y) is the answer to question y about data x.<br /> <br />A solution is given by a representation phi : {0,1}^n -> {0,1}^s and a query algorithm q so that q(phi(x), y) = f(x,y). The time t of the query algorithm is the number of bits it reads in phi(x).<br /> <br />In this paper, we consider the case of <em>succinct</em> representations where s = n + r for some <em>redundancy</em> r << n. For a boolean version of the problem of polynomial evaluation with preprocessing of coefficients, we show a lower bound on the redundancy-query time trade-off of the form <br />(r + 1) t >= Omega(n/log n).<br /> In particular, for very small redundancies r, we get an almost optimal lower bound stating that the query algorithm has to inspect almost the entire data structure (up to a logarithmic factor). We show similar lower bounds for problems satisfying a certain combinatorial property of a coding theoretic flavor. Previously, no omega(m) lower bounds were known on t in the general model for explicit functions, even for very small redundancies.<br /> <br />By restricting our attention to <em>systematic</em> or <em>index</em> structures phi satisfying phi(x) = x · phi*(x) for some map phi* (where · denotes concatenation) we show similar lower bounds on the redundancy-query time trade-off for the natural data structuring problems of Prefix Sum and Substring Search.