Asymptotically tight bounds on the depth of estimated context trees

Abstract

We study the maximum depth of context tree estimates, i.e., the maximum Markov order attainable by an estimated tree model given an (individual) input sequence of length n. We consider two classes of estimators: 1) Penalized maximum likelihood (PML) estimators where a context tree T is obtained by minimizing a cost of the form - log P T (xn)+f(n)|S|, where P ;T (xn) is the ML probability of the input sequence x n under a tree model T, S T is the set of states defined by T, and f(n) is an increasing (penalization) function of n (the popular BIC estimator corresponds to f(n) = α−1/2 log n where α is the size of the input alphabet). 2) MDL estimators based on the KT probability assignment. In each case we derive an asymptotic upper bound, n1/2+o(1), on the estimated depth, and we exhibit explicit input sequences that asymptotically attain the bound up to the term o(1) in the exponent.