Finite size correction for fixed word length Zipf analysis

Abstract

Zipf’s original law deals with the statistics of ranked words in natural languages. It has recently been generalized to “words” defined as n-tuples of symbols derived by translation of real-valued univariate timeseries into a literal sequence. We verify that the rank-frequency plot of these words shows, for fractional Brownian motion, the previously found power laws, but with large finite length corrections. We verify a finite size scaling ansatz for these corrections and, as aresult, demonstrate greatly improved estimates of the (generalized) Zipf exponents. This allows us to find the correct relation between the Zipf exponent and the Hurst exponent characterizing the fractional Brownian motion.