Generalizing the Kraft-McMillan Inequality to Restricted Languages

Abstract

Let /spl lscr/ /sub 1/,/spl lscr/ /sub 2/,...,/spl lscr/ /sub n/ be a (possibly infinite) sequence of nonnegative integers and /spl Sigma/ some D-ary alphabet. The Kraft-inequality states that /spl lscr/ /sub 1/,/spl lscr/ /sub 2/,...,/spl lscr/ /sub n/ are the lengths of the words in some prefix (free) code over /spl Sigma/ if and only if /spl Sigma//sub i=1//sup n/D/sup -/spl lscr/ i//spl les/1. Furthermore, the code is exhaustive if and only if equality holds. The McMillan inequality states that if /spl lscr/ /sub n/ are the lengths of the words in some uniquely decipherable code, then the same condition holds. In this paper we examine how the Kraft-McMillan inequality conditions for the existence of a prefix or uniquely decipherable code change when the code is not only required to be prefix but all of the codewords are restricted to belong to a given specific language L. For example, L might be all words that end in a particular pattern or, if /spl Sigma/ is binary, might be all words in which the number of zeros equals the number of ones.