Overflow Probability of Variable-Length Codes With Codeword Cost

Ryo Nomura

doi:10.1109/tit.2019.2941888

Abstract

Lossless variable-length source coding with codeword cost is considered for general sources. The problem setting, where we impose on unequal costs on code symbols, is called the variable-length coding with codeword cost. In this problem, the infimum of average codeword cost have been determined for general sources. On the other hand, overflow probability, which is defined as the probability of codeword cost being above a threshold, have not been considered yet. In this paper, we determine the infimum of achievable threshold in the first-order sense and the second-order sense for general sources and compute it for some special sources such as i.i.d. sources and mixed sources. A relationship between the overflow probability of variable-length coding and the error probability of fixed-length coding is also revealed. Our analysis is based on the information-spectrum methods.

Highlights

Lossless variable-length source coding with codeword cost is considered for general sources
The overflow probability of codeword length is one of other criteria, which denotes the probability of codeword length per symbol being above a threshold R > 0
In this paper we have considered the overflow probability of codeword cost for general sources and addressed the fundamental limit of the achievable threshold up to the second-order

Summary

Variable-length codes with codeword cost for general source

∞ n=1 of n-dimensional random variables Xn, where each component random variable Xi(n) takes values in a countable set X. It should be noted that each component of Xn may change depending on block length n. This implies that even consistency condition, which means that for any integers m, n (m < n), Xi(m) = Xi(n) holds, may not hold. We consider the situation that there are unequal costs on code symbols. Let us define the cost function over U considered in this paper. We denote cmax = maxu∈U c(u) for short This cost function is called memoryless cost function. We only consider variable-length codes satisfying prefix condition. It should be noted that every variable-length code with prefix condition over unequal costs, satisfies. Throughout this paper, the logarithm is taken to the base K

Overflow Probability of Codeword Cost

First-order and Second-order achievability

General Formula for the Infimum of ε-achievable Overflow Threshold

Strong Converse Property

Computation for Mixed Sources

Concluding Remarks