Minimum Average Codeword Length Research Articles

An entropy minimization histogram mergence scheme and its application in image compression

In this paper, we propose an entropy minimization histogram mergence (EMHM) scheme that can significantly reduce the number of grayscales with nonzero pixel populations (GSNPP) without visible loss to image quality. We proved in theory that the entropy of an image is reduced after histogram mergence and that the reduction in entropy is maximized using our EMHM. The reduction in image entropy is good for entropy encoding considering that the minimum average code word length per source symbol is the entropy of the source signal according to Shannon’s first theorem. Extensive experimental results show that our EMHM can significantly reduce the code length of entropy coding, such as Huffman, Shannon, and arithmetic coding, by over 20% while preserving the image subjective and objective quality very well. Moreover, the performance of some classic lossy image compression techniques, such as the Joint Photographic Experts Group (JPEG), JPEG2000, and Better Portable Graphics (BPG), can be improved by preprocessing images using our EMHM.

An Efficient Construction Strategy for Near-Optimal Variable-Length Error-Correcting Codes

In this letter, we present an efficient cross-entropy (CE)-based algorithm for the design of variable-length error-correcting (VLEC) codes under the joint source and channel coding (JSCC) framework. The algorithm enables us to construct the near-optimal VLEC codes that have the minimum average codeword length (ACL) with low search complexity. The efficiency of the proposed CE-based algorithm makes it possible to construct the VLEC codes that have small ACL values under various free-distance constraints, especially for large-sized signal alphabets.

An A*-Based Algorithm for Constructing Reversible Variable Length Codes with Minimum Average Codeword Length

Variable length codes (VLCs) are widely adopted in many compression standards due to their good coding efficiency on average codeword length. However, an inherent problem with a VLC is that an error of even one bit can cause serious error propagation and thus loss of synchronization at the receiver, which would lead to a series of non-correctly decoded symbols. Reversible variable length codes (RVLCs) were introduced to significantly mitigate this phenomenon. In this work, a method to find an optimal RVLC in terms of the minimum average codeword length is first formulated as a tree-searching problem, and then, instead of performing an exhaustive search, an A*-based construction algorithm is proposed to find an optimal RVLC. The proposed algorithm has been applied to several benchmarks for sources and has found respective optimal symmetric and asymmetric RVLCs.

A note on the construction of error detecting/correcting prefix codes

A k-bit Hamming prefix code is a binary code with the following property: for any codeword x and any prefix y of another codeword, both x and y having the same length, the Hamming distance between x and y is at least k. Given an alphabet A = [ a 1 , … , a n ] with corresponding probabilities [ p 1 , … , p n ] , the k-bit Hamming prefix code problem is to find a k-bit Hamming prefix code for A with minimum average codeword length ∑ i = 1 n p i ℓ i , where ℓ i is the length of the codeword assigned to a i . In this paper, we propose a general approximation algorithm for the k-bit Hamming prefix code problem. Let α k be an O ( r k ( n ) ) -time algorithm for calculating fixed-length codes with Hamming distances k whose codewords are d k ( n ) bits longer than ⌈ log 2 n ⌉ . Our algorithm uses α k to calculate a k-bit Hamming prefix code in O ( r k ( n ) + n log n ) time with an additive error of at most O ( d k ( n ) + log ∗ n ) bits with respect to the optimal prefix code for A, under reasonable assumptions on the function d k .

A structure for fast synchronizing variable-length codes

A new approach for transforming variable-length codes, optimal in the minimum average codeword length sense, into fast synchronizing codes with the same length distribution is proposed in this letter. Based on the adaptation of an intrinsically fast synchronizing structure whose properties are demonstrated, this approach provides good synchronization recovery for a very low computational complexity. Finally, numerical results are presented and compared with reference ones.

Codes with monotonic codeword lengths

Minimum average codeword length coding under the constraint that the codewords are monotonically nondecreasing in length is studied. Bounds on the average length of an optimal “monotonie” code are derived, and sufficient conditions are given such that algorithms for optimal alphabetic codes can be used to find the optimal “monotonic” code.

Huffman-type codes for infinite source distributions

A new sufficient condition is given for an infinite source distribution to share a minimum average codeword length code with the geometric distribution. Thus, some new examples of parametric families of infinite source distributions can be optimally encoded by Huffman-type codes.

Optimum '1'-ended binary prefix codes

The problem of finding a binary prefix code of minimum average codeword length for a given finite probability distribution subject to the requirement that each codeword must end with a 1 is considered. Lower and upper bounds to the performance of the optimum code are derived; the lower bound is tight for certain probability distributions. An algorithm that generates an optimum code for any given distribution is described. >

Synchronization of binary source codes (Corresp.)

The problem of achieving synchronization for variable-length source codes is addressed through the use of self-synchronizing binary prefix-condition codes. Although our codes are suboptimal in the sense of minimum average codeword length, they have the advantages of being generated by an explicit constructive algorithm, having minimal additional redundancy compared with optimal codes-as little as one additional bit introduced into the least likely codeword for a large class of sources-and having statistical synchronizing performance that improves on that of the optimal code in many cases.

Efficient error-limiting variable-length codes

Variable-length codes recursively defined by certain sequential machines are investigated. It is seen that the recursive definition may be used to control error propagation as well as to provide a conceptually simple decoding procedure. Furthermore, the variety of these codes is such that the theoretical minimum average code-word length can be approached quite closely for many distributions. Methods for obtaining efficient codes are discussed, and examples are given.