Lossless Source Coding Using Nested Error Correcting Codes

Abstract

We propose a tree-structured variable-length random binning scheme for lossless source coding. The existing source coding schemes based on turbo codes, low-density parity check codes, and repeat accumulate codes can be regarded as practical implementations of this random binning scheme. For sufficiently large data blocks, we show that the proposed scheme asymptotically achieves the entropy limit. We also derive the distribution of the compression rate achieved by the tree-structured random binning scheme. Comparing this distribution with the distribution obtained using a library of random binning schemes, we show that a nested code can achieve rates close to a library of codes but with much lower encoding/decoding complexity. With lossless turbo source coding being one of the most powerful source compression techniques, we investigate its performance relative to the proposed tree-structured random binning scheme. Our numerical results show that the compression rate achieved by lossless turbo source coding is far from the tree-structured random binning bound. In that, we suggest improvements to enable short-block-length turbo source codes to achieve compression rates close to the tree-structured random binning bound