Cross-entropy-based method to analyse iterative decoding

Abstract

Cross-entropy (CE) in information theory is introduced as a method to analyse iterative decoding. The maximum a posteriori probability decoding algorithm is shown to minimise the CE between the a priori and the extrinsic information under given code constraints. The error-correcting ability of the constituent decoders is evaluated in terms of CE. Analysis on turbo decoding is carried out based on theoretical findings from several aspects, including analysis of convergence rate, derivation of the Eb/N0 threshold for convergence, evaluation of error performance in the ‘error-floor’ region and a design example of asymmetric turbo codes. Compared with conventional methods, the new technique provides stricter prediction on the Eb/N0 threshold for convergence and quicker error performance evaluation. An asymmetric turbo code designed with the guidance of our new method exhibits more than 0.1 dB of gain over that guided by classical bounding techniques in both high and low bit error rate regions. Particularly, since no information on the source is required and the density of the a priori/extrinsic information can be arbitrary, the new technique is valuable for both offline design and online evaluation in practical systems.