A Coding Theorem for a Class of Stationary Channels With Feedback

Abstract

A coding theorem is proved for a class of stationary channels with feedback in which the output Yn = f(Xn-m n, Zn-m n) is the function of the current and past m symbols from the channel input Xn and the stationary ergodic channel noise Zn. In particular, it is shown that the feedback capacity is equal to limnrarr infin supp(x n ||y n-1 ) 1/n I(Xn rarr Yn) where I(Xn rarr Yn) = Sigmai=1 n I(Xi; Yi|Yi-1) denotes the Massey directed information from the channel input to the output, and the supremum is taken over all causally conditioned distributions p(xn||yn-1) = Pii=1 n p(xi|xi-1,yi-1). The main ideas of the proof are a classical application of the Shannon strategy for coding with side information and a new elementary coding technique for the given channel model without feedback, which is in a sense dual to Gallager's lossy coding of stationary ergodic sources. A similar approach gives a simple alternative proof of coding theorems for finite state channels by Yang-Kavcic-Tatikonda, Chen-Berger, and Permuter-Weissman-Goldsmith.