Finite-state Channel Research Articles

Capacity of Finite State Channels With Feedback: Algorithmic and Optimization Theoretic Properties

Capacity of Finite State Channels With Feedback: Algorithmic and Optimization Theoretic Properties

Finite-State Channels With Feedback and State Known at the Encoder

Finite-State Channels With Feedback and State Known at the Encoder

Capacity of Finite-State Channels With Delayed Feedback

Capacity of Finite-State Channels With Delayed Feedback

Typical Error Exponents: A Dual Domain Derivation

This paper shows that the probability that the error exponent of a given code randomly generated from a pairwise-independent ensemble is smaller than a lower bound on the typical random-coding exponent tends to zero as the codeword length tends to infinity. This lower bound is known to be tight for i.i.d. ensembles over the binary symmetric channel and for constant-composition codes over memoryless channels. Our results recover both as special cases and remain valid for arbitrary alphabets, arbitrary channels—for example finite-state channels with memory—, and arbitrary pairwise-independent ensembles. We specialize our results to the i.i.d., constant-composition and cost-constrained ensembles over discrete memoryless channels and to ensembles over finite-state channels.

Feedback Capacity of Ising Channels With Large Alphabet via Reinforcement Learning

We propose a new method to compute the feedback capacity of unifilar finite state channels (FSCs) with memory using reinforcement learning (RL). The feedback capacity was previously estimated using its formulation as a Markov decision process (MDP) with dynamic programming (DP) algorithms. However, their computational complexity grows exponentially with the channel alphabet size. Therefore, we use RL, and specifically, its ability to parameterize value functions and policies with neural networks, to evaluate numerically the feedback capacity of channels with a large alphabet size. The outcome of the RL algorithm is a numerical lower bound on the feedback capacity, which is used to reveal the structure of the optimal solution. The structure is modeled by a graph-based auxiliary random variable that is utilized to derive an analytic upper bound on the feedback capacity with the duality bound. The capacity computation is concluded by verifying the tightness of the upper bound by testing whether it is Bahl-Cocke-Jelinek-Raviv (BCJR) invariant. We demonstrate this method on the Ising channel with an arbitrary alphabet size. For an alphabet size smaller than or equal to 8, we derive the analytic solution of the capacity. Next, the structure of the numerical solution is used to deduce a simple coding scheme that achieves the feedback capacity and serves as a lower bound for larger alphabets. For an alphabet size greater than 8, we present an upper bound on the feedback capacity. For an asymptotically large alphabet size, we present an asymptotic optimal coding scheme.

A Deterministic Algorithm for the Capacity of Finite-State Channels

We propose two modified versions of the classical gradient ascent method to compute the capacity of finite-state channels with Markovian inputs. For the case that the channel mutual information rate is strongly concave in a parameter taking values in a compact convex subset of some Euclidean space, our first algorithm proves to achieve polynomial accuracy in polynomial time and, moreover, for some special families of finite-state channels our algorithm can achieve exponential accuracy in polynomial time under some technical conditions. For the case that the channel mutual information rate may not be strongly concave, our second algorithm proves to be at least locally convergent.

Computable Upper Bounds on the Capacity of Finite-State Channels

We consider the use of the well-known dual capacity bounding technique for deriving upper bounds on the capacity of indecomposable finite-state channels (FSCs) with finite input and output alphabets. In this technique, capacity upper bounds are obtained by choosing suitable test distributions on the sequence of channel outputs. We propose test distributions that arise from certain graphical structures called Q-graphs. As we show in this paper, the advantage of this choice of test distribution is that, for the important sub-classes of unifilar and input-driven FSCs, the resulting upper bounds can be formulated as a dynamic programming (DP) problem, which makes the bounds tractable. We illustrate this for several examples of FSCs, where we are able to solve the associated DP problems explicitly to obtain capacity upper bounds that either match or beat the best previously reported bounds. For instance, for the classical trapdoor channel, we improve the best known upper bound of 0.661 (due to Lutz (2014)) to 0.584, shrinking the gap to the best known lower bound of 0.572, all bounds being in units of bits per channel use.

Computable Upper Bounds on the Capacity of Finite-State Channels

Computable Upper Bounds on the Capacity of Finite-State Channels

An Information Theoretical Analysis of Gap Junction Channels

The propagation of molecules, ions and electrochemical signals in various cell types occurs through hundreds of gap junction channels. A gap junction channel is composed of hundreds of units which are further decomposed to hemichannels. From the computational modeling perspective, each gap junction channel acts as a state machine, switching between different permeability states at known rates. Both the permeabilities and the rates jointly depend on transmembrane and transjunctional voltages. To the best of our knowledge, the effects of the permeability states on the communication performance between cells remain unknown to-date. Here, we use the example of gap junctions between cardiac myocytes (cardiomyocytes) to model a gap junction channel as a finite-state channel. We then apply the Shannon’s information theory to compute the achievable information rates. Our results demonstrate the information theoretical limits of communication over cardiac gap junction channels. The proposed analysis is anticipated to address open-research problems in cardiac modeling, as well as in nano-networking where man-made miniature devices inter-connect with cardiomyocytes to enable novel groundbreaking applications in healthcare.

On the Capacity of Channels With Deletions and States

We consider the class of channels formed from the concatenation of a deletion channel and a finite-state channel. For this class of channels, we show that the operationally-defined capacity is equal to the stationary capacity. We also show that the stationary capacity can be approached by a sequence of Markov processes with increasing Markovian orders. As a by-product, we show that the polar coding scheme proposed by Tal, Pfister, Fazeli, and Vardy [arxiv: 1904.13385 (2019)] achieves the capacity of the binary deletion channel.

Graph-Based Encoders and Their Performance for Finite-State Channels With Feedback

The capacity of unifilar finite-state channels in the presence of feedback is investigated. We derive a new evaluation method to extract graph-based encoders with their achievable rates, and to compute upper bounds to examine their performance. The evaluation method is built upon a recent methodology to derive simple bounds on the capacity using auxiliary directed graphs. While it is not clear whether the upper bound is convex, we manage to formulate it as a convex optimization problem using transformation of the argument with proper constraints. The lower bound is formulated as a non-convex optimization problem, yet, any feasible point to the optimization problem induces a graph-based encoder. In all examples, the numerical results show near-tight upper and lower bounds that can be easily converted to analytic results. For the non-symmetric trapdoor channel and binary fading channels (BFCs), new capacity results are established by computing the corresponding bounds. For all other instances, including the Ising channel, the near-tightness of the achievable rates is shown via a comparison with corresponding upper bounds. Finally, we show that any graph-based encoder implies a simple coding scheme that is based on the posterior matching principle and achieves the lower bound.

Shannon meets Turing: Non-computability and non-approximability of the finite state channel capacity

The capacity of finite state channels (FSCs) has been established as the limit of a sequence of multi-letter expressions only and, despite tremendous effort, a corresponding finite-letter characterization remains unknown to date. This paper analyzes the capacity of FSCs from a fundamental, algorithmic point of view by studying whether or not the corresponding achievability and converse bounds on the capacity can be computed algorithmically. For this purpose, the concept of Turing machines is used which provide the fundamental performance limits of digital computers. To this end, computable continuous functions are studied and properties of computable sequences of such functions are identified. It is shown that the capacity of FSCs is not Banach-Mazur computable which is the weakest form of computability. This implies that there is no algorithm (or Turing machine) that can compute the capacity of a given FSC. As a consequence, it is then shown that either the achievability or converse must yield a bound that is not Banach-Mazur computable. This also means that there exist FSCs for which computable lower and upper bounds can never be tight. To this end, it is further shown that the capacity of FSCs is not approximable, which is an even stricter requirement than non-computability. This implies that it is impossible to find a finite-letter entropic characterization of the capacity of general FSCs. All results hold even for finite input and output alphabets and finite state set. Finally, connections to the theory of effective analysis are discussed. Here, results are only allowed to be proved in a constructive way, while existence results, e.g., proved based on the axiom of choice, are forbidden.

Random coding bound for channels with memory decoding function with partial overlapping. Part 2. Examples and Discussion

Introduction:Suboptimal random coding exponent Er*(R; ψ) for a wide class of finite-state channel models using a mismatched decoding function tp was obtained and presented in the first part of this work. We used tp function represented as a product of a posteriori probabilities of non-overlapped input subblocks of length 2B+1 relative to the overlapped output subblocks of length 2W+1. It has been shown that the computation of function Er*(R; ψ) is reduced to the calculation of the largest eigenvalue of a square non-negative matrix of an order depending on the B and W values.Purpose:Toillustrate the approach developed in the first part of this study with its application to various channel modelled as a probabilistic finite-state machine.Results:We consider channels with state transitions not depending on the input symbol (channels with freely evolving states), and channels with deterministic state transitions, in particular, intersymbol interference channels. We present and discuss numerical results of calculating this random coding exponent in a full range of code rates for some of channel models for which similar results were not obtained before. Practical computations were carried out for relatively small values of B and W. Nevertheless, even for small values of these parameters a good correspondence with some known results for optimal decoding was shown.

Universal Decoding Using a Noisy Codebook

We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems. Both the source that generates the original (clean) codewords, and the channel that corrupts them in generating the noisy codewords, as well as the main channel for communicating the messages, are all modeled by non–unifilar, finite–state systems (hidden Markov models). As in previous works on universal decoding, here too, the average error probability of our proposed universal decoder is shown to be as small as that of the optimal maximum likelihood (ML) decoder, up to a multiplicative factor that is a sub–exponential function of the block length. It therefore has the same error exponent, whenever the ML decoder has a positive error exponent. The universal decoding metric is based on Lempel–Ziv incremental parsing of each noisy codeword jointly with the given channel output vector, but this metric is somewhat different from the one proposed in earlier works on universal decoding for finite–state channels, by Ziv (1985) and by Lapidoth and Ziv (1998). The reason for the difference is that here, unlike in those earlier works, the probability distribution that governs the (noisy) codewords is, in general, not uniform across its support. This non–uniformity of the codeword distribution also makes our derivation more challenging. Another reason for the more challenging analysis is the fact that the effective induced channel between the noisy codeword of the transmitted message and the main channel output is not a finite–state channel in general.

Mean square detectability of LTI systems over finite-state digital block-fading channels

This paper studies the mean square quadratic (MSQ) detectability for multi-output networked systems over finite-state digital block-fading channels. The packet-loss rate of each digital fading channel depends on the channel power gain, as well as packet length and power level used for transmission. A finite-state random process is introduced to model time-varying fading channels, which characterizes various configurations of physical communication environment and/or different channel fading amplitudes. Necessary and sufficient conditions for MSQ detectability over finite-state Markov digital block-fading channels are given in the form of algebraic Riccati equations or linear matrix inequalities (LMIs). The estimation gain is given as a function of estimated/observed channel state. In addition, explicit conditions on network for MSQ detectability over finite-state independent identically distributed (i.i.d.) digital block-fading channels are presented in terms of the unstable poles of the multi-output plant. Finally, an application to Gilbert-Elliott channels (GECs) is provided to demonstrate the derived results.

Finite State Markov Wiretap Channel With Delayed Feedback

The finite-state Markov channel (FSMC), where the channel transition probability is controlled by a state undergoing a Markov process, is a useful model for the mobile wireless communication channel. In this paper, we investigate the security issue in the mobile wireless communication systems by considering the FSMC with an eavesdropper, which we call the finite-state Markov wiretap channel (FSM-WC). We assume that the state is perfectly known by the legitimate receiver and the eavesdropper, and through a noiseless feedback channel, the legitimate receiver sends his received channel output and the state back to the transmitter after some time delay. Inner and outer bounds on the capacity-equivocation regions of the FSM-WC with delayed state feedback and with or without delayed channel output feedback are provided in this paper, and we show that these bounds meet if the eavesdropper’s received symbol is a degraded version of the legitimate receiver’s. The above-mentioned results are further explained via a degraded Gaussian fading example.

A Single-Letter Upper Bound on the Feedback Capacity of Unifilar Finite-State Channels

An upper bound on the feedback capacity of unifilar finite-state channels (FSCs) is derived. A new technique, called the $Q$ -context mapping, is based on a construction of a directed graph that is used for a sequential quantization of the receiver’s output sequences to a finite set of contexts . For any choice of $Q$ -graph, the feedback capacity is bounded by a single-letter expression, $C_{\mathrm {fb}}\leq \sup I(X,S;Y|Q)$ , where the supremum is over $p(x|s,q)$ and the distribution of $(S,Q)$ is their stationary distribution. It is shown that the bound is tight for all unifilar FSCs, where feedback capacity is known: channels where the state is a function of the outputs, the trapdoor channel, Ising channels, the no-consecutive-ones input-constrained erasure channel, and the memoryless channel. Its efficiency is also demonstrated by deriving a new capacity result for the dicode erasure channel; the upper bound is obtained directly from the above-mentioned expression and its tightness is concluded with a general sufficient condition on the optimality of the upper bound. This sufficient condition is based on a fixed point principle of the BCJR equation and, indeed, formulated as a simple lower bound on feedback capacity of unifilar FSCs for arbitrary $Q$ -graphs. This upper bound indicates that a single-letter expression might exist for the capacity of finite-state channels with or without feedback based on a construction of auxiliary random variable with specified structure, such as the $Q$ -graph, and not with i.i.d distribution. The upper bound also serves as a non-trivial bound on the capacity of channels without feedback, a problem that is still open.

Capacity Analysis of Discrete Energy Harvesting Channels

We study the channel capacity of a general discrete energy harvesting channel with a finite battery. Contrary to traditional communication systems, the transmitter of such a channel is powered by a device that harvests energy from a random exogenous energy source and has a finite-sized battery. As a consequence, at each transmission opportunity, the system can only transmit a symbol whose energy is no more than the energy currently available. This new type of power supply introduces an unprecedented input constraint for the channel, which is simultaneously random, instantaneous, and influenced by the full history of the inputs and the energy harvesting process. Furthermore, naturally, in such a channel, the energy information is observed causally at the transmitter. Both of these characteristics pose great challenges for the analysis of the channel capacity. In this paper, we use techniques developed for channels with side information and finite-state channels, to obtain lower and upper bounds on the capacity of energy harvesting channels. In particular, in a general case with Markov energy harvesting processes, we use stationarity and ergodicity theory to compute and optimize the achievable rates for the channels, and derive a series of computable capacity upper and lower bounds.

Capacity of Multilevel NAND Flash Memory Channels

In this paper, we initiate a first information-theoretic study on multilevel NAND flash memory channels with intercell interference. More specifically, for a multilevel NAND flash memory channel under mild assumptions, we first prove that such a channel is indecomposable and it features asymptotic equipartition property; we then further prove that stationary processes achieve its information capacity, and consequently, as the order tends to infinity, its Markov capacity converges to its information capacity; eventually, we establish that its operational capacity is equal to its information capacity. Our results suggest that it is highly plausible to apply the ideas and techniques in the computation of the capacity of finite-state channels, which are relatively better explored, to that of the capacity of multilevel NAND flash memory channels.

Universal decoding for source–channel coding with side information

We consider a setting of Slepian-Wolf coding, where the random bin of the source vector undergoes channel coding, and then decoded at the receiver, based on additional side information, correlated to the source. For a given distribution of the randomly selected channel codewords, we propose a universal decoder that depends on the statistics of neither the correlated sources nor the channel, assuming first that they are both memoryless. Exact analysis of the random-binning/random-coding error exponent of this universal decoder shows that it is the same as the one achieved by the optimal maximum a-posteriori (MAP) decoder. Previously known results on universal Slepian-Wolf source decoding, universal channel decoding, and universal source-channel decoding, are all obtained as special cases of this result. Subsequently, we further generalize the results in two directions: (i) finite-state sources and finite-state channels, along with a universal decoding metric that is based on Lempel-Ziv parsing, and (ii) full (symmetric) Slepian-Wolf coding, where both source streams are separately fed into random-binning source encoders, followed by random channel encoders, which are then jointly decoded by a universal decoder.