Memoryless Communication Channel Research Articles

Distributed Channel Access for Control Over Unknown Memoryless Communication Channels

We consider the distributed channel access problem for a system consisting of multiple control subsystems that close their loop over a shared wireless network. We propose a distributed method for providing deterministic channel access without requiring explicit information exchange between the subsystems. This is achieved by utilizing timers for prioritizing channel access with respect to a local cost which we derive by transforming the control objective cost to a form that allows its local computation. This property is then exploited for developing our distributed deterministic channel access scheme. A framework to verify the stability of the system under the resulting scheme is then proposed. Next, we consider a practical scenario in which the channel statistics are unknown. We propose learning algorithms for learning the parameters of imperfect communication links for estimating the channel quality and, hence, define the local cost as a function of this estimation and control performance. We establish that our learning approach results in collision-free channel access. The behavior of the overall system is exemplified via a proof-of-concept illustrative example, and the efficacy of this mechanism is evaluated for large-scale networks via simulations.

Joint source-channel coding via model predictive control

We propose a joint source-channel coding (JSCC) scheme based on model-predictive control (MPC) for encoding an analog discrete-time source over a memoryless communication channel. The decoder is assumed to be fixed and given, and the MPC encoder is set to minimize the frequency-weighted mean-squared error (FWMSE) while satisfying a channel input constraint. We derive analytic expressions for the associated finite-horizon optimization MPC problem for an asymptotic (in time) average or sum channel input constraint. For the case of complex additive white Gaussian noise channel and an M-quadrature-amplitude modulation (M-QAM) decoder, these expressions can be written explicitly, which allows for an efficient numerical solution. For the case M=4 and considering a frequency weighting criterion that models the sensitivity of the human hearing, we show by simulations that our proposed scheme yields an FWMSE significantly smaller than that yielded by the corresponding non-JSCC scheme (a 2-bit uniform quantizer in tandem with a 4-QAM channel encoder and decoder).

A Path-Based Partial Information Decomposition.

Based on the conceptual basis of information theory, we propose a novel mutual information measure—‘path-based mutual information’. This information measure results from the representation of a set of random variables as a probabilistic graphical model. The edges in this graph are modeled as discrete memoryless communication channels, that is, the underlying data is ergodic, stationary, and the Markov condition is assumed to be applicable. The associated multilinear stochastic maps, tensors, transform source probability mass functions into destination probability mass functions. This allows for an exact expression of the resulting tensor of a cascade of discrete memoryless communication channels in terms of the tensors of the constituting communication channels in the paths. The resulting path-based information measure gives rise to intuitive, non-negative, and additive path-based information components—redundant, unique, and synergistic information—as proposed by Williams and Beer. The path-based redundancy satisfies the axioms postulated by Williams and Beer, the identity axiom postulated by Harder, and the left monotonicity axiom postulated Bertschinger. The ordering relations between redundancies of different joint collections of sources, as captured in the redundancy lattices of Williams and Beer, follow from the data processing inequality. Although negative information components can arise, we speculate that these either result from unobserved variables, or from adding additional sources that are statistically independent from all other sources to a system containing only non-negative information components. This path-based approach illustrates that information theory provides the concepts and measures for a partial information decomposition.

Stabilization of Nonlinear Dynamic Systems Over Limited Capacity Communication Channels

This article addresses the stabilization of noiseless nonlinear dynamic systems over limited capacity communication channels. It is shown that the stability of nonlinear dynamic systems over memory-less communication channels implies an inequality condition between the Shannon channel capacity and the summation of the positive equilibrium Lyapunov exponents of the dynamic system or, equivalently, the logarithms of the magnitude of the unstable eigenvalues of system Jacobian. Furthermore, we propose an encoder, decoder, and a controller to prove that scalar nonlinear dynamic systems are stabilizable under the aforementioned inequality condition over the digital noiseless and the packet erasure channels, respectively, in sure and almost sure senses. The performance of the proposed coding scheme is illustrated by computer simulations.

The Log-Volume of Optimal Codes for Memoryless Channels, Asymptotically Within a Few Nats

Shannon’s analysis of the fundamental capacity limits for memoryless communication channels has been refined over time. In this paper, the maximum volume $M_ \mathrm {avg}^{*}(n,\epsilon )$ of length- $n$ codes subject to an average decoding error probability $\epsilon $ is shown to satisfy the following tight asymptotic lower and upper bounds as $n \to \infty $ : $\underline {A}_\epsilon + o(1) \le \log M_ \mathrm {avg}^{*}(n,\epsilon ) - [nC - \sqrt {nV_\epsilon } \,Q^{-1}(\epsilon ) + ({1}/{2}) \log n] \le \overline {A}_\epsilon + o(1)$ , where $C$ is the Shannon capacity, $V_\epsilon $ is the $\epsilon $ -channel dispersion, or second-order coding rate, $Q$ is the tail probability of the normal distribution, and the constants $\underline {A}_\epsilon $ and $\overline {A}_\epsilon $ are explicitly identified. This expression holds under mild regularity assumptions on the channel, including nonsingularity. The gap $\overline {A}_\epsilon - \underline {A}_\epsilon $ is one nat for weakly symmetric channels in the Cover–Thomas sense, and typically a few nats for other symmetric channels, for the binary symmetric channel, and for the $Z$ channel. The derivation is based on strong large-deviations analysis and refined central limit asymptotics. A random coding scheme that achieves the lower bound is presented. The codewords are drawn from a capacity-achieving input distribution modified by an $O(1/\sqrt {n})$ correction term.

Weighted Performance Function for (r, s)-Entropy of Discrete Memoryless Communication Channel under Single Constraint

where p (xi / yj) and p (xi, yj) are the conditional and joint probabilities respectively. The average mutual information is given by I (X / Y) = H (X) - H (X / Y) (1.3)

Predictions of energy efficient Berger-Levy model neurons with constraints

Information theory has been extensively applied to neuroscience problems. The mutual information between input and output has been postulated as an objective, which neuronal systems may optimize. However, only recently the energy efficiency has been addressed within an information-theoretic framework [1]. Here, the key idea is to consider capacity per unit cost (measured in bits per joule, bpj) as the objective. We are interested in how biologically plausible constraints affect predictions made by this new theory for bpj-maximizing model neurons. More specifically, in our contribution, in line with [1] and [2], a neuron is modeled as a memory-less constant communication channel with a Gamma conditional probability distribution function (PDF) [1]. In this setting, the channel input and output are the excitatory postsynaptic potential intensity, λ, and the inter spike interval (ISI), t, with PDFs f (λ) and fT (t), respectively. We then formulate two new constraints: First, we impose a lower bound tmin on the duration tof ISIs. The rational for this is to account for a maximal firing rate. Second, we consider a peak energy expenditure constraint per ISI as compared to only bounding the expected energy expenditure. This translates into an upper bound tmax on the ISI duration. We then derive the fT (t) (corresponding to valid f (λ)) of a bpj-maximizing neuron for the original unconstrained setting from [1] and in the presence of the above two constraints for different expected ISIs. (Details omitted here for brevity.) Figure 1 shows three fT (t)s obtained in the unconstrained (dashed curves) and constrained settings (solid curves) for tmin = 1and tmax = 5. While the constrained and unconstrained solutions have the same mean, the shape of their fT (t) differ. For comparison

A Finite-Sample, Distribution-Free, Probabilistic Lower Bound on Mutual Information

For any memoryless communication channel with a binary-valued input and a one-dimensional real-valued output, we introduce a probabilistic lower bound on the mutual information given empirical observations on the channel. The bound is built on the Dvoretzky-Kiefer-Wolfowitz inequality and is distribution free. A quadratic time algorithm is described for computing the bound and its corresponding class-conditional distribution functions. We compare our approach to existing techniques and show the superiority of our bound to a method inspired by Fano's inequality where the continuous random variable is discretized.

On Feedback and the Classical Capacity of a Noisy Quantum Channel

In Shannon information theory, the capacity of a memoryless communication channel cannot be increased by the use of feedback from receiver to sender. In this correspondence, the use of classical feedback is shown to provide no increase in the unassisted classical capacity of a memoryless quantum channel when feedback is used across nonentangled input states, or when the channel is an entanglement-breaking channel. This gives a generalization of the Shannon theory for certain classes of feedback protocols when transmitting through noisy quantum communication channels.

Quantum Feedback Channels

In Shannon information theory the capacity of a memoryless communication channel cannot be increased by the use of feedback. In quantum information theory the no-cloning theorem means that noiseless copying and feedback of quantum information cannot be achieved. In this paper, quantum feedback is defined as the unlimited use of a noiseless quantum channel from receiver to sender. Given such quantum feedback, it is shown to provide no increase in the entanglement--assisted capacities of a memoryless quantum channel, in direct analogy to the classical case. It is also shown that in various cases of non-assisted capacities, feedback may increase the capacity of memoryless quantum channels.

Neural networks for modeling nonlinear memoryless communication channels

This paper presents a neural network approach for modeling nonlinear memoryless communication channels. In particular, the paper studies the approximation of the nonlinear characteristics of traveling-wave tube (TWT) amplifiers used in satellite communications. The modeling is based upon multilayer neural networks, trained by the odd and even backpropagation (BP) algorithms. Simulation results demonstrate that neural network models fit the experimental data better than classical analytical TWT models,.

An iterative method for computing the performance of discrete memoryless communication channels

In this paper, we develop an iterative method for computing the performance of discrete memoryless communication channels. The method is used to describe an algorithm which is implemented through a computer program, and numerical results are obtained for selected channels.

The Performance Function of a Discrete Memoryless Communication Channel

Abstract Channel-capacity is a theoretical limit which may never be achieved in practice because various extraneous factors, for example, time-, energy-, and money-constraints, govern the use of the communication channel. A more realistic measure of channel-performance is naturally the maximum of the mutual information subject to the normality condition and the other cost constraints that govern its use. This is what we define as the performance function of a discrete memoryless communication channel. We also describe and prove some useful properties of this function.

Regularizing the Augmented Cost Matrix Arising From the Performance Function of a DMC

Abstract In this paper, we develop and illustrate a very interesting process of determining a valid set of, at most m, linearly independent equations from the normality condition and l (>m) cost constraints that govern the use of a disc: ete memoryless communication channel. This process is called regularization and it is a prerequisite for the numerical computation of channel performance.

On channel capacity per unit cost

Memoryless communication channels with arbitrary alphabets where each input symbol is assigned a cost are considered. The maximum number of bits that can be transmitted reliably through the channel per unit cost is studied. It is shown that, if the input alphabet contains a zero-cost symbol, then the capacity per unit cost admits a simple expression as the maximum normalized divergence between two conditional output distributions. The direct part of this coding theorem admits a constructive proof via Stein's lemma on the asymptotic error probability of binary hypothesis tests. Single-user, multiple-access, and interference channels are studied.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A partial ordering of discrete, memoryless channels

This paper is concerned with the structure of a partial ordering of discrete, memoryless communication channels. These are identified with equivalence classes of stochastic matrices into which the set of all stochastic matrices is partitioned by a relation of matrix inclusion. The relation carries over to channels and induces a partial ordering on them, having the property that if <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{1}</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{2}</tex> are channels such that <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{1}</tex> includes <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{2}</tex> , then if a code exists for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{2}</tex> , there exists a code for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{1}</tex> , whose probability of error is never greater than that of the code for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K_{2}</tex> . Results are derived which specify the equivalence classes of stochastic matrices corresponding to the binary and the symmetric channels and resolve the structure of the partial ordering between them.

Asymptotic Estimates of the Probability of Error for Transmission of Messages over a Discrete Memoryless Communication Channel with a Symmetric Transition Probability Matrix

A memoryless channel with a matrix of transaction probabilities $P = \{ P_{ij} \} $ is considered such that any row of the matrix P is a permutation of any other row and any column is a permutation of any other column. It is supposed that $2^{nH} $ possible messages are to be transmitted with the help of words consisting of n symbols. The asymptotic behavior of the error probability for the optimal code is investigated as is the asymptotic behavior of the expectation of the error probability of the randomly chosen code.