Discrete Noiseless Channels Research Articles

A New Generalized Entropy Measure and its Properties

 In this research article, a new two parametric measure of Entropy  and its corresponding code word length  has been developed. The developed measures are the generalizations to some well known existing measures. Besides, some noiseless coding theorems for discrete noiseless channel have been developed, and the results thus obtained have been verified with the support of an numerical example. Also, at the end of this research article, a comparative study in terms of  monotonic behavior  among the proposed entropy  , Matahi’s entropy  and Tsallis entropy  together  with their respective average code word length measures have been made and graphically displayed.

On Optimal Coding of Non-Linear Dynamical Systems

We consider the problem of zero-delay coding of a dynamical system over a discrete noiseless channel under three estimation criteria concerned with the low-distortion regime. For these three criteria, formulated stochastically in terms of a probability distribution for the initial state, we characterize the smallest channel capacities above which the estimation objectives can be achieved. The results establish further connections between topological and metric entropies of dynamical systems and information theory. AMS Classification: 93E10, 37A35, 93E99, 94A17, 94A29

Stochastic Stabilization of Partially Observed and Multi-Sensor Systems Driven by Unbounded Noise Under Fixed-Rate Information Constraints

We investigate the stabilization of unstable multi-dimensional partially observed single-sensor and multi-sensor linear systems driven by unbounded noise and controlled over discrete noiseless channels. Stability is achieved under fixed-rate communication requirements that are asymptotically tight in the limit of large sampling periods. Through the use of similarity transforms, sampling and random-time drift conditions we obtain a coding and control policy leading to the existence of a unique invariant distribution and finite second moment for the sampled state. We use a vector stabilization scheme in which all modes of the linear system visit a compact set together infinitely often. We prove tight necessary and sufficient conditions for the general multi-sensor case under an assumption related to the structure of such systems. In the absence of this assumption, we give sufficient conditions for stabilization.

Some coding theorem on generalized Havrda-Charvat and Tsallis's entropy

A new measure $L_{\alpha }^{\beta } $, called average code word length of order $\alpha $ and type $\beta $ has been defined and its relationship with a result of generalized Havrda-Charvat and Tsallis's entropy has been discussed. Using $L_{\alpha }^{\beta } $,some coding theorem for discrete noiseless channel has been proved.

Some coding theorem on generalized Havrda-Charvat and Tsallis's entropy

A new measure $L_{\alpha }^{\beta } $, called average code word length of order $\alpha $ and type $\beta $ has been defined and its relationship with a result of generalized Havrda-Charvat and Tsallis's entropy has been discussed. Using $L_{\alpha }^{\beta } $,some coding theorem for discrete noiseless channel has been proved.

Some Coding Theorem Connected on Generalized Renyi's Entropy for Incomplete Power Probability Distribution \(p^{\beta}\)

// A new measure \(L_{\alpha }^{\beta } \), called average codeword length of order \(\alpha\) for incomplete power probability distribution \(p^{\beta}\) has been defined and its relationship with a result of generalized Renyi's entropy has been discussed. Using \(L_{\alpha }^{\beta }\), some coding theorem for discrete noiseless channel has been proved.

Choosing the parameters of multiple‐repetition strategies

AbstractThe class of multiple‐repetition strategies is known to achieve capacity for specific discrete memoryless channels with noiseless feedback. We show, given an arbitrary discrete memoryless channel with noiseless feedback, that the repetition parameters should be chosen close to the self information of a transmission error to maximise the transmission rate. Furthermore, it is indicated how close the channel capacity can be approached in the general case. Finally, we compare some results on a related problem: the capacity of a discrete noiseless channel with constraints on the input symbols. Copyright © 2006 AEIT

A combinatorial approach to finding the capacity of the discrete noiseless channel

Shannon defined the capacity of the discrete noiseless channel (DNC) and considered a finite-state model from which the capacity can be calculated. Alternatively, the DNC may be (and often is) represented by a finite list of forbidden strings. In this correspondence, we demonstrate the application of combinatorial techniques to finding the Shannon capacity of the DNC directly from the forbidden list. In our derivations, the case of noninteger symbol durations, as introduced by Csiszar, is considered.

Enumerative Combinatorics and Shannon's Theory of Discrete Noiseless Channels

In this paper we demonstrate the application of combinatorial techniques as a way to finding the Shannon capacity of the discrete noiseless channel.

On the capacity and error-correcting codes of write-efficient memories

Write-efficient memories (WEMs) were introduced by Ahlswede and Zhang (1989) as a model for storing and updating information on a rewritable medium with cost constraints. We note that the research work of Justesen and Hoholdt (1984) on maxentropic Markov chains actually provide a method for calculating the capacity of WEM. By using this method, we derive a formula for the capacity of WEM with a double-permutation cost matrix. Furthermore, some capacity theorems are established for a special class of WEM called deterministic WEM. We show that the capacity of deterministic WEM is equal to the logarithm of the largest eigenvalue of the corresponding connectivity matrix, it is interesting to note that the deterministic WEM behaves like the discrete noiseless channels of Shannon (1948). By specializing our results, we also obtain some interesting properties for the maximization problem of information functions with multiple variables which are difficult to obtain otherwise. Finally, we present a method for constructing error-correcting codes for WEM with the Hamming distance as the cost function. The covering radius of linear codes plays an important role in the constructions.

Construction of Bounded Delay Codes for Discrete Noiseless Channels

Algorithms are described for constructing synchronous (fixed rate) codes for discrete noiseless channels where the constraints can be modeled by finite state machines. The methods yield two classes of codes with minimum delay or look-ahead.

A General Method for Channel Coding

A procedure is described for constructing minimum delay codes for discrete noiseless channels. The method is based on a simple recursive algorithm for finding a set of coding paths.

Some theorems on noiseless coding

For a generalized random variable X , a measure L ¯ X of average code length is defined. Using L ¯ X , some coding theorems for discrete noiseless channels are proved. Some new concepts like strongly decipherable codes, absolutely strongly optimal codes, etc. are also proposed. It is asserted that if the celebrated noiseless coding theorem of Shannon (for complete random variables) is to be extended properly for generalized random variables, then the coding must be done in a strongly decipherable way.

On a coding theorem connected with Rényi's entropy

A new measure L(α), called average code length of order α, has been defined and its relationship with Rényi's entropy has been discussed. Using L(α), a coding theorem for discrete noiseless channels has been proved.

Optimal variable length codes (arbitrary symbol cost and equal code word probability)

The problem of constructing minimum-redundancy prefix codes for the general discrete noiseless channel without constraints is solved for unequal code letter costs, provided that the symbols encoded are assumed to be equally probable. A graphical technique is developed for solving the problem for which the code words are equally probable and are constructed from r symbols where r is greater than or equal to two. A method is given for constructing an optimal exhaustive prefix code. This method is then generalized to the extent that the exhaustive constraint is deleted, thereby resulting in an algorithm, designated ACE for arbitrary symbol cost and equal code word probability, which solves the stated problem.

Block codes for a class of constrained noiseless channels

A class of discrete noiseless channels having upper and lower bounds on the separation between adjacent nonzero input symbols is considered. Recursion relations are derived for determining the number of input sequences which satisfy the constraints for all block lengths, and the asymptotic information rate is calculated. Applications to compaction and synchronization are discussed. An optimal algebraic block coding scheme for such channels is developed.

On synchronous variable length coding for discrete noiseless chrannels

A method is presented for finding the shortest variable length codes with a given bit per symbol ratio for discrete noiseless channels. The central feature of the procedure is a dynamic programming algorithm for determining the optimal code paths. Bounds are found for the channel capacity.

Necessary and sufficient conditions on conditional probabilities to maximize entropy

Shannon (1948) showed that, by a proper choice of the conditional probabilities of the symbols in a discrete noiseless channel of the finite-state type (which possesses certain inherent constraints), the entropy of symbols on such a channel could be maximized. To date these known sufficient conditions on the conditional probabilities have been assumed to also be the necessary conditions. In this brief paper it is shown that the sufficient conditions as originally stated by Shannon are indeed also the necessary conditions for the range of symbol lengths which are of interest.

A note on the capacity of discrete noiseless channels

Diese Mitteilung gibt einen mathematischen Beweis einer Darstellung vonShannon, betreffend die Kanalkapazitat einer Klasse von einfachen Kanalen ohne Gedachtnis.

Minimum-redundancy coding for the discrete noiseless channel

This paper gives a method for constructing minimum-redundancy prefix codes for the general discrete noiseless channel without constraints. The costs of code letters need not be equal, and the symbols encoded are not assumed to be equally probable. A solution had previously been given by Huffman [10] in 1952 for the special case in which all code letters are of equal cost. The present development is algebraic. First, structure functions are defined, in terms of which necessary and sufficient conditions for the existence of prefix codes may be stated. From these conditions, linear inequalities are derived which may be used to characterize prefix codes. Gomory's integer programming algorithm is then used to construct optimum codes subject to these inequalities; computational experience is presented to demonstrate the practicability of the method. Finally, some additional coding problems are discussed and a problem of classification is treated.