Coding Theorem Research Articles

A New Channel Coding Technique to Approach the Channel Capacity

After Shannon’s 1948 channel coding theorem, we have witnessed many channel coding techniques developed to achieve the Shannon limit. A wide range of channel codes is available with different complexity levels and error correction performance. Many powerful coding schemes have been deployed in the power-limited Additive White Gaussian Noise (AWGN) channel. However, it seems like we have arrived at an end of advancement path, for most of the existing channel codes. This article introduces a new coding technique that can either be used as the last coding stage of concatenated coding scheme or in parallel configuration with other powerful channel codes to achieve reliable error performance with moderately complex decoding. We will go through an example to understand the overall approach of the proposed coding technique, and finally we will look at some simulation results over an AWGN channel to demonstrate its potential.

Binary codes of the symplectic generalized quadrangle of even order

Let $$q$$q be a prime power and $$W(q)$$W(q) be the symplectic generalized quadrangle of order $$q$$q. For $$q$$q even, let $$\mathcal {O}$$O (respectively, $$\mathcal {E}$$E, $$\mathcal {T}$$T) be the binary linear code spanned by the ovoids (respectively, elliptic ovoids, Tits ovoids) of $$W(q)$$W(q) and $$\Gamma $$Γ be the graph defined on the set of ovoids of $$W(q)$$W(q) in which two ovoids are adjacent if they intersect at one point. For $$\mathcal {A}\in \{\mathcal {E},\mathcal {T},\mathcal {O}\}$$A?{E,T,O}, we describe the codewords of minimum and maximum weights in $$\mathcal {A}$$A and its dual $$\mathcal {A}^{\perp }$$A?, and show that $$\mathcal {A}$$A is a one-step completely orthogonalizable code (Theorem 1.1). We prove that, for $$q>2$$q>2, any blocking set of $$PG(3,q)$$PG(3,q) with respect to the hyperbolic lines of $$W(q)$$W(q) contains at least $$q^2+q+1$$q2+q+1 points and equality holds if and only if it is a hyperplane of $$PG(3,q)$$PG(3,q) (Theorem 1.3). We deduce that a clique in $$\Gamma $$Γ has size at most $$q$$q (Theorem 1.4).

Coding Theorems for Hybrid Channels. II

The present paper continues investigation of the capacities of measurement (quantum-classical) channels in the most general setting, initiated in [A. A. Kuznetsova and A. S. Holevo, Theory Probab. Appl., 58 (2014), pp. 264--285]. The proof of coding theorems is given for the classical capacity and entanglement-assisted classical capacity of the measurement channel with arbitrary output alphabet, without assuming that the channel is given by a bounded operator-valued density.

Comparative Study of Generalized Quantitative-Qualitative Inaccuracy Fuzzy Measures for Noiseless Coding Theorem and 1:1 Codes

In coding theory, we study various properties of codes for application in data compression, cryptography, error correction, and network coding. The study of codes is introduced in Information Theory, electrical engineering, mathematics, and computer sciences for the transmission of data through reliable and efficient methods. We have to consider how coding of messages can be done efficiently so that the maximum number of messages can be sent over a noiseless channel in a given time. Thus, the minimum value of mean codeword length subject to a given constraint on codeword lengths has to be founded. In this paper, we have introduced mean codeword length of orderαand typeβfor 1:1 codes and analyzed the relationship between average codeword length and fuzzy information measures for binary 1:1 codes. Further, noiseless coding theorem associated with fuzzy information measure has been established.

Almost Sure Convergence Coding Theorems of One-Shot and Multi-Shot Tunstall Codes for Stationary Memoryless Sources

Almost Sure Convergence Coding Theorems of One-Shot and Multi-Shot Tunstall Codes for Stationary Memoryless Sources

Codes on Graphs: Fundamentals

This paper develops a fundamental theory of realizations of linear and group codes on general graphs using elementary group theory, including basic group duality theory. Principal new and extended results include: normal realization duality; analysis of systems-theoretic properties of fragments of realizations and their connections; minimal = trim and proper theorem for cycle-free codes; results showing that all constraint codes except interface nodes may be assumed to be trim and proper, and that the interesting part of a cyclic realization is its 2-core; notions of observability and controllability for fragments, and related tests; relations between state-trimness and controllability, and dual state-trimness and observability.

Second-Order Slepian-Wolf Coding Theorems for Non-Mixed and Mixed Sources

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Analogously, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system to establish the source coding theorem in the second-order sense. The Slepian-Wolf coding problem for correlated sources is one of typical problems in the multiterminal information theory. In particular, Miyake and Kanaya, and Han have established the first-order source coding theorems for general correlated sources. On the other hand, in general, the second-order achievable rate problem for the Slepian-Wolf coding system with general sources remains still open up to present. In this paper, we present the analysis concerning the second-order achievable rates for general sources, which are based on the information spectrum methods developed by Han and Verdu. Moreover, we establish the explicit second-order achievable rate region for independently and identically distributed (i.i.d.) correlated sources with countably infinite alphabets and mixtures of i.i.d. correlated sources, respectively, using the relevant asymptotic normality.

Some Source Coding Theorems and 1:1 Coding Based on Generalized Inaccuracy Measure of Order $$\alpha $$ α and Type $$\beta $$ β

In this paper, we have established some noiseless coding theorems for a generalized parametric ‘useful’ inaccuracy measure of order \(\alpha \) and type \(\beta \) and generalized mean codeword length. Further, lower bounds on exponentiated useful code length for the best 1:1 code have been obtained in terms of the useful inaccuracy of order \(\alpha \) and type \(\beta \) and the generalized average useful codeword length.

Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics

Statistical entropy was introduced by Shannon as a basic concept in information theory measuring the average missing information in a random source. Extended into an entropy rate, it gives bounds in coding and compression theorems. In this paper, I describe how statistical entropy and entropy rate relate to other notions of entropy that are relevant to probability theory (entropy of a discrete probability distribution measuring its unevenness), computer sciences (algorithmic complexity), the ergodic theory of dynamical systems (Kolmogorov–Sinai or metric entropy) and statistical physics (Boltzmann entropy). Their mathematical foundations and correlates (the entropy concentration, Sanov, Shannon–McMillan–Breiman, Lempel–Ziv and Pesin theorems) clarify their interpretation and offer a rigorous basis for maximum entropy principles. Although often ignored, these mathematical perspectives give a central position to entropy and relative entropy in statistical laws describing generic collective behaviours, and provide insights into the notions of randomness, typicality and disorder. The relevance of entropy beyond the realm of physics, in particular for living systems and ecosystems, is yet to be demonstrated.

Universal Bounds on the Derivatives of the Symbol Error Rate for Arbitrary Constellations

The symbol error rate (SER) of the minimum distance detector under additive white Gaussian noise is studied in terms of generic bounds and higher order derivatives for arbitrary constellations. A general approach is adopted so that the recent results on the convexity/concavity and complete monotonicity properties of the SER can be obtained as special cases. Novel universal bounds on the SER, which depend only on the constellation dimensionality, minimum and maximum constellation distances are obtained. It is shown that the sphere hardening argument in the channel coding theorem can be derived using the proposed bounds. Sufficient conditions based on the positive real roots (with odd multiplicity) of an explicitly-specified polynomial are presented to determine the signs of the SER derivatives of all orders in signal-to-noise ratio. Furthermore, universal bounds are given for the SER derivatives of all orders. As an example, it is shown that the proposed bounds yield a better characterization of the SER for arbitrary two-dimensional constellations over the complete monotonicity property derived recently.

Some Coding Theorems on Fuzzy Entropy Function Depending Upon Parameter R and Ѵ

In the literature of information theory several types of coding theorems involving fuzzy entropy functions exists. In this paper, some new fuzzy coding theorems have been obtained involving utilities. The fuzzy coding theorems obtained here are not only new but also generalizes some well known results available in the literature.

Some Coding Theorems on Fuzzy Entropy Function Depending Upon Parameter R and Ѵ

In the literature of information theory several types of coding theorems involving fuzzy entropy functions exists. In this paper, some new fuzzy coding theorems have been obtained involving utilities. The fuzzy coding theorems obtained here are not only new but also generalizes some well known results available in the literature.

Coding Theorems for Hybrid Channels

The main result of this paper is the coding theorem for the entanglement-assisted capacity of measurement channel with arbitrary alphabet (Theorem \refcea). As a by-product, a number of results concerning entropies of hybrid (classical-quantum) systems are obtained, as well as the capacities of channels with hybrid output, which are used in the proof of Theorem 3.

Formalization of Shannon’s Theorems

The most fundamental results of information theory are Shannon's theorems. These theorems express the bounds for (1) reliable data compression and (2) data transmission over a noisy channel. Their proofs are non-trivial but are rarely detailed, even in the introductory literature. This lack of formal foundations is all the more unfortunate that crucial results in computer security rely solely on information theory: this is the so-called "unconditional security". In this article, we report on the formalization of a library for information theory in the SSReflect extension of the Coq proof-assistant. In particular, we produce the first formal proofs of the source coding theorem, that introduces the entropy as the bound for lossless compression, and of the channel coding theorem, that introduces the capacity as the bound for reliable communication over a noisy channel.

A characterization of some odd sets in projective spaces of order 4 and the extendability of quaternary linear codes

A set \({\fancyscript{K}}\) in PG(r, 4), r ≥ 2, is odd if every line meets \({\fancyscript{K}}\) in an odd number of points. An odd set \({\fancyscript{K}}\) in PG(r, 4), r ≥ 3, is FH-free if there is no plane meeting \({\fancyscript{K}}\) in a Fano plane or in a non-singular Hermitian curve. We prove that an odd set \({\fancyscript{K}}\) contains a hyperplane of PG(r, 4) if and only if \({\fancyscript{K}}\) is FH-free. As an application to coding theory, a new extension theorem for quaternary linear codes is given.

On Convexity of Error Rates in Digital Communications

Convexity properties of error rates of a class of decoders, including the ML/min-distance one as a special case, are studied for arbitrary constellations, bit mapping and coding. Earlier results obtained for the AWGN channel are extended to a wide class of noise densities, including unimodal and spherically-invariant noise. Under these broad conditions, symbol and bit error rates are shown to be convex functions of the SNR in the high-SNR regime with an explicitly-determined threshold, which depends only on the constellation dimensionality and minimum distance, thus enabling an application of the powerful tools of convex optimization to such digital communication systems in a rigorous way. It is the decreasing nature of the noise power density around the decision region boundaries that insures the convexity of symbol error rates in the general case. The known high/low SNR bounds of the convexity/concavity regions are tightened and no further improvement is shown to be possible in general. The high SNR bound fits closely into the channel coding theorem: all codes, including capacity-achieving ones, whose decision regions include the hardened noise spheres (from the noise sphere hardening argument in the channel coding theorem) satisfies this high SNR requirement and thus has convex error rates in both SNR and noise power. We conjecture that all capacity-achieving codes have convex error rates. Convexity properties in signal amplitude and noise power are also investigated. Some applications of the results are discussed. In particular, it is shown that fading is convexity-preserving and is never good in low dimensions under spherically-invariant noise, which may also include any linear diversity combining.

Corrections to “Hash Property and Coding Theorems for Sparse Matrices and Maximum-Likelihood Coding” [May 10 2143-2167

There is a flaw in the statement of Lemma 5 in the above titled paper (ibid., vol. 56, no. 5, pp. 2143-2167, May 2010), which is used in the proof of Ths. 4, 6, and 7 in the same paper. Revisions are presented here.

Source-Channel Coding Theorems for the Multiple-Access Relay Channel

We study reliable transmission of arbitrarily correlated sources over multiple-access relay channels (MARCs) and multiple-access broadcast relay channels (MABRCs). In MARCs only the destination is interested in reconstructing the sources, while in MABRCs both the relay and the destination want to reconstruct them. In addition to arbitrary correlation among the source signals at the users, both the relay and the destination have side information correlated with the source signals. Our objective is to determine whether a given pair of sources can be losslessly transmitted to the destination for a given number of channel symbols per source sample, defined as the source-channel rate. Sufficient conditions for reliable communication based on operational separation, as well as necessary conditions on the achievable source-channel rates are characterized. Since operational separation is generally not optimal for MARCs and MABRCs, sufficient conditions for reliable communication using joint source-channel coding schemes based on a combination of the correlation preserving mapping technique with Slepian-Wolf source coding are also derived. For correlated sources transmitted over fading Gaussian MARCs and MABRCs, we present conditions under which separation (i.e., separate and stand-alone source and channel codes) is optimal. This is the first time optimality of separation is proved for MARCs and MABRCs.

Conditional Kolmogorov complexity and universal probability

The Coding Theorem of L.A. Levin connects unconditional prefix Kolmogorov complexity with the universal semiprobability mass function. There are conditional versions referred to in several publications but as yet there exist no written proofs. Under the classic definition of conditional probability, there is no conditional version of the Coding Theorem. We give the appropriate definitions and provide complete proofs of the conditional version of the Coding Theorem.

Certain Coding Theorems Based on Generalized Inaccuracy Measure of Order $\alpha$ and Type $\beta$ and 1:1 Coding

In this paper, A new mean codeword length $\L^t_{\beta}(U)$ is defined. We have established some noiseless coding theorems based on generalized inaccuracy measure of order $\alpha$ and type $\beta$. Further, we have defined mean codeword length $\L^t_{\beta, 1:1}(U)$ for the best one-to-one code. Also we have shown that the mean codeword lengths $\L^t_{\beta, 1:1}(U)$ for the best one-to-one code (not necessarily uniquely decodable) are shorter than the mean codeword length $\L^t_{\beta}(U)$. Moreover, we have studied tighter bounds of $\L^t_{\beta}(U)$. Keywords: Generalized inaccuracy measures; Codeword; mean codeword length; Kraft's inequality; Holder's inequality. 2010 Mathematics Subject Classification: 94A15, 94A17, 94A24, 26D15.