Discrete Memoryless Channel Research Articles

Information Theoretical Clustering Is Hard to Approximate

An impurity measures I : R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</sup> → R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> is a function that assigns a d-dimensional vector v to a non-negative value I(v) so that the more homogeneous v, with respect to the values of its coordinates, the larger its impurity. A well known example of impurity measures is the entropy impurity. We study the problem of clustering based on the entropy impurity measures. Let V be a collection of n many d-dimensional vectors with non-negative components. Given V and an impurity measure I, the goal is to find a partition n of V into k groups V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , . . . , V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sub> so as to minimize the sum of the impurities of the groups in P, i.e., I(P) = Σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i=1</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sup> I (Σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">v∈Vi</sub> v). Impurity minimization has been widely used as quality assessment measure in probability distribution clustering (KL-divergence) as well as in categorical clustering. However, in contrast to the case of metric based clustering, the current knowledge of impurity measure based clustering in terms of approximation and in approximability results is very limited. Here, we contribute to change this scenario by proving that the problem of finding a clustering that minimizes the Entropy impurity measure is APX-hard, i.e., there exists a constant ε > 0 such that no polynomial time algorithm can guarantee (1 + ε)-approximation under the standard complexity hypothesis P ≠ N P . The in approximability holds even when all vectors have the same 11 norm. This result provides theoretical limitations on the computational efficiency that can be achievable in the quantization of discrete memoryless channels, a problem that has recently attracted significant attention in the signal processing community. In addition, it also solve a question that remained open in previous work on this topic [Chaudhuri and McGregor COLT 08; Ackermann et. al. ECCC 11].

Second- and Third-Order Asymptotics of the Continuous-Time Poisson Channel

The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. We also obtain bounds on the third-order coding rate. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution induced by Wyner's discretized channel and the construction of an appropriate $\epsilon$-net of the input probability simplex. While the achievability proof follows the general program to prove the third-order term for non-singular discrete memoryless channels put forth by Polyanskiy, several non-standard techniques -- such as new definitions and bounds on the probabilities of typical sets using logarithmic Sobolev inequalities -- are employed to handle the continuous nature of the channel.

Distributed Hypothesis Testing Based on Unequal-Error Protection Codes

Coding and testing schemes for binary hypothesis testing over noisy networks are proposed and their corresponding type-II error exponents are derived. When communication is over a discrete memoryless channel (DMC), our scheme combines Shimokawa-Han-Amari's hypothesis testing scheme with Borade-Nakiboglu-Zheng's unequal error protection (UEP) for channel coding where source and channel codewords are simultaneously decoded. The resulting exponent is optimal for the newly introduced class of generalized testing against conditional independence. When communication is over a multi-access channel (MAC), our scheme combines hybrid coding with UEP. The resulting error exponent over the MAC is optimal in the case of generalized testing against conditional independence with independent observations at the two sensors when the MAC decomposes into two individual DMCs. In this case, separate source-channel coding is sufficient and no UEP is required. This same conclusion holds also under arbitrarily correlated sensor observations when testing is against independence.

On the Commitment Capacity of Unfair Noisy Channels

Noisy channels are a valuable resource from a cryptographic point of view. They can be used for exchanging secret-keys as well as realizing other cryptographic primitives such as commitment and oblivious transfer. To be really useful, noisy channels have to be considered in the scenario where a cheating party has some degree of control over the channel characteristics. Damgard et al. (EUROCRYPT 1999) proposed a more realistic model where such level of control is permitted to an adversary, the so called unfair noisy channels, and proved that they can be used to obtain commitment and oblivious transfer protocols. Given that noisy channels are a precious resource for cryptographic purposes, one important question is determining the optimal rate in which they can be used. The commitment capacity has already been determined for the cases of discrete memoryless channels and Gaussian channels. In this work we address the problem of determining the commitment capacity of unfair noisy channels. We compute a single-letter characterization of the commitment capacity of unfair noisy channels. In the case where an adversary has no control over the channel (the fair case) our capacity reduces to the well-known capacity of a discrete memoryless binary symmetric channel.

Stochastic Belief Propagation Polar Decoding With Efficient Re-Randomization

Polar codes, the first class of codes achieving the capacity of symmetric binary-input discrete memoryless channels (B-DMCs), have drawn intensive attentions. For speed-insensitive applications, such as implantable devices, there is a good need on complexity and robustness. In this article, stochastic computing is introduced to allieviate the contradiction between complexity and performance for polar decoders. We proposed a stochastic belief propagation (BP) polar decoder and corresponding novel approaches to improve its decoding performance. Based on the understanding of stochastic computing correlation (SCC) in polar coding, the efficient stage-wise re-randomization scheme and its implementation are proposed and carefully analyzed. Hardware architectural optimizations and corresponding ASIC implementation are also given and demonstrate advantages over the state-of-the-art.

The Bit Error Performance and Information Transfer Rate of SPAD Array Optical Receivers

In this paper the photon counting characteristics, the information rate and the bit error performance of single-photon avalanche diode (SPAD) arrays are investigated. It is shown that for sufficiently large arrays, the photocount distribution is well approximated by a Gaussian distribution with dead-time-dependent mean and variance. Because of dead time, the SPAD array channel is subject to counting losses, part of which are due to inter-slot interference (ISI) distortions. Consequently, this channel has memory. The information rate of this channel is assessed. Two auxiliary discrete memoryless channels (DMCs) are proposed which provide upper and lower bounds on the SPAD array information rate. It is shown that in sufficiently large arrays, ISI is negligible and the bounds are tight. Under such conditions, the SPAD array channel is precisely modelled as a memoryless channel. A discrete-time Gaussian channel with input-dependent mean and variance is adopted and the properties of the capacity-achieving input distributions are studied. Using a numerical algorithm, the information rate and the capacity-achieving input distributions, subject to peak and average power constraints are obtained. Furthermore, the bit error performance of a SPAD-based system with on-off keying (OOK) is evaluated for various array sizes, dead times and background count levels.

The PSK Channel Capacity Estimation under Dynamic Plasma Sheath Channel

During the reentry process, the plasma sheath covering the surface of the hypersonic aircraft will cause the amplitude attenuation and phase jitter of the communication electromagnetic waves. Channel parameters such as the electron density and collision frequency of the plasma sheath reflect the changing trend of the plasma sheath, and these parameters can be measured by physical means. However, these parameters cannot directly reflect the change of the channel communication ability and cannot directly serve the design of communication methods in the plasma sheath. Due to the particularity of the plasma sheath, the traditional channel estimation method for Additive White Gaussian Noise channels will no longer be applicable. This paper presents a channel capacity estimation method for dynamic plasma sheath. First, the plasma sheath is equivalent to a discrete input continuous output memoryless channel, and then the channel capacity expression is derived according to Shannon formula. Finally, the channel capacity of the dynamic plasma sheath is estimated by calculating the transition probability density function. The simulation results show that the channel capacity of the dynamic plasma sheath is affected by both the signal-to-noise ratio (SNR) and the dynamic parameters of the plasma sheath. When the electron density is small, the channel capacity is mainly affected by the SNR. As the electron density increases, the dynamic parameters of the plasma sheath gradually become the main factor affecting the channel capacity. This method is a theoretical analysis of the channel capacity when the channel parameters of the plasma channel are known, and it is meaningful for conducting the work of communication methods design.

Belief Propagation Decoding of Polar Codes Using Intelligent Post-Processing

Polar code is a channel coding method that has been proved to be able to reach Shannon capacity in the binary discrete memoryless channel. Because of the superior performance and low encoding and decoding complexity, polar code has attracted extensive attention in the industry and been chosen as the channel coding scheme for the control channel in the scenario of EMBB in 5G mobile communication. In this work, we propose an intelligent BP decoding algorithm of polar code based on smart post-processing. We employ the neural network to classify the output data of regular BP decoding into “good-bit” and “bad-bit” categories. We also design a strategy to search the bits, which are most probably incorrect from the “bad-bit” group for post-processing. Then, we can invert the “bad-bit” to correct the residual error in the Belief Propagation (BP) iterative process. Simulation results prove that the proposed algorithm can achieve at least 0.5dB error correction performance enhancement compared with the regular BP decoding with slight computation complexity and energy consumption increase.

Computing Classical-Quantum Channel Capacity Using Blahut-Arimoto Type Algorithm: A Theoretical and Numerical Analysis.

Based on Arimoto’s work in 1972, we propose an iterative algorithm for computing the capacity of a discrete memoryless classical-quantum channel with a finite input alphabet and a finite dimensional output, which we call the Blahut–Arimoto algorithm for classical-quantum channel, and an input cost constraint is considered. We show that, to reach accuracy, the iteration complexity of the algorithm is upper bounded by where n is the size of the input alphabet. In particular, when the output state is linearly independent in complex matrix space, the algorithm has a geometric convergence. We also show that the algorithm reaches an accurate solution with a complexity of , and in the special case, where m is the output dimension, is the relative entropy of two distributions, and is a positive number. Numerical experiments were performed and an approximate solution for the binary two-dimensional case was analysed.

An Efficient Software Stack Sphere Decoder for Polar Codes

Proved to achieve the symmetric capacity of the binary-input discrete memoryless channels, polar codes have been chosen for the eMBB control channels in the 5th generation mobile communication systems. Besides the main decoding algorithms like successive cancellation (SC) decoding and CRC-aid SC list (CA-SCL) decoding, sphere decoder (SD) and list SD (LSD) are the alternatives for short codes with less required memory bits. Existing SD and LSD attain high calculation complexity, for SD requires a back-tracking process and LSD needs a large list size $L$ to achieve satisfying performance. To reduce complexity, an efficient software stack sphere decoder (ESSD) based on the synchronous determination is firstly proposed in this article. With the dynamic set-by-set decoding in the stack structure, it achieves the lowest complexity in SD-based decoders (SD/LSD/ESSD) while sharing the same performance on low-rate codes and high-rate codes. Compared with the CA-SCL decoder, the complexity and latency of the proposed ESSD are also competitive at high signal-to-noise-ratio on the displayed codes. Implemented on C++ , the proposed ESSD reduces $\text{44.77}\%$ latency compared with CA-SCL-32 for $\mathcal {P}(128,120)$ at the BER of $10^{-5}$ with $E_b/N_0 = \text{7}$ dB.

Channel capacity in brain–computer interfaces

Objective. Adapted from the concept of channel capacity, the information transfer rate (ITR) has been widely used to evaluate the performance of a brain–computer interface (BCI). However, its traditional formula considers the model of a discrete memoryless channel in which the transition matrix presents very particular symmetries. As an alternative to compute the ITR, this work indicates a more general closed-form expression—also based on that channel model, but with less restrictive assumptions—and, with the aid of a selection heuristic based on a wrapper algorithm, extends such formula to detect classes that deteriorate the operation of a BCI system. Approach. The benchmark is a steady-state visually evoked potential (SSVEP)-based BCI dataset with 40 frequencies/classes, in which two scenarios are tested: (1) our proposed formula is used and the classes are gradually evaluated in the order of the class labels provided with the dataset; and (2) the same formula is used but with the classes evaluated progressively by a wrapper algorithm. In both scenarios, the canonical correlation analysis (CCA) is the tool to detect SSVEPs. Main results. Before and after class selection using this alternative ITR, the average capacity among all subjects goes from 3.71 1.68 to 4.79 0.70 bits per symbol, with p -value <0.01, and, for a supposedly BCI-illiterate subject, her/his capacity goes from 1.53 to 3.90 bits per symbol. Significance. Besides indicating a consistent formula to compute ITR, this work provides an efficient method to perform channel assessment in the context of a BCI experiment and argues that such method can be used to study BCI illiteracy.

A simplified decoding algorithm for multi-CRC polar codes

Polar codes represent one of the major breakthroughs in 5G standard, and have been proven to be able to achieve the symmetric capacity of binary-input discrete memoryless channels using the successive cancellation list (SCL) decoding algorithm. However, the SCL algorithm suffers from a large amount of memory overhead. This paper proposes an adaptive simplified decoding algorithm for multiple cyclic redundancy check (CRC) polar codes. Simulation results show that the proposed method can reduce the decoding complexity and memory space. It can also acquire the performance gain in the low signal to noise ratio region.

An Efficient Software List Sphere Decoder for Polar Codes

Polar codes, first achieving the capacity of symmetric binary-input discrete memoryless channels (B-DMCs), have been standardized for eMBB control channels. Since 5G cellular requires flexible architecture which is realized by the software defined networking paradigm, efficient polar decoder is anticipated. Though successive cancellation list (SCL) decoder achieves satisfactory performance, it requires a large amount of memory. For short control channel codes, sphere decoder (SD) is an alternative, but costs unbearable time complexity at low signal-to-noise ratio. List sphere decoder (LSD) abandons the radius and keeps a list of best paths to gain a fixed complexity. However, LSD needs a large list size L for satisfactory performance. In this paper, an efficient software LSD with path pruning and efficient sorting is proposed. We recall the radius as the bound to delete the paths out of the sphere at very early levels. Since L is dynamic, efficient sorting is proposed to reduce the copy operations. Implemented with C++, the proposed decoder can reduce up to 65.3% latency compared with the original LSD, with the same performance and lower complexity.

Covert Communication With Polynomial Computational Complexity

This paper develops a concatenated coding scheme with polynomial computational complexity for covert communication over Binary Symmetric Channels (BSCs) and binary-input Discrete Memoryless Channels (DMCs). Our setting is as follows — a transmitter Alice wishes to potentially reliably transmit a message to a receiver Bob, while ensuring that the transmission taking place is covert with respect to a warden Willie (who hears Alice’s transmission over another independent channel). Prior works showed that Alice can reliably and covertly transmit $\mathcal {O}(\sqrt {{n}})$ message bits over n channel uses, but one drawback is that the computational complexity of the codes designed scales as $2^{\Theta (\sqrt {{n}})}$ . In this work we provide a capacity-achiveing coding scheme with provable guarantees on both reliability and covertness, and its computational complexity grows polynomially in the blocklength n .

Tracking Unstable Autoregressive Sources Over Discrete Memoryless Channels

We consider the problem of tracking, in realtime, an unstable autoregressive (AR) source over a discrete memoryless channel (DMC). We present computable achievable bounds on the optimal tracking error for general DMCs, and we particularize these bounds to the binary erasure, packet erasure, and binary symmetric channels. The achievable bounds in this paper are proved using a partially separate source quantization and channel coding architecture. We do not use complete or strict separation in usual Shannon sense: 1) the quantiser’s resolution is optimized against the error-correction capabilities of the channel code and the channel code is optimized against an AR Hamming distortion function matched to the source (a weighted Hamming distortion function that provides unequal error protection to different parts of the AR source). The achievability results for general DMCs are proved by combining the AR Hamming distortion function with new realtime (streaming) versions of the random coding union and dependence testing bounds . When applied to erasure channels, these general bounds combine with simple converses to demonstrate that the channel’s cutoff rate plays an important role in realtime tracking.

Distributed Hypothesis Testing Over Discrete Memoryless Channels

A distributed binary hypothesis testing (HT) problem involving two parties, one referred to as the observer and the other as the detector is studied. The observer observes a discrete memoryless source (DMS) and communicates its observations to the detector over a discrete memoryless channel (DMC). The detector observes another DMS correlated with that at the observer, and performs a binary hypothesis test on the joint distribution of the two DMS’s using its own observed data and the information received from the observer. The trade-off between the type I error probability and the type II error-exponent of the HT is explored. Single-letter lower bounds on the optimal type II error-exponent are obtained by using two different coding schemes, a separate HT and channel coding scheme and a joint HT and channel coding scheme based on hybrid coding for the matched bandwidth case. Exact single-letter characterization of the same is established for the special case of testing against conditional independence, and it is shown to be achieved by the separate HT and channel coding scheme. An example is provided where the joint scheme achieves a strictly better performance than the separation based scheme.

A New Iterative Algorithm for Computing the Correct Decoding Probability Exponent of Discrete Memoryless Channels

Dueck and Korner’s reliability function for discrete memoryless channels for rates above the capacity coincides with Arimoto’s exponent of correct decoding probability. The two exponent functions are described by seemingly different optimization problems over the space of probability distributions. Arimoto gave an iterative algorithm for solving the optimization problem that appears in his exponent function. However, no algorithm to solve the optimization problem that appears in Dueck and Korner’s exponent has been proposed. This paper proposes a new iterative algorithm for solving the minimization problem in Dueck and Korner’s exponent. In the proposed algorithm, a double minimization form with respect to two joint distributions on input and output symbols is introduced. This double minimization is connected to another double minimization that appears in Arimoto’s algorithm. Such a connection leads to a quadruple minimization problem, by which the match of Arimoto and Dueck-Korner exponents is easily proved.

On Finite Alphabet Iterative Decoding of LDPC Codes With High-Order Modulation

This work aims to apply the mutual information-maximizing finite alphabet iterative decoding (MIM-FAID) method to decode low-density parity-check (LDPC) codes with high-order modulation. To this end, we propose an efficient model, through which our task becomes to design an MIM quantizer for a binary-input continuous memoryless channel (CMC) with log-likelihood ratios (LLRs) as the output. We then properly discretize the LLRs to convert the CMC to a discrete memoryless channel (DMC) such that we can obtain the MIM quantizer by using dynamic programming. Both theoretical analysis and simulation results verify the effectiveness of our proposed method.

Covert Communication Over a $K$ -User Multiple-Access Channel

We consider a scenario in which $K$ transmitters attempt to communicate covert messages reliably to a legitimate receiver over a discrete memoryless multiple-access channel (MAC) while simultaneously escaping detection from an adversary who observes their communication through another discrete memoryless MAC. We assume that each transmitter may use a secret key that is shared only between itself and the legitimate receiver. We show that each of the $K$ transmitters can transmit on the order of $\sqrt {n}$ reliable and covert bits per $n$ channel uses, exceeding which, the warden will be able to detect the communication. We identify the optimal pre-constants of the scaling, which leads to a complete characterization of the covert capacity region of the $K$ -user binary-input MAC. We show that, asymptotically, all sum-rate constraints are inactive unlike the traditional MAC capacity region. We also characterize the channel conditions that have to be satisfied for the transmitters to operate without a secret key.

Covert Secret Key Generation With an Active Warden

We investigate the problem of covert and secret key generation over a state-dependent discrete memoryless channel with one-way public discussion in which an adversary, the warden, may arbitrarily choose the channel state. We develop an adaptive protocol that, under conditions that we explicitly specify, not only allows the transmitter and the legitimate receiver to exchange a secret key but also conceals from the active warden whether the protocol is being run. When specialized to passive adversaries that do not control the channel state, we partially characterize the covert secret key capacity. In particular, the covert secret key capacity is sometimes equal to the covert capacity of the channel, so that secrecy comes "for free."