General Discrete Memoryless Channel Research Articles

Identification Over Compound Multiple-Input Multiple-Output Broadcast Channels

The identification capacity region of the compound broadcast channel is determined under an average error criterion, where the sender has no channel state information. We give single-letter identification capacity formulas for discrete channels and multiple-input multiple-output Gaussian channels under an average input constraint. The capacity theorems apply to general discrete memoryless broadcast channels. This is in contrast to the transmission setting, where the capacity is only known for special cases, notably the degraded broadcast channel and the multiple-input multiple-output broadcast channel with private messages. Furthermore, the identification capacity region of the compound multiple-input multiple-output broadcast channel can be larger than the transmission capacity region. This is a departure from the single-user behavior of identification, since the identification capacity of a single-user channel equals the transmission capacity.

Optimal Rates of Teaching and Learning Under Uncertainty

In this paper, we consider a recently-proposed model of teaching and learning under uncertainty, in which a teacher receives independent observations of a single bit corrupted by binary symmetric noise, and sequentially transmits to a student through another binary symmetric channel based on the bits observed so far. After a given number <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> of transmissions, the student outputs an estimate of the unknown bit, and we are interested in the exponential decay rate of the error probability as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> increases. We propose a novel block-structured teaching strategy in which the teacher encodes the number of 1s received in each block, and show that the resulting error exponent is the binary relative entropy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D\left({\frac {1}{2}\|\max (p,q)}\right)$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula> are the noise parameters. This matches a trivial converse result based on the data processing inequality, and settles two conjectures of [Jog and Loh, 2021] and [Huleihel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> , 2019]. In addition, we show that the computation time required by the teacher and student is linear in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> . We also study a more general setting in which the binary symmetric channels are replaced by general binary-input discrete memoryless channels. We provide an achievability bound and a converse bound, and show that the two coincide in certain cases, including (i) when the two channels are identical, and (ii) when the student-teacher channel is a binary symmetric channel. More generally, we give sufficient conditions under which our learning rate is the best possible for block-structured protocols.

Error Exponents in the Bee Identification Problem

The bee identification problem is a problem of properly recognizing a massive amount of data (a numerous amount of bees in a beehive, for example) which have been mixed and corrupted by noise. We derive various error exponents in the bee identification problem under two different decoding rules. Under naïve decoding, which decodes each bee independently of the others, we analyze a general discrete memoryless channel and a relatively wide family of stochastic decoders. Upper and lower bounds to the random coding error exponent are derived and proved to be equal at relatively high coding rates. Then, we propose a lower bound on the error exponent of the typical random code, which improves upon the random coding exponent at low coding rates. We also derive a third bound, which is related to expurgated codes, which turns out to be strictly higher than the other bounds, also at relatively low rates. We show that the universal maximum mutual information decoder is optimal with respect to the typical random code and the expurgated code. Moving further, we derive error exponents under optimal decoding, the relatively wide family of symmetric channels, and the maximum likelihood decoder. We first propose a random coding lower bound, and then, an improved bound which stems from an expurgation process. We show numerically that our second bound strictly improves upon the random coding bound at an intermediate range of coding rates, where a bound derived in a previous work no longer holds.

Second Order Analysis for Joint Source-Channel Coding With General Channel and Markovian Source

We derive the optimal second-order rates in joint source-channel coding when the channel is a general discrete memoryless channel and the source is an irreducible and ergodic Markov process. In contrast, previous studies solved it only when the channel satisfies a certain unique-variance condition and the source is subject to an independent and identical distribution. We also compare the joint source-channel scheme with the separation scheme in the second-order regime, while a previous study made a notable comparison with numerical calculation. To discuss these two topics, we introduce two kinds of new distribution families, switched Gaussian convolution distribution and *-product distribution, which are defined by modifying the Gaussian distribution.

On the Evaluation of Marton’s Inner Bound for Two-Receiver Broadcast Channels

Marton’s inner bound is the best known achievable rate region for a general two-receiver discrete memoryless broadcast channel. In this paper, we establish improved bounds on the cardinalities of the auxiliary random variables appearing in this inner bound to the true rate region. We combine a perturbation technique, along with a representation using concave envelopes of information-theoretic functions that involve the use of auxiliary random variables, to achieve this improvement. The new cardinality bounds lead to a proof that a randomized-time-division strategy achieves every rate triple in Marton’s region for binary input broadcast channels. This extends the result by Hajek and Pursley which showed that the Cover-van der Muelen region was exhausted by the randomized-time-division strategy.

Error Exponents of Typical Random Codes

We define the error exponent of the typical random code (TRC) as the long-block limit of the negative normalized expectation of the logarithm of the error probability of the random code, as opposed to the traditional random coding error exponent, which is the limit of the negative normalized logarithm of the expectation of the error probability. For the ensemble of uniformly randomly drawn fixed composition codes, we provide exact error exponents of TRCs for a general discrete memoryless channel and a wide class of (stochastic) decoders, collectively referred to as the generalized likelihood decoder (GLD). This ensemble of fixed composition codes is shown to be no worse than any other ensemble of independent codewords that are drawn under a permutation-invariant distribution (e.g., independent identically distributed codewords). We also present relationships between the error exponent of the TRC and the ordinary random coding error exponent, as well as the expurgated exponent for the GLD. Finally, we demonstrate that our analysis technique is applicable also to more general communication scenarios, such as list decoding (for fixed-size lists) as well as the decoding with an erasure/list option in Forney’s sense.

On the Capacity of Cloud Radio Access Networks With Oblivious Relaying

We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users’ codebooks, i.e., they perform oblivious processing. The destination, or central processor, however, is informed about the users’ codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay nodes are independent given the users’ inputs. We show that both relaying a-la Cover–El Gamal, i.e., compress-and-forward with joint decompression and decoding, and “noisy network coding” are optimal. The proof of the converse part establishes, and utilizes, connections with the Chief Executive Officer source coding problem under logarithmic loss distortion measure. Extensions to general discrete memoryless channels are also investigated. In this case, we establish the inner and outer bounds on the capacity region. For memoryless Gaussian channels within the studied class of channels, we characterize the capacity region when the users are constrained to time-share among Gaussian codebooks. Furthermore, we also discuss the suboptimality of separate decompression and decoding and the role of time sharing.

Capacity and Rate Regions of a Class of Broadcast Interference Channels

In this paper, a class of broadcast interference channels (BIC) is investigated, where one of the two broadcast receivers is subject to interference coming from a point-to-point transmission. For a general discrete memoryless broadcast interference channel (DM-BIC), an achievable scheme based on message splitting, superposition, and binning is proposed and a concise representation of the corresponding achievable rate region $\mathcal {R}$ is obtained. Two partial-order broadcast conditions interference-oblivious less noisy and interference-cognizant less noisy are defined, thereby extending the usual less noisy condition for a regular broadcast channel by taking interference into account. Under these conditions, a reduced form of $\mathcal {R}$ is shown to be equivalent to a rate region based on a simpler scheme, where the broadcast transmitter uses only superposition. Furthermore, if interference is strong for the interference-oblivious less noisy DM-BIC, the capacity region is given by the aforementioned two equivalent rate regions. For the interference-cognizant less noisy DM-BIC, it is argued that the strong but not very strong interference condition does not exist, and in this case, the capacity region for the very strong interference is obtained. For a Gaussian broadcast interference channel, channel parameters are categorized into three regimes. For the first two regimes, which are closely related to the two partial-order broadcast conditions, achievable rate regions are derived by specializing the corresponding achievable schemes of DM-BICs with Gaussian input distributions. The entropy power inequality-based outer bounds are obtained by combining bounding techniques for a Gaussian broadcast channel and a Gaussian interference channel. These inner and outer bounds lead to either exact or approximate characterizations of capacity regions and sum capacity under various conditions. For the remaining complementing regime, inner and outer bounds are also provided.

Energy-Efficient Communication in the Presence of Synchronization Errors

Communication systems are traditionally designed to have tight transmitter–receiver synchronization. This requirement has negligible overhead in the high-signal-to-noise ratio (SNR) regime. However, in many applications, such as wireless sensor networks, communication needs to happen primarily in the energy-efficient regime of low SNR, where requiring tight synchronization can be highly suboptimal. In this paper, we model the noisy channel with synchronization errors as a duplication/deletion/substitution channel. For this channel, we propose a new communication scheme that requires only loose transmitter–receiver synchronization. We show that the proposed scheme is asymptotically optimal for the Gaussian channel with synchronization errors in terms of energy efficiency as measured by the rate per unit energy. In the process, we also establish that the lack of synchronization causes negligible loss in energy efficiency. We further show that, for a general discrete memoryless channel with synchronization errors and a general input cost function admitting a zero-cost symbol, the rate per unit cost achieved by the proposed scheme is within a factor two of the information-theoretic optimum.

Dynamic Quantizers in Presence of Noisy Channels

In this paper, we deal with the stabilizability of a scalar plant through a noisy discrete channel. Our scheme can be regarded as a variation of the controllers based on dynamic quantizers introduced in [3] and [4], but our analysis based on Lyapunov function allows us to take into consideration more general discrete memoryless channels. Given a plant and a discrete memoryless channel we will provide, using control Lyapunov function ideas, a condition for the existence of an encoder/controller pair with a priori fixed memory that achieves stability. A construction procedure is also provided.

Bounds on Maximum Likelihood Decoding Performance for Linear Codes at Low Rates

For a given linear code (ensemble), upper bounds on the error probability under maximum likelihood decoding are investigated at low rates. A class of symmetric memoryless channels suitable for Bhattacharyya-type bounds, which are particularly of importance at low rates, is introduced. Over a symmetric channel, a lower bound on the error exponent is derived for a given linear code. A sufficient condition for achieving the expurgated exponent, which is the best among known error exponents at low rates, within a fixed discrepancy is given. Over a general discrete memoryless channel, the same analysis provides a lower bound on the average error exponent for an ensemble of coset codes generated by a given linear code. The bounding technique is extended to the case of generalized maximum likelihood decoding with erasure and list- decoding options.

Analysis of Deterministic Binary Interference Channels Via a General Outer Bound

We present a new outer bound for the general two-user discrete memoryless interference channel (IFC) and use it to establish the capacity region of the binary erasure IFC, whose determination was left open in . We also show that there are essentially two deterministic binary IFCs, in addition to the binary erasure IFC, whose capacity regions are not obvious from previous results. We determine the capacity region of one of these and apply the aforementioned general outer bound to obtain the best available bound on the maximum achievable sum-rate for the other. We also show that the new general outer bound is tight for one-sided deterministic IFCs that belong to the class studied by El Gamal and Costa.

Error Exponents for the Gaussian Channel With Active Noisy Feedback

We study the best exponential decay in the block length of the probability of error that can be achieved in the transmission of a single bit over the Gaussian channel with an active noisy Gaussian feedback link. We impose an expected block power constraint on the forward link and study both almost-sure and expected block power constraints on the feedback link. In both cases the best achievable error exponents are finite and grow approximately proportionally to the larger between the signal-to-noise ratios on the forward and feedback links. The error exponents under almost-sure block power constraints are typically strictly smaller than under expected constraints. Some of the results extend to communication at arbitrary rates below capacity and to general discrete memoryless channels.

Secure broadcasting using multiple antennas

We consider three different secure broadcasting scenarios: i) Broadcast channels with common and confidential messages (BCC), ii) multi-receiver wiretap channels with public and confidential messages, and iii) compound wiretap channels. The BCC is a broadcast channel with two users, where in addition to the common message sent to both users, a private message, which needs to be kept hidden as much as possible from the other user, is sent to each user. In this model, each user treats the other user as an eavesdropper. The multi-receiver wiretap channel is a broadcast channel with two legitimate users and an external eavesdropper, where the transmitter sends a pair of public and confidential messages to each legitimate user. Although there is no secrecy concern about the public messages, the confidential messages need to be kept perfectly secret from the eavesdropper. The compound wiretap channel is a compound broadcast channel with a group of legitimate users and a group of eavesdroppers. In this model, the transmitter sends a common confidential message to the legitimate users, and this confidential message needs to be kept perfectly secret from all eavesdroppers. In this paper, we provide a survey of the existing information-theoretic results for these three forms of secure broadcasting problems, with a closer look at the Gaussian multiple-input multiple-output (MIMO) channel models. We also present the existing results for the more general discrete memory-less channel models, as they are often the first step in obtaining the capacity results for the corresponding Gaussian MIMO channel models.

Rate Regions for Relay Broadcast Channels

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> A partially cooperative relay broadcast channel (RBC) is a three-node network with one source node and two destination nodes (destinations 1 and 2) where destination 1 can act as a relay to assist destination 2. Inner and outer bounds on the capacity region of the discrete memoryless partially cooperative RBC are obtained. When the relay function is disabled, the inner bound reduces to an inner bound on the capacity region of broadcast channels that includes an inner bound of Marton, and Gel'fand and Pinsker. The outer bound reduces to a new outer bound on the capacity region of broadcast channels that generalizes an outer bound of Marton to include a common message, and that generalizes an outer bound of Gel'fand and Pinsker to apply to general discrete memoryless broadcast channels. The proof for the outer bound simplifies the proof of Gel'fand and Pinsker that was based on a recursive approach. Four classes of RBCs are studied in detail. For the partially cooperative RBC with degraded message sets, inner and outer bounds are obtained. For the semideterministic partially cooperative RBC and the orthogonal partially cooperative RBC, the capacity regions are established. For the parallel partially cooperative RBC with unmatched degraded subchannels, the capacity region is established for the case of degraded message sets. The capacity is also established when the source node has only a private message for destination 2, i.e., the channel reduces to a parallel relay channel with unmatched degraded subchannels. </para>

Broadcast Channels With Cooperating Decoders

We consider the problem of communicating over the general discrete memoryless broadcast channel (BC) with partially cooperating receivers. In our setup, receivers are able to exchange messages over noiseless conference links of finite capacities, prior to decoding the messages sent from the transmitter. In this paper we formulate the general problem of broadcast with cooperation. We first find the capacity region for the case where the BC is physically degraded. Then, we give achievability results for the general broadcast channel, for both the two independent messages case and the single common message case.

Towards an information theory of large networks: an achievable rate region

We study communication networks of arbitrary size and topology and communicating over a general vector discrete memoryless channel (DMC). We propose an information-theoretic constructive scheme for obtaining an achievable rate region in such networks. Many well-known capacity-defining achievable rate regions can be derived as special cases of the proposed scheme. A few such examples are the physically degraded and reversely degraded relay channels, the Gaussian multiple-access channel, and the Gaussian broadcast channel. The proposed scheme also leads to inner bounds for the multicast and allcast capacities. Applying the proposed scheme to a specific wireless network of n nodes located in a region of unit area, we show that a transport capacity of /spl Theta/(n) bit-meters per second (bit-meters/s) is feasible in a certain family of networks, as compared to the best possible transport capacity of /spl Theta/(/spl radic/n) bit-meters/s in Gupta et al. (2000), where the receiver capabilities were limited. Even though the improvement is shown for a specific class of networks, a clear implication is that designing and employing more sophisticated multiuser coding schemes can provide sizable gains in at least some large wireless networks.

Lower bounds on the quantum capacity and highest error exponent of general memoryless channels

Tradeoffs between the information rate and fidelity of quantum error-correcting codes are discussed. Quantum channels to be considered are those subject to independent errors and modeled as tensor products of copies of a general completely positive linear map, where the dimension of the underlying Hilbert space is a prime number. On such a quantum channel, the highest fidelity of a quantum error-correcting code of length $n$ and rate R is proven to be lower bounded by 1 - \exp [-n E(R) + o(n)] for some function E(R). The E(R) is positive below some threshold R', which implies R' is a lower bound on the quantum capacity. The result of this work applies to general discrete memoryless channels, including channel models derived from a physical law of time evolution, or from master equations.

Random-access systems with a time varying channel

A random multiple-access system which employs a collision resolution algorithm for accessing a common time-varying channel is considered. The channel is assumed to be in one of two states. In each state, the channel is characterized by a general discrete memoryless channel and the transitions between the two states are assumed to be Markovian. A set of sufficient conditions for the ergodicity (stability) of the system is established. As an example, the case when the Capetanakis tree algorithm operates with a time-varying noisy channel is examined. The stability of this system is analyzed, and the throughput is evaluated.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Partial converse for a relay channel

A special case of the general discrete memoryless relay channel is studied. This channel consists of an input x, a relay output z, a channel output y, and a noiseless channel of capacity C/sub r/ allowing the relay sender to send additional information to the decoder. The channel is described by a collection of distributions (p(y, z mod x)). The decoder uses the channel output y and a function f(z) of the relay output at a rate no greater than C/sub r/ to recover the message. Some results on the capacity of the channel are obtained. >