Source-channel Separation Theorem Research Articles

Reliable and Secure Short-Packet Communications in Untrusted Diamond Relay Networks

This paper investigates short-packet communications over a diamond relay network with two untrusted relays (potential eavesdroppers) with the purpose of guaranteeing reliability and security simultaneously. As the performance metric, reliable-and-secure probability (RSP), considering both reliability and secrecy, is defined from the perspective of physical layer security. An analytical approach is proposed to investigate reliability and security by taking into account the characteristics of the short-packet transmission. RSPs are numerically obtained via the analytical framework by utilizing the source-channel separation theorem for the source-to-relay transmission and chief executive officer (CEO) problem analyses for the relay-to-destination transmission. It is found that the optimal RSP is achieved when the contributions from the source-relay transmission and the relay-destination transmission are balanced, even without support from a friendly jamming signal or artificial noise.

Timeliness optimization of unmanned aerial vehicle lossy communications for internet-of-things

Information freshness is a key factor for Internet-of-Things (IoT) to make appropriate decisions and operations. This paper proposes an analytical framework for evaluating the timeliness performance of the IoT system based on Unmanned Aerial Vehicle (UAV) lossy communications. The performance analysis consists of the outage probability analysis and the Age-of-Information (AoI) analysis with outages. To begin with, we solve a lossy coding problem formulated from the UAV communication system, and derive a closed-form expression of the outage probability based on Shannon’s lossy source-channel separation theorem. Then, we characterize the Peak AoI (PAoI) for the considered system, and further minimize the PAoI by deriving the optimal rate for generating information. Moreover, we analyze the system performance through theoretical calculations and simulations. The results indicate that the optimal server utilization ratio is always no larger than 0.5. In practical applications, we can utilize the proposed analytical framework to determine the system parameters which guarantee the timeliness performance of UAV lossy communications.

Joint source-channel coding for 6G communications

In order to provide ultra low-latency and high energy-efficient communication for intelligences, the sixth generation (6G) wireless communication networks need to break out of the dilemma of the depleting gain of the separated optimization paradigm. In this context, this paper provides a comprehensive tutorial that overview how joint source-channel coding (JSCC) can be employed for improving overall system performance. For the purpose, we first introduce the communication requirements and performance metrics for 6G. Then, we provide an overview of the source-channel separation theorem and why it may not hold in practical applications. In addition, we focus on two new JSCC schemes called the double low-density parity-check (LDPC) codes and the double polar codes, respectively, giving their detailed coding and decoding processes and corresponding performance simulations. In a nutshell, this paper constitutes a tutorial on the JSCC scheme tailored to the needs of future 6G communications.

Cooperative Lossy Communications in Unmanned Aerial Vehicle Networks: Age-of-Information With Outage Probability

This article analyzes the robustness and timeliness for cooperative lossy communications in unmanned aerial vehicle (UAV) networks. The analytical framework consists of two steps: 1) calculating the outage probability, and 2) characterizing the Age-of-Information (AoI) for a given outage probability. Initially, we determine the outage probability based on the Shannon's lossy source-channel separation theorem. Numerical results indicate that joint decoding reduces the outage probability, and the system can achieve higher diversity order for less stringent distortion requirement. Then, we derive a closed-form expression of the lower bound on the average AoI for the communication system where outage events are constrained to an acceptable level. Moreover, we conduct a series of simulations for verifying the lower bound on the average AoI and evaluating the impact of UAV locations on the average AoI. It is demonstrated that the lower bound is tight when the server utilization ratio is either relatively busy or idle. For the case with a relatively large outage probability, we propose an intermittent transmission scheme to refine the AoI performance without extra energy consumption. The effectiveness of the intermittent transmission scheme for reducing the average AoI is verified by both the simulations and the lower bound analysis.

Rate-Distortion and Outage Probability Analyses for Single Helper Assisted Lossy Communications

The primary objective of this paper is to investigate the performance improvement provided by a helper, with the aim of applications to a lossy communication system in wireless sensor networks (WSNs) and/or Internet of Things (IoT). Initially, we formulate the system model as a problem of multiterminal source coding with a helper, and determine an inner bound on the achievable rate-distortion region for binary sources. Based on Shannon's lossy source-channel separation theorem, we further derive the outage probability of the system over block Rayleigh fading channels. The numerical results indicate that a helper can obviously expand the achievable rate-distortion region and provide diversity gains which in turn decrease the outage probability. Finally, we evaluate the practical performance of outage probability through simulations with exclusive-or (XOR) as an example of the helper structure. Both the simulation and theoretical results show second order diversity by introducing a helper, and lower outage probability by increasing the acceptable distortions.

A Source-Channel Separation Theorem With Application to the Source Broadcast Problem

A converse method is developed for the source broadcast problem. Specifically, it is shown that the separation architecture is optimal for a variant of the source broadcast problem and the associated source-channel separation theorem can be leveraged, via a reduction argument, to establish a necessary condition for the original problem, which unifies several existing results in the literature. Somewhat surprisingly, this method, albeit based on the source-channel separation theorem, can be used to prove the optimality of non-separation based schemes and determine the performance limits in certain scenarios where the separation architecture is suboptimal.

A Lower Bound Analysis of Hamming Distortion for a Binary CEO Problem With Joint Source-Channel Coding

A two-node binary chief executive officer (CEO) problem is investigated. Noise-corrupted versions of a binary sequence are forwarded by two nodes to a single destination node over orthogonal additive white Gaussian noise (AWGN) channels. We first reduce the binary CEO problem to a binary multiterminal source coding problem, of which an outer bound for the rate-distortion region is derived. The distortion function is then established by evaluating the relationship between the binary CEO and multiterminal source coding problems. A lower bound approximation on the Hamming distortion (HD) is obtained by minimizing a distortion function subject to constraints obtained based on the source-channel separation theorem. Encoding/decoding algorithms using concatenated convolutional codes and a joint decoding scheme are used to verify the lower bound on the HD. It is found that the theoretical lower bounds on the HD and the computer simulation-based bit error rate performance curves have the same tendencies. The differences in the threshold signal-to-noise ratio between the theoretical lower bounds and those obtained by simulations are around 1.5 dB in AWGN channel. The theoretical lower bound on the HD in block Rayleigh fading channel is also evaluated by performing Monte Carlo simulation.

On the source-channel separation theorem for infinite source alphabets

The single-user source-channel separation theorem has been proved for many classes of sources and channels, including sources with finite or countably infinite alphabets. Typically, the source-channel separation theorem is first proved for sources with a finite alphabet, and then the results are extended to sources with a countably infinite alphabet. This paper considers the direct extension of the source-channel separation theorem for some classes of sources with a finite alphabet to a countably infinite alphabet. Specifically, we provide a solution for memoryless sources and arbitrary channels. It is then discussed how this approach may be extended to the case of general sources and arbitrary channels.

Quantum Rate Distortion, Reverse Shannon Theorems, and Source-Channel Separation

We derive quantum counterparts of two key theorems of classical information theory, namely, the rate distortion theorem and the source-channel separation theorem. The rate-distortion theorem gives the ultimate limits on lossy data compression, and the source-channel separation theorem implies that a two-stage protocol consisting of compression and channel coding is optimal for transmitting a memoryless source over a memoryless channel. In spite of their importance in the classical domain, there has been surprisingly little work in these areas for quantum information theory. In the present paper, we prove that the quantum rate distortion function is given in terms of the regularized entanglement of purification. We also determine a single-letter expression for the entanglement-assisted quantum rate distortion function, and we prove that it serves as a lower bound on the unassisted quantum rate distortion function. This implies that the unassisted quantum rate distortion function is non-negative and generally not equal to the coherent information between the source and distorted output (in spite of Barnum's conjecture that the coherent information would be relevant here). Moreover, we prove several quantum source-channel separation theorems. The strongest of these are in the entanglement-assisted setting, in which we establish a necessary and sufficient codition for transmitting a memoryless source over a memoryless quantum channel up to a given distortion.

On Optimum Parameter Modulation–Estimation From a Large Deviations Perspective

We consider the problem of jointly optimum modulation and estimation of a real-valued random parameter, conveyed over an additive white Gaussian noise (AWGN) channel, where the performance metric is the large deviations behavior of the estimator, namely, the exponential decay rate (as a function of the observation time) of the probability that the estimation error would exceed a certain threshold. Our basic result is in providing an exact characterization of the fastest achievable exponential decay rate, among all possible modulator-estimator (transmitter-receiver) pairs, where the modulator is limited only in the signal power, but not in bandwidth. This exponential rate turns out to be given by the reliability function of the AWGN channel. We also discuss several ways to achieve this optimum performance, and one of them is based on quantization of the parameter, followed by optimum channel coding and modulation, which gives rise to a separation-based transmitter, if one views this setting from the perspective of joint source-channel coding. This is in spite of the fact that, in general, when error exponents are considered, the source-channel separation theorem does not hold true. We also discuss several observations, modifications and extensions of this result in several directions, including other channels, and the case of multidimensional parameter vectors. One of our findings concerning the latter, is that there is an abrupt threshold effect in the dimensionality of the parameter vector: below a certain critical dimension, the probability of excess estimation error may still decay exponentially, but beyond this value, it must converge to unity.

Joint Source–Channel Coding for Transmitting Correlated Sources Over Broadcast Networks

We consider a set of S independent encoders that must transmit a set of correlated sources through a network of noisy, independent, broadcast channels to T receivers, with no interference at the receivers. For the general problem of sending correlated sources through broadcast networks, it is known that the source-channel separation theorem breaks down and the achievable rate region as well as the proper method of coding are unknown. For our scenario, however, we establish the optimal rate region using a form of joint source-channel coding. When the optimal channel input distribution from transmitter i to receiver j is independent of j, our result has a max-flow/min-cut interpretation. Specifically, in this case, our result implies that if it is possible to send the sources to each receiver separately while ignoring the others, then it is possible to send to all receivers simultaneously.

Finite State Channels With Time-Invariant Deterministic Feedback

We consider capacity of discrete-time channels with feedback for the general case where the feedback is a time-invariant deterministic function of the output samples. Under the assumption that the channel states take values in a finite alphabet, we find a sequence of achievable rates and a sequence of upper bounds on the capacity. The achievable rates and the upper bounds are computable for any N, and the limits of the sequences exist. We show that when the probability of the initial state is positive for all the channel states, then the capacity is the limit of the achievable-rate sequence. We further show that when the channel is stationary, indecomposable, and has no intersymbol interference (ISI), its capacity is given by the limit of the maximum of the (normalized) directed information between the input X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> and the output Y <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> , i.e., C=lim <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nrarrinfin</sub> (1/n)max I(X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> rarrY <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> ) where the maximization is taken over the causal conditioning probability Q(x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> parz <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N-1</sup> ) defined in this paper. The main idea for obtaining the results is to add causality into Gallager's results on finite state channels. The capacity results are used to show that the source-channel separation theorem holds for time-invariant determinist feedback, and if the state of the channel is known both at the encoder and the decoder, then feedback does not increase capacity.

Joint Source–Channel Codes for MIMO Block-Fading Channels

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We consider transmission of a continuous amplitude source over an <emphasis><formula formulatype="inline"><tex>$L$</tex></formula></emphasis>-block Rayleigh-fading <emphasis><formula formulatype="inline"><tex>$M_t \times M_r$</tex></formula></emphasis> multiple-input multiple-output (MIMO) channel when the channel state information is only available at the receiver. Since the channel is not ergodic, Shannon's source–channel separation theorem becomes obsolete and the optimal performance requires a joint source–channel approach. Our goal is to minimize the expected end-to-end distortion, particularly in the high signal-to-noise ratio (SNR) regime. The figure of merit is the distortion exponent, defined as the exponential decay rate of the expected distortion with increasing SNR. We provide an upper bound and lower bounds for the distortion exponent with respect to the bandwidth ratio among the channel and source bandwidths. For the lower bounds, we analyze three different strategies based on layered source coding concatenated with progressive superposition or hybrid digital/analog transmission. In each case, by adjusting the system parameters we optimize the distortion exponent as a function of the bandwidth ratio. We prove that the distortion exponent upper bound can be achieved when the channel has only one degree of freedom, that is <emphasis><formula formulatype="inline"><tex>$L=1$</tex></formula></emphasis>, and <emphasis><formula formulatype="inline"><tex>$\min\{M_t,M_r\}=1$</tex> </formula></emphasis>. When we have more degrees of freedom, our achievable distortion exponents meet the upper bound for only certain ranges of the bandwidth ratio. We demonstrate that our results, which were derived for a complex Gaussian source, can be extended to more general source distributions as well. </para>

Computation Over Multiple-Access Channels

The problem of reliably reconstructing a function of sources over a multiple-access channel (MAC) is considered. It is shown that there is no source-channel separation theorem even when the individual sources are independent. Joint source-channel strategies are developed that are optimal when the structure of the channel probability transition matrix and the function are appropriately matched. Even when the channel and function are mismatched, these computation codes often outperform separation-based strategies. Achievable distortions are given for the distributed refinement of the sum of Gaussian sources over a Gaussian multiple-access channel with a joint source-channel lattice code. Finally, computation codes are used to determine the multicast capacity of finite-field multiple-access networks, thus linking them to network coding.

Multiterminal Source–Channel Communication Over an Orthogonal Multiple-Access Channel

We consider the problem of multiterminal source-channel communication where a number of distributed and possibly correlated sources are transmitted through an orthogonal multiple access channel to a common destination. We provide a characterization of the optimal tradeoff between the transmission cost Gamma and the distortion vector D as measured against individual sources. Our approach consists of two steps: 1) a multiple-letter characterization of the rate-distortion region of the multiterminal source coding and 2) a source-channel separation theorem ensuring that all achievable pairs of (Gamma, D) can be obtained by combining the rate-distortion region and the orthogonal multiple access channel capacity region. As a corollary, we determine the optimal power and distortion tradeoff in a quadratic Gaussian sensor network under orthogonal multiple access, and show that separate source and channel coding strictly outperforms the uncoded (amplify-forward) transmission, and is in fact optimal in this case. This result is in sharp contrast to the case of nonorthogonal multiple access for which separate source and channel coding is not only suboptimal but also strictly inferior to uncoded transmission.

On Multistage Successive Refinement for Wyner–Ziv Source Coding With Degraded Side Informations

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In this correspondence, we provide a complete characterization of the rate-distortion region for the <emphasis emphasistype="boldital">multistage</emphasis> successive refinement of the Wyner–Ziv source coding problem with degraded side informations at the decoder. Necessary and sufficient conditions for a source to be successively refinable along a distortion vector are subsequently derived. A source–channel separation theorem is provided when the descriptions are sent over independent channels for the multistage case. Furthermore, the notion of generalized successive refinability with multiple degraded side informations is introduced. This notion captures whether progressive encoding to satisfy multiple distortion constraints for different side informations is as good as encoding without progressive requirement. Necessary and sufficient conditions for generalized successive refinability are given. It is shown that the following two sources are generalized successively refinable: 1) the Gaussian source with degraded Gaussian side informations and 2) the doubly symmetric binary source when the worse side information is a constant. Thus for both cases, the failure of being successively refinable is only due to the inherent uncertainty on which side information will occur at the decoder, but not the progressive encoding requirement. </para>

On the shannon cipher system with a capacity-limited key-distribution channel

We consider the Shannon cipher system in a setting where the secret key is delivered to the legitimate receiver via a channel with limited capacity. For this setting, we characterize the achievable region in the space of three figures of merit: the security (measured in terms of the equivocation), the compressibility of the cryptogram, and the distortion associated with the reconstruction of the plaintext source. Although lossy reconstruction of the plaintext does not rule out the option that the (noisy) decryption key would differ, to a certain extent, from the encryption key, we show, nevertheless, that the best strategy is to strive for perfect match between the two keys, by applying reliable channel coding to the key bits, and to control the distortion solely via rate-distortion coding of the plaintext source before the encryption. In this sense, our result has a flavor similar to that of the classical source-channel separation theorem. Some variations and extensions of this model are discussed as well

On the Capacity of Large Gaussian Relay Networks

The capacity of a particular large Gaussian relay network is determined in the limit as the number of relays tends to infinity. Upper bounds are derived from cut-set arguments, and lower bounds follow from an argument involving uncoded transmission. It is shown that in cases of interest, upper and lower bounds coincide in the limit as the number of relays tends to infinity. Hence, this paper provides a new example where a simple cut-set upper bound is achievable, and one more example where uncoded transmission achieves optimal performance. The findings are illustrated by geometric interpretations. The techniques developed in this paper are then applied to a sensor network situation. This is a network joint source-channel coding problem, and it is well known that the source-channel separation theorem does not extend to this case. The present paper extends this insight by providing an example where separating source from channel coding does not only lead to suboptimal performance-it leads to an exponential penalty in performance scaling behavior (as a function of the number of nodes). Finally, the techniques developed in this paper are extended to include certain models of ad hoc wireless networks, where a capacity scaling law can be established: When all nodes act purely as relays for a single source-destination pair, capacity grows with the logarithm of the number of nodes.

The source-channel separation theorem revisited

The single-user separation theorem of joint source-channel coding has been proved previously for wide classes of sources and channels. We find an information-stable source/channel pair which does not satisfy the separation theorem. New necessary and sufficient conditions for the transmissibility of a source through a channel are found, and we characterize the class of channels for which the separation theorem holds regardless of the source statistics.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>