Continuous Alphabet Research Articles

On the Data Processing Theorem in the Semi–deterministic Setting

Data processing lower bounds on the expected distortion are derived in the finite-alphabet semi-deterministic setting, where the source produces a deterministic, individual sequence, but the channel model is probabilistic, and the decoder is subjected to various kinds of limitations, e.g., decoders implementable by finite-state machines, with or without counters, and with or without a restriction of common reconstruction with high probability. Some of our bounds are given in terms of the Lempel-Ziv complexity of the source sequence or the reproduction sequence. We also demonstrate how some analogous results can be obtained for classes of linear encoders and linear decoders in the continuous alphabet case.

Expurgated Random-Coding Ensembles: Exponents, Refinements, and Connections

This paper studies expurgated random-coding bounds and exponents for channel coding with a given (possibly suboptimal) decoding rule. Variations of Gallager's analysis are presented, yielding several asymptotic and non-asymptotic bounds on the error probability for an arbitrary codeword distribution. A simple non-asymptotic bound is shown to attain an exponent of Csisz\'ar and K\"orner under constant-composition coding. Using Lagrange duality, this exponent is expressed in several forms, one of which is shown to permit a direct derivation via cost-constrained coding which extends to infinite and continuous alphabets. The method of type class enumeration is studied, and it is shown that this approach can yield improved exponents and better tightness guarantees for some codeword distributions. A generalization of this approach is shown to provide a multi-letter exponent which extends immediately to channels with memory. Finally, a refined analysis expurgated i.i.d. random coding is shown to yield a O\big(\frac{1}{\sqrt{n}}\big) prefactor, thus improving on the standard O(1) prefactor. Moreover, the implied constant is explicitly characterized.

Mismatched Decoding: Error Exponents, Second-Order Rates and Saddlepoint Approximations

This paper considers the problem of channel coding with a given (possibly suboptimal) maximum-metric decoding rule. A cost-constrained random-coding ensemble with multiple auxiliary costs is introduced, and is shown to achieve error exponents and second-order coding rates matching those of constant-composition random coding, while being directly applicable to channels with infinite or continuous alphabets. The number of auxiliary costs required to match the error exponents and second-order rates of constant-composition coding is studied, and is shown to be at most two. For independent identically distributed random coding, asymptotic estimates of two well-known non-asymptotic bounds are given using saddlepoint approximations. Each expression is shown to characterize the asymptotic behavior of the corresponding random-coding bound at both fixed and varying rates, thus unifying the regimes characterized by error exponents, second-order rates, and moderate deviations. For fixed rates, novel exact asymptotics expressions are obtained to within a multiplicative 1+o(1) term. Using numerical examples, it is shown that the saddlepoint approximations are highly accurate even at short block lengths.

On multiple‐access relay channel with common message

AbstractIn this paper, we obtain a general achievable rate region for discrete memoryless multiple‐access relay channel (DM‐MARC) with common message via partial decode and forward strategy and regular block Markov encoding/backward decoding. This general achievable rate region, as an extension of the achievability part of the relay channel via partial decode and forward strategy to the MARC, subsumes the previous known achievable rate regions for multiple‐access channel with common message and MARC. Also, by applying Fano's inequality, we derive an outer bound for DM‐MARC with common message. By extending the discrete alphabet results to the continuous alphabet, an inner bound for Gaussian MARC (GMARC) is achieved. As expected, our results for the Gaussian MARC includes the previous results for the Gaussian multiple‐access channel with and without common message and the Gaussian relay channel as special cases. Finally, by considering a specific topology for MARC, performance of GMARC with common message versus the position of the relay is investigated numerically. Also, it is shown that our achievable rate region is larger than the previous rate region of GMARC with common message with irregular encoding/successive decoding. Copyright © 2014 John Wiley & Sons, Ltd.

Optimization of radar phase-coded signals for multiple target detection

We address the design of radar phase-coded signals for the detection of multiple targets in the presence of a coloured Gaussian disturbance with a similarity constraint and a detection constraint. The weighted average signal-to-noise ratio (SNR), which characterises the overall level of multiple targets detection performance, is used as the cost function for the optimization of the radar waveform with a constraint on the degree of similarity with a prefixed radar signal and a constraint on the minimal SNR threshold for every target. We formulate the optimization problem in terms of a non-convex quadratically constrained quadratic program (QCQP) that is NP-hard. Hence, we propose a similarity and detection constrained signal (SDCS) design algorithm relying on both the semidefinite relaxation (SDR) technique and the randomization technique to obtain an accurate approximation of the optimal solution for the case with a continuous phase alphabet and the case with a finite phase alphabet. Finally, the results of extensive simulations demonstrate that the phase-coded signals generated with our approach are accurate approximations of the optimal signals and that a trade-off exists between the detection performance and the requirements on the signal similarity and the minimal SNR threshold.

Curves on Flat Tori and Analog Source-Channel Codes

In this paper we consider the problem of transmitting a continuous alphabet discrete-time source over an AWGN channel. We propose a constructive scheme based on a set of curves on the surface of a N-dimensional sphere. Our approach shows that the design of good codes for this communication problem is related to geometrical properties of spherical codes and projections of N-dimensional rectangular lattices. Theoretical comparisons with some previous works in terms of the mean square error as a function of the channel SNR as well as simulations are provided.

Photon losses depending on polarization mixedness

We introduce a quantum channel describing photon losses depending on the degree of polarization mixedness. This can be regarded as a model of quantum channel with correlated errors between discrete and continuous degrees of freedom. We consider classical information over a continuous alphabet encoded on weak coherent states as well as classical information over a discrete alphabet encoded on single photons using dual rail representation. In both cases we study the one-shot capacity of the channel and its behaviour in terms of correlation between losses and polarization mixedness.

Polynomial based analog source-channel codes - [transactions papers

In many communication applications one is interested in transmitting a time-discrete analog-valued (i.e. continuous alphabet) source over a time-discrete analog channel. We study this problem in the case of bandwidth expansion, in the sense that one source sample, X, is transmitted over N-orthogonal channels. An analog source-channel code based on orthogonal polynomials is proposed and analyzed. The code can be generated using a Gram-Schmidt procedure, to fit virtually any source distribution.

Information Theory and Network Coding (Yeung, R.W.; 2008) [Book review

This is an updated and extended version of the author's 2002 book A First Course in Information Theory. The current book consists of two parts. Part I contains 16 chapters, 14 of them basically identical with those of the first version, not addressing networks. The two new chapters cover differential entropy and continuous alphabet (mainly Gaussian) channels. The second part is of main interest; it provides a comprehensive, state-of-the-art presentation of the theory of network coding. Much of the material is new compared with the 2002 book, and did not even exist in 2002; the rest has also been substantially updated. The book is well written. A commendable new feature is that each chapter ends with a short and clear summary. As before, each chapter is complemented with problems. This is a valuable book that many scientists will use as a reference. It can also be used as a textbook and looks particularly suitable for special purpose courses.

Joint Source–Channel Coding Excess Distortion Exponent for Some Memoryless Continuous-Alphabet Systems

We investigate the joint source-channel coding (JSCC) excess distortion exponent <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EJ</i> (the exponent of the probability of exceeding a prescribed distortion level) for some memoryless communication systems with continuous alphabets. We first establish upper and lower bounds for <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EJ</i> for systems consisting of a memoryless Gaussian source under the squared-error distortion fidelity criterion and a memoryless additive Gaussian noise channel with a quadratic power constraint at the channel input. A necessary and sufficient condition for which the two bounds coincide is provided, thus exactly determining the exponent. This condition is observed to hold for a wide range of source-channel parameters. As an application, we study the advantage in terms of the excess distortion exponent of JSCC over traditional tandem (separate) coding for Gaussian systems. A formula for the tandem exponent is derived in terms of the Gaussian source and Gaussian channel exponents, and numerical results show that JSCC often substantially outperforms tandem coding. The problem of transmitting memoryless Laplacian sources over the Gaussian channel under the magnitude-error distortion is also carried out. Finally, we establish a lower bound for <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EJ</i> for a certain class of continuous source-channel pairs when the distortion measure is a metric.

Polynomial based analog source-channel codes

In many communication applications one is interested in transmitting a time-discrete analog-valued (i.e. continuous alphabet) source over a time-discrete analog channel. We study this problem in the case of bandwidth expansion, in the sense that one source sample, X, is transmitted over N orthogonal channels. An analog source-channel code based on orthogonal polynomials is proposed and analyzed. The code can be generated using a Gram-Schmidt procedure, to fit virtually any source distribution.

Capacity Analysis for Continuous Alphabet Channels With Side Information, Part II: MIMO Channels

In this second part of our two-part paper, we consider the capacity analysis for wireless mobile systems with multiple-antenna architectures. We apply the results of the first part to a commonly known baseband, discrete-time multiple-antenna system where both the transmitter and receiver know the channel's statistical law. We analyze the capacity for additive white Gaussian noise (AWGN) channels, fading channels with full channel state information (CSI) at the receiver, fading channels with no CSI, and fading channels with partial CSI at the receiver. For each type of channels, we study the capacity value as well as issues such as the existence, uniqueness, and characterization of the capacity-achieving measures for different types of moment constraints. The results are applicable to both Rayleigh and Rician fading channels in the presence of arbitrary line-of-sight and correlation profiles.

Critical error rate of quantum-key-distribution protocols versus the size and dimensionality of the quantum alphabet

A quantum-information analysis of how the size and dimensionality of the quantum alphabet affect the critical error rate of the quantum-key-distribution (QKD) protocols is given on an example of two QKD protocols---the six-state and \ensuremath{\infty}-state (i.e., a protocol with continuous alphabet) ones. In the case of a two-dimensional Hilbert space, it is shown that, under certain assumptions, increasing the number of letters in the quantum alphabet up to infinity slightly increases the critical error rate. Increasing additionally the dimensionality of the Hilbert space leads to a further increase in the critical error rate.

Capacity Analysis for Continuous-Alphabet Channels With Side Information, Part I: A General Framework

Capacity analysis for channels with side information at the receiver has been an active area of interest. This problem is well investigated for the case of finite-alphabet channels. However, the results are not easily generalizable to the case of continuous-alphabet channels due to analytical difficulties inherent with continuous alphabets. In the first part of this two-part paper, we address an analytical framework for capacity analysis of continuous alphabet channels with side information at the receiver. For this purpose, we establish novel necessary and sufficient conditions for weak* continuity and strict concavity of the mutual information. These conditions are used in investigating the existence and uniqueness of the capacity-achieving measures. Furthermore, we derive necessary and sufficient conditions that characterize the capacity value and the capacity-achieving measure for continuous-alphabet channels with side information at the receiver.

Design of Channel-Estimate-Dependent Space–Time Block Codes

So far, the assumption of no channel knowledge at the transmitter has generally been inherent in the design of space-time codes. This paper, on the other hand, assumes that quantized channel information obtained from a feedback link is available at the transmitter and investigates how such channel information can be incorporated into the design of unstructured space-time block codes. Efficient codes are found by means of a gradient search over a continuous alphabet. Simulation results for an uncorrelated Rayleigh fading scenario using two and four transmit antennas and one receive antenna show the benefits of the code designs.

Finite sample identifiability of multiple constant modulus sources

We prove that mixtures of continuous alphabet constant modulus sources can be identified with probability 1 with a finite number of samples (under noise-free conditions). This strengthens earlier results which only considered an infinite number of samples. The proof is based on the linearization technique of the analytical constant modulus algorithm (ACMA), together with a simple inductive argument. We then study the finite-alphabet case. In this case, we provide a subexponentially decaying upper bound on the probability of nonidentifiability for a finite number of samples. We show that under practical assumptions, this upper bound is tighter than the currently known bound. We then provide an improved exponentially decaying upper bound for the case of L-PSK signals (L is even).

Curves on a sphere, shift-map dynamics, and error control for continuous alphabet sources

We consider two codes based on dynamical systems, for transmitting information from a continuous alphabet, discrete-time source over a Gaussian channel. The first code, a homogeneous spherical code, is generated by the linear dynamical system s/spl dot/=As, with A a square skew-symmetric matrix. The second code is generated by the shift map s/sub n/=b/sub n/s/sub n-1/(mod 1). The performance of each of these codes is determined by the geometry of its locus or signal set, specifically, its arc length and minimum distance, suitably defined. We show that the performance analyses for these systems are closely related, and derive exact expressions and bounds for relevant geometric parameters. We also observe that the lattice /spl Zopf//sup N/ underlies both modulation systems and we develop a fast decoding algorithm that relies on this observation. Analytic results show that for fixed bandwidth expansion, good scaling behavior of the mean squared error is obtained relative to the channel signal-to-noise ratio (SNR). Particularly interesting is the resulting observation that sampled, exponentially chirped modulation codes are good bandwidth expansion codes.

Entropic measures, Markov information sources and complexity

The concept of entropy plays a major part in communication theory. The Shannon entropy is a measure of uncertainty with respect to a priori probability distribution. In algorithmic information theory the information content of a message is measured in terms of the size in bits of the smallest program for computing that message. This paper discusses the classical entropy and entropy rate for discrete or continuous Markov sources, with finite or continuous alphabets, and their relations to program-size complexity and algorithmic probability. The accent is on ideas, constructions and results; no proofs will be given.

Failure of successive refinement for symmetric Gaussian mixtures

We show that information refinement fails for a continuous alphabet source whose input distribution is the symmetric mixture of normal densities with means -/spl mu/ and /spl mu/ and common variance /spl sigma//sup 2/.

Low-delay communication for Rayleigh fading channels: an application of the multiple description quantizer

We show the multiple description scalar quantizer (MDSQ), when used over a slow fading Rayleigh channel, results in a smaller interleaving delay than traditional channel code-based approaches and substantially outperforms a system using a maximum ratio combiner. We consider a continuous alphabet source without memory over a slow Rayleigh fading channel. Specifically, it is initially assumed that the Rayleigh parameter remains constant over a specific time interval and the demodulator has available perfect estimates of the Rayleigh parameter. An analysis of the problem is presented and theoretical justification is provided for using the MDSQ. For a Gaussian source without memory, performance comparisons are presented against a traditional maximum ratio combiner (MRC)-based system as well as against channel code-based systems. It is shown that the performance of the MDSQ-based system dominates the MRC-based system at all channel signal-to-noise ratios and is superior to channel code-based systems when the interleaving delay is constrained. Simulation results for a mobile radio channel model at 15 m.p.h. indicate a reduction in the interleaving delay by at least a factor of three.