Source-channel Coding Research Articles

Design of Code Pair for Protograph-LDPC Codes-Based JSCC System With Joint Shuffled Scheduling Decoding Algorithm

Although there are many studies on code optimization of the joint source-channel coding (JSCC) system based on double protograph low-density parity-check codes with the joint belief propagation (JBP) algorithm, but it is still unknown whether the source code and channel code (as a code pair) can perform well when the joint shuffled scheduling decoding (JSSD) algorithm is adopted. In this letter, two decoding threshold analysis algorithms, including joint shuffled protograph extrinsic information transfer (PEXIT) and source shuffled PEXIT algorithm, are proposed to calculate joint/source decoding thresholds for this system with the JSSD algorithm. With the proposed algorithms, it is found that the optimized code pairs for this system with the JBP algorithm may not perform well with the JSSD algorithm, implying that code pair with the JSSD algorithm needs to be redesigned. Then, a two-stage optimized framework is proposed to design the code pair for this system with the JSSD algorithm. Simulations and decoding threshold analysis both show that the proposed code pair for this system with the JSSD algorithm can obtain lower error floor and better waterfall performance than the existing code pairs.

Bandwidth-Agile Image Transmission With Deep Joint Source-Channel Coding

We propose deep learning based communication methods for adaptive-bandwidth transmission of images over wireless channels. We consider the scenario in which images are transmitted progressively in layers over time or frequency, and such layers can be aggregated by receivers in order to increase the quality of their reconstructions. We investigate two scenarios, one in which the layers are sent sequentially, and incrementally contribute to the refinement of a reconstruction, and another in which the layers are independent and can be retrieved in any order. Those scenarios correspond to the well known problems of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">successive refinement</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multiple descriptions</i> , respectively, in the context of joint source-channel coding (JSCC). We propose DeepJSCC- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> , an innovative solution that uses convolutional autoencoders, and present three architectures with different complexity trade-offs. To the best of our knowledge, this is the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. Numerical results show that DeepJSCC- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> can learn to transmit the source progressively with negligible losses in the end-to-end performance compared with a single transmission. Moreover, DeepJSCC- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes, with the additional advantage of graceful degradation with channel SNR.

A Type-Aware Coding Approach to Joint Source-Channel Coding

In this letter, we propose a type-aware coding approach to joint source-channel coding (JSCC) and present a concrete construction with polarization-adjusted convolutional (PAC) codes. A data sequence is first encoded by enumerative coding and then encoded by a data-dependent PAC code, resulting in a transmitted codeword. At the receiver, a type-search decoding algorithm is employed, and the implementation complexity can be reduced by searching only those typical types. Simulation results show that the presented JSCC scheme can outperform the conventional tandem coding scheme with a fixed-to-fixed-length source code and achieve a performance within the gap between the upper and lower bound on the minimum FER of finite-length JSCCs.

Joint Early Stopping Criterions for Protograph LDPC Codes-Based JSCC System in Images Transmission.

In this paper, a joint early stopping criterion based on cross entropy (CE), named joint CE criterion, is presented for double-protograph low-density parity-check (DP-LDPC) codes-based joint source-channel coding (JSCC) systems in images transmission to reduce the decoding complexity and decrease the decoding delay. The proposed early stopping criterion adopts the CE from the output likelihood ratios (LLRs) of the joint decoder. Moreover, a special phenomenon named asymmetry oscillation-like convergence (AOLC) in the changing process of CE is uncovered in the source decoder and channel decoder of this system meanwhile, and the proposed joint CE criterion can reduce the impact from the AOLC phenomenon. Comparing to the counterparts, the results show that the joint CE criterion can perform well in the decoding complexity and decoding latency in the low–moderate signal-to-noise ratio (SNR) region and achieve performance improvement in the high SNR region with appropriate parameters, which also demonstrates that this system with joint CE is a low-latency and low-power system.

Two-Way Source-Channel Coding

We propose an adaptive lossy joint source-channel coding (JSCC) scheme for sending correlated sources over two-terminal discrete-memoryless two-way channels (DM-TWCs). The main idea is to couple the independent operations of the terminals via an adaptive coding mechanism, which can mitigate cross-interference resulting from simultaneous channel transmissions and concurrently exploit the sources' correlation to reduce the end-to-end reconstruction distortions. Our adaptive JSCC scheme not only subsumes existing lossy coding methods for two-way simultaneous communication but also improves their performance. Furthermore, we derive outer bounds for our two-way lossy transmission problem and establish complete JSCC theorems in some special settings. In these special cases, a non-adaptive separate source-channel coding (SSCC) scheme achieves the optimal performance, thus simplifying the design of the source-channel communication system.

Attention-based neural joint source-channel coding of text for point to point and broadcast channel

Attention-based neural joint source-channel coding of text for point to point and broadcast channel

Performance Analysis of Sphere Packed Aided Differential Space-Time Spreading with Iterative Source-Channel Detection.

The introduction of 5G with excessively high speeds and ever-advancing cellular device capabilities has increased the demand for high data rate wireless multimedia communication. Data compression, transmission robustness and error resilience are introduced to meet the increased demands of high data rates of today. An innovative approach is to come up with a unique setup of source bit codes (SBCs) that ensure the convergence and joint source-channel coding (JSCC) correspondingly results in lower bit error ratio (BER). The soft-bit assisted source and channel codes are optimized jointly for optimum convergence. Source bit codes assisted by iterative detection are used with a rate-1 precoder for performance evaluation of the above mentioned scheme of transmitting sata-partitioned (DP) H.264/AVC frames from source through a narrowband correlated Rayleigh fading channel. A novel approach of using sphere packing (SP) modulation aided differential space time spreading (DSTS) in combination with SBC is designed for the video transmission to cope with channel fading. Furthermore, the effects of SBC with different hamming distances but similar coding rates is explored on objective video quality such as peak signal to noise ratio (PSNR) and also the overall bit error ratio (BER). EXtrinsic Information Transfer Charts (EXIT) are used for analysis of the convergence behavior of SBC and its iterative scheme. Specifically, the experiments exhibit that the proposed scheme of error protection of SBC = 6 outperforms the SBCs having same code rate, but with = 3 by 3 dB with PSNR degradation of 1 dB. Furthermore, simulation results show that a gain of 27 dB is achieved with SBC having code rate 1/3 compared to the benchmark Rate-1 SBC codes.

Joint source-channel coding via model predictive control

We propose a joint source-channel coding (JSCC) scheme based on model-predictive control (MPC) for encoding an analog discrete-time source over a memoryless communication channel. The decoder is assumed to be fixed and given, and the MPC encoder is set to minimize the frequency-weighted mean-squared error (FWMSE) while satisfying a channel input constraint. We derive analytic expressions for the associated finite-horizon optimization MPC problem for an asymptotic (in time) average or sum channel input constraint. For the case of complex additive white Gaussian noise channel and an M-quadrature-amplitude modulation (M-QAM) decoder, these expressions can be written explicitly, which allows for an efficient numerical solution. For the case M=4 and considering a frequency weighting criterion that models the sensitivity of the human hearing, we show by simulations that our proposed scheme yields an FWMSE significantly smaller than that yielded by the corresponding non-JSCC scheme (a 2-bit uniform quantizer in tandem with a 4-QAM channel encoder and decoder).

A Low Distortion Audio Self-Recovery Algorithm Robust to Discordant Size Content Replacement Attack

Although the development of watermarking techniques has enabled designers to tackle normal processing attacks (e.g., amplitude scaling, noise addition, re-compression), robustness against malicious attacks remains a challenge. The discordant size content replacement attack is an attack against watermarking schemes which performs content replacement that increases or reduces the number of samples in the signal. This attack modifies the content and length of the signal, as well as desynchronizes the position of the watermark and its removal. In this paper, a source-channel coding approach for protecting an audio signal against this attack was applied. Before applying the source-channel encoding, a decimation technique was performed to reduce by one-half the number of samples in the original signal. This technique allowed compressing at a bit rate of 64 kbps and obtaining a watermarked audio signal with an excellent quality scale. In the watermark restoration, an interpolation was applied after the source-channel decoding to recover the content and the length. The procedure of decimation–interpolation was taken because it is a linear and time-invariant operation and is useful in digital audio. A synchronization strategy was designed to detect the positions where the number of samples in the signal was increased or reduced. The restoration ability of the proposed scheme was tested with a mathematical model of the discordant size content replacement attack. The attack model confirmed that it is necessary to design a synchronizing strategy to correctly extract the watermark and to recover the tampered signal. Experimental results show that the scheme has better restoration ability than state-of-the-art schemes. The scheme was able to restore a tampered area of around 20% with very good quality, and up to 58.3% with acceptable quality. The robustness against the discordant size content replacement attack was achieved with a transparency threshold above −2.

Joint Source-Channel Coding Over Additive Noise Analog Channels Using Mixture of Variational Autoencoders

In this paper, we present a learning scheme for Joint Source-Channel Coding (JSCC) over analog independent additive noise channels. We formulate the learning problem by showing that the minimization loss function from rate-distortion theory, is upper bounded by the loss function of the Variational Autoencoder (VAE). We show that when the source dimension is greater than the channel dimension, the encoding of two source samples in the neighborhood of each other need not be near each other. Such discontinuous projection needs to be accounted for by using multiple encoders and selecting an encoder to encode samples on a particular side of the discontinuity. We explore two selection methodologies, one based on an intuitive rule and the other where it is posed as a learning task in a Mixture-of-Experts (MoE) setup. We analyze the gradients of these methods and reason why the latter is better at avoiding local optima. We show the efficacy of the proposed methodology by simulating the performance of the system for JSCC of Gaussian sources over AWGN channels and showing that the learned solutions are close to or better than the ones proposed earlier. The proposed methodology is also naturally capable of generalizing to other source distributions which we showcase by simulating for Laplace sources. The learned systems are also robust to changes in channel conditions. Further, a single system can be trained to generalize over a range of channel conditions provided the channel conditions are known at both the transmitter and the receiver. Finally, we evaluate our proposed methodology on three different image datasets and showcase consistent improvement over existing methods due to the VAE formulation.

Minimax Theorems for Finite Blocklength Lossy Joint Source-Channel Coding over an Arbitrarily Varying Channel

Minimax Theorems for Finite Blocklength Lossy Joint Source-Channel Coding over an Arbitrarily Varying Channel

Communicating Correlated Sources Over MAC and Interference Channels II: Joint Source-Channel Coding

We present the second part of our work on communicating correlated sources over multiple access (MAC) and interference channels (IC). Specifically, we undertake a Shannon-theoretic study of the above scenarios and focus on characterizing sufficient conditions for lossless recoverability of the sources at the decoder(s). We enhance the fixed block-length (B-L) coding technique by incorporating the technique of inducing source correlation onto channel inputs, originally discovered by Cover, El Gamal and Salehi. In contrast to the first part, performance analysis of a joint source-channel decoder poses new challenges. We enhance the earlier developed suite of coding and analytical tools to overcome these challenges and derive (simplified) single-letter characterizations for a new set of sufficient conditions for both scenarios. For both the MAC and IC problems, the derived sufficient conditions are (i) subsumed in the current known tightest, and (ii) strictly weaker for identified examples. Lastly, we propose simple `plug-in' approaches that can further weaken the derived sufficient conditions. Our findings enable us to subsume Dueck's findings (1981) and go even further for the example considered therein.

A Trellis-Coded Quantization Approach to Transmitting Correlated Gaussian Sources Over a Fading MAC Without Transmitter-CSI

It is known that source-channel separation is sub-optimal for communicating correlated Gaussian sources over a Gaussian multiple access channel (GMAC). Considering a two-to-one GMAC which undergoes Rayleigh block-fading, we present a novel approach to practical joint source-channel coding for the scenario in which the common receiver has instantaneous CSI but only the CSI distribution is available to the individual transmitters. This approach, referred to as source-channel trellis-coded vector quantization (SC-TCVQ), simply relies on using TCVQs as fixed-rate source-channel encoders. One key issue is the optimization of the TCVQ codebooks to the mean channel signal-to-noise ratio (CSNR), and to this end, we present an analytical method to obtain the rates required for codebook design. Another key issue is the joint estimation of the sources at the receiver, for which we present a detector-estimator based on the Cartesian product of the two encoder-trellises. Simulation results show that the proposed SC-TCVQ codes, in some cases, can even beat the asymptotic performance bound for a separate source-channel code consisting of a distributed vector quantizer and capacity achieving channel codes. SC-TCVQ appears to be the best known practical code design to date for the given communication problem.

A Unified Framework for One-Shot Achievability via the Poisson Matching Lemma

We introduce a fundamental lemma called the Poisson matching lemma, and apply it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels, channel resolvability and wiretap channels. Our one-shot bounds improve upon the best known one-shot bounds in most of the aforementioned settings (except multiple access channels, channel resolvability and wiretap channels, where we recover bounds comparable to the best known bounds), with shorter proofs in some settings even when compared to the conventional asymptotic approach using typicality. The Poisson matching lemma replaces both the packing and covering lemmas, greatly simplifying the error analysis. This paper extends the work of Li and El Gamal on Poisson functional representation, which mainly considered variable-length source coding settings, whereas this paper studies fixed-length settings, and is not limited to source coding, showing that the Poisson functional representation is a viable alternative to typicality for most problems in network information theory.

Quantum Graphs as Quantum Relations

The “noncommutative graphs” which arise in quantum error correction are a special case of the quantum relations introduced in Weaver (Quantum relations. Mem Am Math Soc 215(v–vi):81–140, 2012). We use this perspective to interpret the Knill–Laflamme error-correction conditions (Knill and Laflamme in Theory of quantum error-correcting codes. Phys Rev A 55:900-911, 1997) in terms of graph-theoretic independence, to give intrinsic characterizations of Stahlke’s noncommutative graph homomorphisms (Stahlke in Quantum zero-error source-channel coding and non-commutative graph theory. IEEE Trans Inf Theory 62:554–577, 2016) and Duan, Severini, and Winter’s noncommutative bipartite graphs (Duan et al., op. cit. in Zero-error communication via quantum channels, noncommutative graphs, and a quantum Lovász number. IEEE Trans Inf Theory 59:1164–1174, 2013), and to realize the noncommutative confusability graph associated to a quantum channel (Duan et al., op. cit. in Zero-error communication via quantum channels, noncommutative graphs, and a quantum Lovász number. IEEE Trans Inf Theory 59:1164–1174, 2013) as the pullback of a diagonal relation. Our framework includes as special cases not only purely classical and purely quantum information theory, but also the “mixed” setting which arises in quantum systems obeying superselection rules. Thus we are able to define noncommutative confusability graphs, give error correction conditions, and so on, for such systems. This could have practical value, as superselection constraints on information encoding can be physically realistic.

Optimal Design of Joint Protomatrix for DP-LDPC Codes-Based JSCC System Over on-Body Channel

A high-quality data transmission scheme in wireless body area networks is implemented by a joint source-channel coding (JSCC) system with $M$ -ary differential chaos shift keying modulation over an on-body channel. Current DP-LDPC code pairs, which have shown good bit-error ratio performance over additive white Gaussian noise channel and Rayleigh fading channel, cannot perform well over on-body channel. In this paper, the DP-LDPC codes are redesigned for the JSCC system with new methods depending on the on-body channel characteristics. A protomatrix dimension-restrictive searching algorithm for DP-LDPC codes in the JSCC system is proposed according to the on-body channel bandwith and source statistic, which is effective to predesign the dimension of the code pair for a new JSCC system. In this way, the source-channel code rates are appropriately allocated to leave more source redundancy for improving the error floor. Moreover, the joint protomatrix is reconstructed into two kinds of compact structures by simplifying the edge linking relationships and modifying joint protograph extrinsic information transfer algorithm, which decreases the induction of searching entries to further accelerate the code searching speed. The searched code pairs show better bit-error ratio performances compared with existing DP-LDPC codes. The power consumption in the new joint coding system is lower than the separate coding system in WBAN on the physical layer.

Joint Source-Channel Coding for Semantics-Aware Grant-Free Radio Access in IoT Fog Networks

A fog-radio access network (F-RAN) architecture is studied for an Internet-of-Things (IoT) system in which wireless sensors monitor a number of multi-valued events and transmit in the uplink using grant-free random access to multiple edge nodes (ENs). Each EN is connected to a central processor (CP) via a finite-capacity fronthaul link. In contrast to conventional information-agnostic protocols based on separate source-channel (SSC) coding, where each device uses a separate codebook, this paper considers an information-centric approach based on joint source-channel (JSC) coding via a non-orthogonal generalization of type-based multiple access (TBMA). By leveraging the semantics of the observed signals, all sensors measuring the same event share the same codebook (with non-orthogonal codewords), and all such sensors making the same local estimate of the event transmit the same codeword. The F-RAN architecture directly detects the events values without first performing individual decoding for each device. Cloud and edge detection schemes based on Bayesian message passing are designed and trade-offs between cloud and edge processing are assessed.

Deep Joint Source-Channel Coding for Multi-Task Network

Multi-task learning (MTL) is an efficient way to improve the performance of related tasks by sharing knowledge. However, most existing MTL networks run on a single end and are not suitable for collaborative intelligence (CI) scenarios. In this work, we propose an MTL network with a deep joint source-channel coding (JSCC) framework, which allows operating under CI scenarios. We first propose a feature fusion based MTL network (FFMNet) for joint object detection and semantic segmentation. Compared with other MTL networks, FFMNet gets higher performance with fewer parameters. Then FFMNet is split into two parts, which run on a mobile device and an edge server respectively. The feature generated by the mobile device is transmitted through the wireless channel to the edge server. To reduce the transmission overhead of the intermediate feature, a deep JSCC network is designed. By combining two networks together, the whole model achieves 512x compression for the intermediate feature and a performance loss within 2% on both tasks. At last, by training with noise, the FFMNet with JSCC is robust to various channel conditions and outperforms the separate source and channel coding scheme.

Forward-Aware Information Bottleneck-Based Vector Quantization for Noisy Channels

The main focus will be on the indirect Joint Source-Channel Coding problem in which a noisy observation of the source has to be quantized ahead of transmission over an error-prone forward link to a remote processing unit. To that end, we present here a complete extension to the preliminary Information Bottleneck method by providing the formal optimal solution to this newly established Variational Principle , together with an algorithm, the Forward-Aware Vector Information Bottleneck (FAVIB) , to pragmatically tackle its underlying non-convex design optimization. FAVIB extends the current state-of-the-art approaches via capacitating a full sweep over the entire gamut of the trade-off parameter. Consequently, the trajectory of all achievable points in the Information-Compression plane becomes traversable via soft mappings. It will be shown that, by enjoying an inherent error protection, this novel compression scheme can obviate the call for separate channel coding on the forward path.

Analysis and Improvement of Error-Floor Performance for JSCC Scheme Based on Double Protograph LDPC Codes

In a joint source-channel coding (JSCC) system, excessive source compression can cause an error floor. In the original JSCC scheme based on double protograph LDPC (DP-LDPC) codes, only connections exist between check nodes (CNs) of the source protograph and variable nodes (VNs) of the channel protograph, and error floors are observed. In this paper, we investigate the joint protograph that forms the basis of a DP-LDPC code. The joint protograph consists of a source protograph, a channel protograph, and connections between the two protographs. Our main focus is on the complete-source-variable-linking (CSVL) protomatrix of the joint protograph, which determines the error floor of the DP-LDPC code. We propose a generalized source protograph extrinsic information transfer (GSP-EXIT) algorithm for evaluating the source decoding thresholds of the CSVL protomatrix. Based on the proposed algorithm, we analyze CSVL protomatrices with regular or irregular source protographs. We present design criteria for connections between VNs of the source protograph and CNs of the channel protograph. Such design rules will result in a higher source decoding threshold, which implies a lower error floor. We also propose a differential evolution algorithm for optimizing the source protograph. We present/compare analytical results and/with simulation results, and conclude that they are consistent.