Slepian-Wolf Research Articles

Efficient information reconciliation for high-dimensional quantum key distribution

The information reconciliation phase in quantum key distribution has significant impact on the range and throughput of any QKD system. We explore this stage for high-dimensional QKD implementations and introduce two novel methods for reconciliation. The methods are based on nonbinary LDPC codes and the Cascade algorithm, respectively, and achieve efficiencies close the Slepian–Wolf bound on q-ary symmetric channels.

Universal almost Optimal Compression and Slepian-wolf Coding in Probabilistic Polynomial Time

In a lossless compression system with target lengths, a compressor 𝒞 maps an integer m and a binary string x to an m -bit code p , and if m is sufficiently large, a decompressor 𝒟 reconstructs x from p . We call a pair ( m,x ) achievable for (𝒞,𝒟) if this reconstruction is successful. We introduce the notion of an optimal compressor 𝒞 opt by the following universality property: For any compressor-decompressor pair (𝒞,𝒟), there exists a decompressor 𝒟 ′ such that if (m,x) is achievable for (𝒞,𝒟), then ( m + Δ , x ) is achievable for (𝒞 opt , 𝒟 ′ ), where Δ is some small value called the overhead. We show that there exists an optimal compressor that has only polylogarithmic overhead and works in probabilistic polynomial time. Differently said, for any pair (𝒞,𝒟), no matter how slow 𝒞 is, or even if 𝒞 is non-computable, 𝒞 opt is a fixed compressor that in polynomial time produces codes almost as short as those of 𝒞. The cost is that the corresponding decompressor is slower. We also show that each such optimal compressor can be used for distributed compression, in which case it can achieve optimal compression rates as given in the Slepian–Wolf theorem and even for the Kolmogorov complexity variant of this theorem.

Performance analysis of one‐source‐with‐one‐helper transmission over shadowed κ$\kappa$‐μ$\mu$ fading multiple access channels

The performance of correlated sources transmission over multiple access shadowed κ $\kappa$ - μ $\mu$ fading channels is investigated, in which one of the correlated sources needs to be recovered at the destination, whereas the other serves as a helper. The sufficient condition for lossless coding is determined by the intersection of the modified Slepian–Wolf region and the multiple access channel region. The outage probability upper bounds are derived based on the sufficient condition, with the Gaussian codebook capacity and the constellation constrained capacity, respectively. The difference between the outage probabilities derived with the two kinds of capacities is found to be very minor, when the spectrum efficiency or source rate is low; however, with high spectrum efficiency or high source rate, such difference becomes significant. A closed-form outage approximation is also obtained at the high signal-to-noise ratio region. The accuracy of the analytical results is verified by the Monte-Carlo simulations. It is found that shadowing significantly affects the outage performance, however, it has no effect on the diversity gain. Furthermore, the power allocation between the source and the helper is studied to minimize the outage probability and it is found that generally more power should be allocated to the helper in the case with higher source-helper correlation.

Distributed source coding for utilization of inter/intra source correlation

In many practical distributed source coding (DSC) applications, correlation information plays several important roles at decoding, which provides the prior knowledge of the source statistics for the decoding algorithms to start work. Therefore, the accurate utilization of such correlation information has a significant impact on decoding performance. In this work, a hybrid DSC (HYDSC) scheme is proposed to generate the “soft” information about the clean source symbols, which allows the channel code-based DSC scheme to effectively utilize the inter/intra source correlation without requiring prior knowledge of its statistics, so as to further improve the decoding performance. Further performance improvement, at the cost of moderate increasing in computational complexity, can be achieved by performing additional iterations on the basis of the HYDSC scheme. We mainly focus on, in particular, applications where Slepian–Wolf coding and usual source coding are jointly used to exploit both the correlation among the adjoining source symbols and the correlation between the sources. Experiment results indicate that the presented scheme not only show superior performance from an error correction viewpoint, but also achieve better compression ratio in the case of strong internal correlation of the source sequence, as compared with the previous DSC techniques.

Distributed Compression of Graphical Data

In contrast to time series, graphical data is data indexed by the vertices and edges of a graph. Modern applications such as the internet, social networks, genomics and proteomics generate graphical data, often at large scale. The large scale argues for the need to compress such data for storage and subsequent processing. Since this data might have several components available in different locations, it is also important to study distributed compression of graphical data. In this paper, we derive a rate region for this problem which is a counterpart of the Slepian–Wolf theorem. We characterize the rate region when the statistical description of the distributed graphical data can be modeled as being one of two types – as a member of a sequence of marked sparse Erdős–Rényi ensembles or as a member of a sequence of marked configuration model ensembles. Our results are in terms of a generalization of the notion of entropy introduced by Bordenave and Caputo in the study of local weak limits of sparse graphs. Furthermore, we give a generalization of this result for Erdős–Rényi and configuration model ensembles with more than two sources.

On More General Distributions of Random Binning for Slepian–Wolf Encoding

Traditionally, ensembles of Slepian-Wolf (SW) codes are defined such that every bin of each $n$-vector of each source is randomly drawn under the uniform distribution across the sets $\{0,1,\ldots,2^{nR_X}-1\}$ and $\{0,1,\ldots,2^{nR_Y}-1\}$, where $R_X$ and $R_Y$ are the coding rates of the two sources, $X$ and $Y$, respectively. In a few more recent works, where only one source, say, $X$, is compressed and the other one, $Y$, serves as side information available at the decoder, the scope is extended to variable-rate S-W (VRSW) codes, where the rate is allowed to depend on the type class of the source string, but still, the random-binning distribution is assumed uniform within the corresponding, type-dependent, bin index set. In this expository work, we investigate the role of the uniformity of the random binning distribution from the perspective of the trade-off between the reliability (defined in terms of the error exponent) and the compression performance (measured from the viewpoint of the source coding exponent). To this end, we study a much wider class of random-binning distributions, which includes the ensemble of VRSW codes as a special case, but it also goes considerably beyond. We first show that, with the exception of some pathological cases, the smaller ensemble, of VRSW codes, is as good as the larger ensemble in terms the trade-off between the error exponent and the source coding exponent. Notwithstanding this finding, the wider class of ensembles proposed is motivated in two ways. The first is that it outperforms VRSW codes in the above-mentioned pathological cases, and the second is that it allows robustness: in the event of a system failure that causes unavailability of the compressed bit-stream from one of the sources, it still allows reconstruction of the other source within some controllable distortion.

A Low-Complexity and Adaptive Distributed Source Coding Design for Model Aggregation in Distributed Learning

A major bottleneck in distributed learning is the communication overhead of exchanging intermediate model update parameters between the worker nodes and the parameter server. Recently, it is found that local gradients among different worker nodes are correlated. Therefore, distributed source coding (DSC) can be applied to increase communication efficiency by exploiting such correlation. However, it is highly non-trivial to exploite the gradient correlations in distributed learning due to the unknown and time-varying gradient correlation. In this paper, we first propose a DSC framework, named successive Wyner-Ziv coding, for distributed learning based on quantization and Slepian-Wolf (SW) coding. We prove that the proposed framework can achieve the theoretically minimum communication cost from an information theory perspective. We also propose a low-complexity and adaptive DSC for distributed learning, including a gradient statistics estimator, rate controller, and a log-likelihood ratio (LLR) computer. The gradient statistics estimator estimates the gradient statistics online based only on the quantized gradients at previous iterations, hence it does not introduce extra communication cost. The computation complexity of the rate controller and the LLR computer is reduced to a linear growth in the number of worker nodes by introducing a semi-analytical Monte Carlo simulation. Finally, we design a DSC-based distributed learning process and find that the extra delay introduced by DSC does not scale with the number of worker nodes.

Source Symbol Purging-Based Distributed Conditional Arithmetic Coding.

A distributed arithmetic coding algorithm based on source symbol purging and using the context model is proposed to solve the asymmetric Slepian–Wolf problem. The proposed scheme is to make better use of both the correlation between adjacent symbols in the source sequence and the correlation between the corresponding symbols of the source and the side information sequences to improve the coding performance of the source. Since the encoder purges a part of symbols from the source sequence, a shorter codeword length can be obtained. Those purged symbols are still used as the context of the subsequent symbols to be encoded. An improved calculation method for the posterior probability is also proposed based on the purging feature, such that the decoder can utilize the correlation within the source sequence to improve the decoding performance. In addition, this scheme achieves better error performance at the decoder by adding a forbidden symbol in the encoding process. The simulation results show that the encoding complexity and the minimum code rate required for lossless decoding are lower than that of the traditional distributed arithmetic coding. When the internal correlation strength of the source is strong, compared with other DSC schemes, the proposed scheme exhibits a better decoding performance under the same code rate.

Trade-offs between Error Exponents and Excess-Rate Exponents of Typical Slepian-Wolf Codes.

Typical random codes (TRCs) in a communication scenario of source coding with side information in the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one-to-one manner. As a consequence, the error probability decreases dramatically. The random binning error exponent and the error exponent of the TRCs are derived and proved to be equal to one another in a few important special cases. We show that the performance under optimal decoding can be attained also by certain universal decoders, e.g., the stochastic likelihood decoder with an empirical entropy metric. Moreover, we discuss the trade-offs between the error exponent and the excess-rate exponent for the typical random semi-deterministic code and characterize its optimal rate function. We show that for any pair of correlated information sources, both error and excess-rate probabilities exponential vanish when the blocklength tends to infinity.

Universal Decoding for Asynchronous Slepian-Wolf Encoding

We consider the problem of (almost) lossless source coding of two correlated memoryless sources using separate encoders and a joint decoder, that is, Slepian-Wolf (S-W) coding. In our setting, the encoding and decoding are asynchronous, i.e., there is a certain relative delay between the two sources. Neither the source parameters nor the relative delay are known to the encoders and the decoder. Since we assume that both encoders implement standard random binning, which does not require such knowledge anyway, the focus of this work is on the decoder. Our main contribution is in proposing a universal decoder, that independent of the unknown source parameters and the relative delay, and at the same time, is asymptotically as good as the optimal maximum a posteriori probability (MAP) decoder in the sense of the random coding error exponent achieved. Consequently, the achievable rate region is also the same as if the source parameters and the delay were known to the decoder.

A general achievable rate region and two certain capacity regions for Slepian–Wolf multiple‐access relay channel with non‐causal side information at one encoder

A general achievable rate region and two certain capacity regions for Slepian–Wolf multiple‐access relay channel with non‐causal side information at one encoder

Non-Asymptotic Classical Data Compression With Quantum Side Information

In this paper, we analyze classical data compression with quantum side information (also known as the classical-quantum Slepian–Wolf protocol) in the so-called large and moderate deviation regimes. In the non-asymptotic setting, the protocol involves compressing classical sequences of finite length <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> and decoding them with the assistance of quantum side information. In the large deviation regime, the compression rate is fixed, and we obtain bounds on the error exponent function, which characterizes the minimal probability of error as a function of the rate. Devetak and Winter showed that the asymptotic data compression limit for this protocol is given by a conditional entropy. For any protocol with a rate below this quantity, the probability of error converges to one asymptotically and its speed of convergence is given by the strong converse exponent function. We obtain finite blocklength bounds on this function, and determine exactly its asymptotic value. In the moderate deviation regime for the compression rate, the latter is no longer considered to be fixed. It is allowed to depend on the blocklength <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , but assumed to decay slowly to the asymptotic data compression limit. Starting from a rate above this limit, we determine the speed of convergence of the error probability to zero and show that it is given in terms of the conditional information variance.

Outage Probability of One-Source-With-One- Helper Sensor Systems in Block Rayleigh Fading Multiple Access Channels

Two correlated sources are transmitted over a block Rayleigh fading multiple access channel (MAC), where information recovery from only one of the sources is aimed at, and the other is used as a helper which helps recover the source sequences at the destination. In this article, the successful transmission is defined as in the case the transmission rates satisfy the intersection of both the MAC region and the Slepian-Wolf (SW) region with a helper, referred to as h-SW region, even though it is a sufficient condition. Then, the outage is defined as the average probability that the two regions do not have an intersection. The explicit expressions of the outage probability of the system are derived, in the form of multiple integrals with respect to the probability density function of instantaneous signal-to-noise ratios of each link. To have an in-depth insight regarding the contributions of some special cases to the successful transmission probability, the occurrence probabilities of these cases are also evaluated. The results indicate that outage probabilities decrease as the correlation between the source and helper information increases. The impacts of geometric gains of each link and the correlation of each link's fading variation are also considered. The most significant finding of this article is that h-SW over a MAC improves the throughput efficiency compared to the orthogonal transmission both in independent and correlated block Rayleigh fading channels without sacrificing the outage probability.

Symbol positions‐based Slepian–Wolf coding with application to distributed video coding

In this study, the authors will show that coding the positions of the symbols, instead of their values, can be a good way to implement efficient Slepian–Wolf (SW) coding and can reduce the complexity of both the encoder and the decoder. The authors will also propose a practical distributed video coding (DVC) system that exploits this idea. This system will use binary maps to indicate the positions of the most probable symbols, instead of separating them into bitplanes. Simulations show that this position-based SW coding allows a simple and more efficient DVC system with improved rate-distortion performance, compared to the bitplane-based DVC system that uses the same side information. The memory requirements at the encoder are reduced by about 50% and the number of channel coding operations is also reduced. This DVC system also allows to use an easy way to control quantisation, reduces the decoding latency, and allows fast parallel decoding.

Coding Theorems for Asynchronous Slepian–Wolf Coding Systems

The Slepian-Wolf (SW) coding system is a source coding system with two encoders and a decoder, where these encoders independently encode source sequences from two correlated sources into codewords, and the decoder reconstructs both source sequences from the codewords. In this paper, we consider the situation in which the SW coding system is asynchronous, i.e., each encoder samples a source sequence with some unknown delay. We assume that delays are unknown but maximum and minimum values of possible delays are known to encoders and the decoder. We also assume that sources are discrete stationary memoryless and the probability mass function (PMF) of the sources is unknown but the system knows that it belongs to a certain set of PMFs. For this asynchronous SW coding system, we clarify the achievable rate region which is the set of rate pairs of encoders such that the decoding error probability vanishes as the blocklength tends to infinity. We show that this region does not always coincide with that of the synchronous SW coding system in which each encoder samples a source sequence without any delay.

Improved Finite Blocklength Converses for Slepian–Wolf Coding via Linear Programming

A new finite blocklength converse for the Slepian–Wolf coding problem, which significantly improves on the best-known converse due to Miyake and Kanaya, is presented. To obtain this converse, an extension of the linear programming (LP)-based framework for finite blocklength point-to-point coding problems is employed. However, a direct application of this framework demands a complicated analysis for the Slepian–Wolf problem. An analytically simpler approach is presented, wherein LP-based finite blocklength converses for this problem are synthesized from point-to-point lossless source coding problems with perfect side-information at the decoder. New finite blocklength converses for these point-to-point problems are derived by employing the LP-based framework, and the new converse for Slepian–Wolf coding is obtained by an appropriate combination of these converses.

Ensemble Performance of Biometric Authentication Systems Based on Secret Key Generation

We study the ensemble performance of biometric authentication systems, based on secret key generation, which work as follows. In the enrollment stage, an individual provides a biometric signal that is mapped into a secret key and a helper message, the former being prepared to become available to the system at a later time (for authentication), and the latter is stored in a public database. When an authorized user requests authentication, claiming his/her identity as one of the subscribers, he/she has to provide a biometric signal again, and then the system, which retrieves also the helper message of the claimed subscriber, produces an estimate of the secret key, that is finally compared with the secret key of the claimed user. In case of a match, the authentication request is approved, otherwise, it is rejected. Referring to an ensemble of systems based on Slepian–Wolf binning, we provide a detailed analysis of the false–reject (FR) and false–accept (FA) probabilities, for a wide class of stochastic decoders. We also derive converse bounds. The converse bound of the FA probability matches the direct theorem, whereas the one for the FR probability is tight for some ranges of rates. Finally, we outline derivations of the secrecy leakage (for the typical code in the ensemble) and the privacy leakage.

Guessing with Distributed Encoders.

Two correlated sources emit a pair of sequences, each of which is observed by a different encoder. Each encoder produces a rate-limited description of the sequence it observes, and the two descriptions are presented to a guessing device that repeatedly produces sequence pairs until correct. The number of guesses until correct is random, and it is required that it have a moment (of some prespecified order) that tends to one as the length of the sequences tends to infinity. The description rate pairs that allow this are characterized in terms of the Rényi entropy and the Arimoto–Rényi conditional entropy of the joint law of the sources. This solves the guessing analog of the Slepian–Wolf distributed source-coding problem. The achievability is based on random binning, which is analyzed using a technique by Rosenthal.

Generalized Compute-Compress-and-Forward

Compute-and-forward (CF) harnesses interference in wireless communications by exploiting structured coding. The key idea of CF is to compute integer combinations of code words from multiple source nodes, rather than to decode individual code words by treating others as noise. Compute-compress-and-forward (CCF) can further enhance the network performance by introducing compression operations at receivers. In this paper, we develop a more general compression framework, termed generalized CCF (GCCF), where the compression function involves the selection of message segments over finite fields. We show that GCCF achieves a broader compression rate region than CCF. We also compare our compression rate region with the fundamental Slepian–Wolf (SW) region. We show that GCCF is optimal in the sense of achieving the minimum total compression rate. We also establish the criteria under which GCCF achieves the SW region. In addition, we consider a two-hop relay network employing the GCCF scheme. We formulate a sum-rate maximization problem and develop an approximate algorithm to solve the problem. Numerical results are presented to demonstrate the performance superiority of GCCF over CCF and other schemes.

On Multichannel Random Access for Correlated Sources

In this paper, we consider multichannel ALOHA for a wireless sensor network (WSN), where sensors’ signals are correlated. The impact of the number of channels on the length of information that an access point (AP) can collect is investigated. Under a certain distributed signal source model, in order for the AP to have a sufficiently long length of information, we show that the number of channels is to be larger than the square of the average number of active nodes. Thus, when the number of channels is limited, we need to use re-transmissions. To this end, we propose an adaptive transmission approach that can effectively exploit the correlation between the collected signals at the AP and nodes’ signals using Slepian–Wolf coding to shorten the total transmission time, which results in a more energy efficient WSN.