Deterministic Encoding Research Articles

Deterministic K-Identification for Future Communication Networks: The Binary Symmetric Channel Results

Numerous applications of the Internet of Things (IoT) feature an event recognition behavior where the established Shannon capacity is not authorized to be the central performance measure. Instead, the identification capacity for such systems is considered to be an alternative metric, and has been developed in the literature. In this paper, we develop deterministic K-identification (DKI) for the binary symmetric channel (BSC) with and without a Hamming weight constraint imposed on the codewords. This channel may be of use for IoT in the context of smart system technologies, where sophisticated communication models can be reduced to a BSC for the aim of studying basic information theoretical properties. We derive inner and outer bounds on the DKI capacity of the BSC when the size of the goal message set K may grow in the codeword length n. As a major observation, we find that, for deterministic encoding, assuming that K grows exponentially in n, i.e., K=2nκ, where κ is the identification goal rate, then the number of messages that can be accurately identified grows exponentially in n, i.e., 2nR, where R is the DKI coding rate. Furthermore, the established inner and outer bound regions reflects impact of the input constraint (Hamming weight) and the channel statistics, i.e., the cross-over probability.

Quadratically Constrained Myopic Adversarial Channels

We study communication in the presence of a jamming adversary where quadratic power constraints are imposed on the transmitter and the jammer. The jamming signal is allowed to be a function of the codebook, and a noncausal but noisy observation of the transmitted codeword. For a certain range of the noise-to-signal ratios (NSRs) of the transmitter and the jammer, we are able to characterize the capacity of this channel under deterministic encoding or stochastic encoding, i.e., with no common randomness between the encoder/decoder pair. For the remaining NSR regimes, we determine the capacity under the assumption of a small amount of common randomness (at most <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\log (n)$ </tex-math></inline-formula> bits in one sub-regime, and at most <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Omega ({n})$ </tex-math></inline-formula> bits in the other sub-regime) available to the encoder-decoder pair. Our proof techniques involve a novel myopic list-decoding result for achievability, and a Plotkin-type push attack for the converse in a subregion of the NSRs, both of which may be of independent interest. We also give bounds on the strong secrecy capacity of this channel assuming that the jammer is simultaneously eavesdropping.

Codes for Correcting Localized Deletions

We consider the problem of constructing binary codes for correcting deletions that are localized within certain parts of the codeword that are unknown a priori. The model that we study is when δ ≤ w deletions are localized in a window of size w bits. These δ deletions do not necessarily occur in consecutive positions, but are restricted to the window of size w. The localized deletions model is a generalization of the bursty model, in which all the deleted bits are consecutive. In this paper, we construct new explicit codes for the localized model, based on the family of Guess & Check codes which was previously introduced by the authors. The codes that we construct can correct, with high probability, δ ≤ w deletions that are localized in a single window of size w, where w grows with the block length. Moreover, these codes are systematic; have low redundancy; and have efficient deterministic encoding and decoding algorithms. We also generalize these codes to deletions that are localized within multiple windows in the codeword.

Deterministic Shuffling Networks to Implement Stochastic Circuits in Parallel

Stochastic computing (SC) in recent years has been defined as a digital computation approach that operates on streams of random bits that represent probability values. SC can perform complex tasks with much smaller hardware footprints compared with conventional binary methods, but previous methods on SC circuits operated on serial bit streams, which leads to high-latency implementations. This article presents a significant improvement over previous work; it provides a deterministic parallel bit shuffling network that can use a simple deterministic thermometer encoding of data, resulting in zero random fluctuation and high accuracy, yet keeping the output bit-stream length constant. We use core “stochastic” logic circuits that do not employ constant coefficients, making them significantly smaller than traditional stochastic logic that use a significant amount of resources to generate such coefficients. Our experiments show that compared with previous SC methods, our method has up to $3\times $ smaller mean absolute error, and better area $\times $ delay and power efficiency. Compared with conventional binary methods, our method is better in terms of area $\times $ delay at 8-bit resolution. It shows better power efficiency ( $40\times $ , $18\times $ , and $8\times $ Gops/W at 8-, 10-, and 12-bit resolutions) compared with conventional binary.

Estimation Theoretic Optimal Encoding Design for Secure Transmission of Multiple Parameters

In this paper, optimal deterministic encoding of a vector parameter is investigated in the presence of an eavesdropper. The objective is to minimize the expectation of the conditional Cramer–Rao bound at the intended receiver, while satisfying an individual secrecy constraint on the mean-squared error of estimating each parameter at the eavesdropper. The eavesdropper is modeled to employ the linear minimum mean-squared error estimator based on the noisy observation of the encoded parameter without being aware of encoding. First, the problem is formulated as a constrained optimization problem in the space of vector-valued functions. Then, two practical solution strategies are developed based on nonlinear individual encoding and affine joint encoding of parameters. Theoretical results on the solutions of the proposed strategies are provided for various scenarios on channel conditions and parameter distributions. Finally, numerical examples are presented to illustrate the performance of the proposed solution approaches.

Minimal Characterization of Shannon-Type Inequalities Under Functional Dependence and Full Conditional Independence Structures

The minimal set of Shannon-type inequalities (or elemental inequalities) plays a central role in efficiently determining whether a given inequality is in fact Shannon-type or not and in computing the linear programming bound for network coding capacity. In many cases, random variables under consideration are subject to additional constraints, such as functional dependence and conditional independence constraints. For example, functional dependence constraints are common in many communication problems due to deterministic encoding and decoding constraints. In other situations, the variables involved may form a Markov chain or in general a Markov random field, leading to conditional independence constraint. Subject to additional constraints, the challenge is how to identify the non-redundant inequalities. While one can always numerically determine the non-redundant inequalities (subject to additional linear equality constraints), it will be instrumental and also important if the non-redundant inequalities can be listed explicitly. In this paper, we show that this is achievable under the functional dependence and full conditional independence constraints.

Ratio List Decoding

We extend the notion of list decoding to ratio list decoding which involves a list decoder whose list size is specified as a function of the number of messages $M_{n}$ and the block length $n$ . We present the necessary and sufficient conditions on $M_{n}$ for the existence of code sequences which enable reliable list decoding with respect to the desired list size $L(M_{n},n)$ . It is shown that the ratio-capacity, defined as the supremum of achievable normalized logarithms of the ratio $r(M_{n},n)=M_{n}/L(M_{n},n)$ , is equal to the Shannon channel capacity $C$ , for both stochastic and deterministic encoding. Allowing for random list size, we are able to deduce some properties of identification codes, where the decoder’s output can be viewed as a list of messages corresponding to decision regions that include the channel output. We further address the regime of mismatched list decoding, in which the list constitutes of the codewords that accumulate the highest score values (jointly with the channel output) according to some given function. We study the case of deterministic encoding and mismatched ratio list decoding. We establish similar necessary and sufficient conditions for the existence of code sequences which enable reliable mismatched list decoding with respect to the desired list size $L(M_{n},n)$ , and we show that the ratio-capacity with mismatched decoding is equal to the mismatch capacity. Focusing on the case of an exponential list size $L_{n}=e^{n \Theta }$ , its comparison with ordinary mismatched decoding shows that the increase in capacity is by $\Theta $ bits per channel used for all channels and decoding metrics. Several properties of the average error probability in the setup of mismatched list decoding with deterministic list size are provided.

Collective Secrecy Over the &lt;inline-formula&gt; &lt;tex-math notation="LaTeX"&gt;$K$ &lt;/tex-math&gt; &lt;/inline-formula&gt;-Transmitter Multiple Access Channel

This paper studies the problem of secure communication over a $K$ -transmitter multiple access channel (MAC) in the presence of an external eavesdropper, subject to a collective secrecy constraint (i.e., information leakage rate to an eavesdropper on a collection of messages that are from a pre-specified subset of the $K$ transmitters, say $\mathcal {S}\subseteq \mathcal {K}=\{1,2,\ldots, K\}$ , is made vanishing). Since secrecy is of concern only to transmitters $\{i|i\in \mathcal {S}\}$ but not to transmitters $\{i|i\in \mathcal {S}^{c}\}$ , where $\mathcal {S}^{c}=\mathcal {K}\backslash \mathcal {S}$ , different transmission strategies could be employed at transmitters $\{i|i\in \mathcal {S}^{c}\}$ . Consider the following two scenarios: 1) transmitters $\{i|i\in \mathcal {S}^{c}\}$ use deterministic encoders (which are conventionally used for MAC without secrecy), competing for the channel resource (i.e., being competitive) and 2) transmitters $\{i|i\in \mathcal {S}^{c}\}$ use stochastic encoders, helping to hide other transmitters’ messages from the eavesdropper (i.e., being cooperative). As a result, we establish the respective $\mathcal {S}$ -collective secrecy achievable rate regions and demonstrate the advantage of being cooperative theoretically and numerically. To this end, in addition to the standard techniques, our results build upon two techniques. The first is a generalization of Chia-El Gamal’s lemma on entropy bound for a set of codewords given partial information. The second is to utilize a compact representation of a list of sets that, together with submodular properties of mutual information functions involved, leads to an efficient Fourier-Motzkin elimination. These two approaches allow us to derive achievable regions in this work, and could also be of independent interest in other context.

ECRB-Based Optimal Parameter Encoding Under Secrecy Constraints

In this paper, optimal deterministic encoding of a scalar parameter is investigated in the presence of an eavesdropper. The aim is to minimize the expectation of the conditional Cramer–Rao bound at the intended receiver while keeping the mean-squared error (MSE) at the eavesdropper above a certain threshold. First, optimal encoding functions are derived in the absence of secrecy constraints for any given prior distribution on the parameter. Next, an optimization problem is formulated under a secrecy constraint and various solution approaches are proposed. Also, theoretical results on the form of the optimal encoding function are provided under the assumption that the eavesdropper employs a linear minimum mean-squared error (MMSE) estimator. Numerical examples are presented to illustrate the theoretical results and to investigate the performance of the proposed solution approaches.

Optimal Parameter Encoding Based on Worst Case Fisher Information Under a Secrecy Constraint

In this letter, optimal deterministic encoding of a uniformly distributed scalar parameter is performed in the presence of an eavesdropper. The objective is to maximize the worst case Fisher information of the parameter at the intended receiver while keeping the mean-squared error (MSE) at the eavesdropper above a certain level. The eavesdropper is modeled to employ the linear minimum MSE estimator based on the encoded version of the parameter. First, the optimal encoding function is derived when there exist no secrecy constraints. Next, to obtain the solution of the problem in the presence of the secrecy constraint, the form of the encoding function that maximizes the MSE at the eavesdropper is explicitly derived for any given level of worst case Fisher information. Then, based on this result, a low-complexity algorithm is provided to calculate the optimal encoding function for the given secrecy constraint. Finally, numerical examples are presented.

Identification via the Broadcast Channel

The identification (ID) capacity region of the two-receiver broadcast channel (BC) is shown to be the set of rate-pairs for which, for some distribution on the channel input, each receiver's ID rate does not exceed the mutual information between the channel input and the channel output that it observes. Moreover, the capacity region's interior is achieved by codes with deterministic encoders. The results are obtained under the average-error criterion, which requires that each receiver reliably identify its message whenever the message intended for the other receiver is drawn at random. They hold also for channels whose transmission capacity region is to-date unknown. Key to the proof is a new ID code construction for the single-user channel. Extensions to the BC with one-sided feedback and the three-receiver BC are also discussed: inner bounds on their ID capacity regions are obtained, and those are shown to be in some cases tight.

On the Combinatorial Version of the Slepian–Wolf Problem

We study the following combinatorial version of the Slepian–Wolf coding scheme. Two isolated Senders are given binary strings $X$ and $Y$ , respectively; the length of each string is equal to $n$ , and the Hamming distance between the strings is at most $\alpha n$ . The Senders compress their strings and communicate the results to the Receiver. Then, the Receiver must reconstruct both the strings $X$ and $Y$ . The aim is to minimize the lengths of the transmitted messages. For an asymmetric variant of this problem (where one of the Senders transmits the input string to the Receiver without compression) with deterministic encoding, a nontrivial bound was found by Orlitsky and Viswanathany. In this paper, we prove a new lower bound for the schemes with syndrome coding, where at least one of the Senders uses linear encoding of the input string. For the combinatorial Slepian–Wolf problem with randomized encoding, the theoretical optimum of communication complexity was known earlier, even though effective protocols with optimal lengths of messages remained unknown. We close this gap and present a polynomial-time-randomized protocol that achieves the optimal communication complexity.

Tracking photon jumps with repeated quantum non-demolition parity measurements.

Quantum error correction is required for a practical quantum computer because of the fragile nature of quantum information. In quantum error correction, information is redundantly stored in a large quantum state space and one or more observables must be monitored to reveal the occurrence of an error, without disturbing the information encoded in an unknown quantum state. Such observables, typically multi-quantum-bit parities, must correspond to a special symmetry property inherent in the encoding scheme. Measurements of these observables, or error syndromes, must also be performed in a quantum non-demolition way (projecting without further perturbing the state) and more quickly than errors occur. Previously, quantum non-demolition measurements of quantum jumps between states of well-defined energy have been performed in systems such as trapped ions, electrons, cavity quantum electrodynamics, nitrogen-vacancy centres and superconducting quantum bits. So far, however, no fast and repeated monitoring of an error syndrome has been achieved. Here we track the quantum jumps of a possible error syndrome, namely the photon number parity of a microwave cavity, by mapping this property onto an ancilla quantum bit, whose only role is to facilitate quantum state manipulation and measurement. This quantity is just the error syndrome required in a recently proposed scheme for a hardware-efficient protected quantum memory using Schrödinger cat states (quantum superpositions of different coherent states of light) in a harmonic oscillator. We demonstrate the projective nature of this measurement onto a region of state space with well-defined parity by observing the collapse of a coherent state onto even or odd cat states. The measurement is fast compared with the cavity lifetime, has a high single-shot fidelity and has a 99.8 per cent probability per single measurement of leaving the parity unchanged. In combination with the deterministic encoding of quantum information in cat states realized earlier, the quantum non-demolition parity tracking that we demonstrate represents an important step towards implementing an active system that extends the lifetime of a quantum bit.

Indifferentiable deterministic hashing to elliptic and hyperelliptic curves

At Crypto 2010, Brier et al. proposed the first construction of a hash function into ordinary elliptic curves that was indifferentiable from a random oracle, based on Icart’s deterministic encoding from Crypto 2009. Such a hash function can be plugged into any cryptosystem that requires hashing into elliptic curves, while not compromising proofs of security in the random oracle model. However, the proof relied on relatively involved tools from algebraic geometry, and only applied to Icart’s deterministic encoding from Crypto 2009. In this paper, we present a new, simpler technique based on bounds of character sums to prove the indifferentiability of similar hash function constructions based on essentially any deterministic encoding to elliptic curves or curves of higher genus, such as the algorithms by Shallue, van de Woestijne and Ulas, or the Icart-like encodings recently presented by Kammerer, Lercier and Renault. In particular, we get the first constructions of well-behaved hash functions to Jacobians of hyperelliptic curves. Our technique also provides more precise estimates on the statistical behavior of those deterministic encodings and the hash function constructions based on them. Additionally, we can derive pseudorandomness results for partial bit patterns of such encodings.

On the expressiveness and decidability of higher-order process calculi

In higher-order process calculi, the values exchanged in communications may contain processes. A core calculus of higher-order concurrency is studied; it has only the operators necessary to express higher-order communications: input prefix, process output, and parallel composition. By exhibiting a deterministic encoding of Minsky machines, the calculus is shown to be Turing complete. Therefore its termination problem is undecidable. Strong bisimilarity, however, is shown to be decidable. Furthermore, the main forms of strong bisimilarity for higher-order processes (higher-order bisimilarity, context bisimilarity, normal bisimilarity, barbed congruence) coincide. They also coincide with their asynchronous versions. A sound and complete axiomatization of bisimilarity is given. Finally, bisimilarity is shown to become undecidable if at least four static (i.e., top-level) restrictions are added to the calculus.

Codes for Deletion and Insertion Channels With Segmented Errors

We consider deletion channels and insertion channels under an additional segmentation assumption: the input consists of disjoint segments of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</i> consecutive bits, with at most one error per segment. Under this assumption, we demonstrate simple and computationally efficient deterministic encoding and decoding schemes that achieve a high provable rate even under worst case errors. We also consider more complex schemes that experimentally achieve higher rates under random error.

An Analysis on Optimal Rates of BDB Encoders for RLL and MTR Systems

In most recording channels, modulation codes are employed to transform user data to sequences that satisfy some desirable constraint. Run-length-limited (RLL(d,k)) and maximum transition run (MTR(j,k)) systems are examples of constraints that improve timing and detection performance. A modulation encoder typically takes the form of a finite-state machine. Alternatively, a look-ahead encoder can be used instead of a finite-state encoder to reduce complexity. Its encoding process involves a delay called look-ahead. If the input labeling of a look-ahead encoder allows block decodability, the encoder is called a bounded-delay-encodable block-decodable (BDB) encoder. These classes of encoders can be viewed as generalizations of the well-known deterministic and block-decodable encoders. Other related classes are finite-anticipation and sliding-block decodable encoders. In this paper, we clarify the relationship among these encoders. We also discuss the characterization of look-ahead and BDB encoders using the concept of path-classes. To minimize encoder complexity, look-ahead is desired to be small. We show that for nonreturn to zero inverted (NRZI) versions of RLL|,(0,k),RLL(1,k), and RLL(d,infin), a BDB encoder does not yield a higher rate than an optimal block-decodable encoder. However, for RLL(d,k) such that dges4 and d+2lesk<infin, we present a BDB encoder with look-ahead one that has a higher rate than any block-decodable encoder. For MTR, we prove that no BDB encoder is asymptotically better than an optimal BDB encoder with look-ahead one

Optimal block-type-decodable encoders for constrained systems

A constrained system is presented by a finite-state labeled graph. For such systems, we focus on block-type-decodable encoders, comprising three classes known as block, block-decodable, and deterministic encoders. Franaszek (1968) gives a sufficient condition which guarantees the equality of the optimal rates of block-decodable and deterministic encoders for the same block length. We introduce another sufficient condition, called the straight-line condition, which yields the same result. Run-length limited RLL(d,k) and maximum transition run MTR(j,k) constraints are shown to satisfy both conditions. In general, block-type-decodable encoders are constructed by choosing a subset of states of the graph to be used as encoder states. Such a subset is known as a set of principal states. For each type of encoder and each block length, a natural problem is to find a set of principal states which maximizes the code rate. We show how to compute the asymptotically optimal sets of principal states for deterministic encoders and how they are related to the case of large but finite block lengths. We give optimal sets of principal states for MTR(j,k)-block-type-decodable encoders for all codeword lengths. Finally we compare the code rate of nonreturn to zero inverted (NRZI) encoders to that of corresponding nonreturn to zero (NRZ) and signed NRZI encoders.

Identification plus transmission over channels with perfect feedback

We determine the region of all identification and transmission rate-pairs achievable over a discrete memoryless channel (DMC) with perfect and instantaneous feedback, for both randomized and deterministic encoding. As a by-product, we also have a new proof of Kemperman's (1973) strong converse to Shannon's coding theorem for DMC's with feedback.

A method for proving multiterminal source coding theorems

The two-step method used by Wyner and Ziv to prove the Wyner-Ziv theorem is extended to prove sliding-block source coding theorems for coding a general finite-alphabet ergodic multiterminal source with respect to a single-letter fidelity criterion. The first step replaces every stochastic encoding in the network by a deterministic sliding-block encoding. The second step involves using the Slepian-Wolf theorem to adjust the sliding-block encodings so that they will have the desired rate. while introducing little additional distortion. The method is applied to give quick proofs of sliding-block versions of theorems of Berger, Kaspi, and Tung. Since the method obtains sliding-block coders directly without first obtaining block coders, the block coding results can be obtained as an easy corollary. The direct methods for proving results on block coding are more difficult and do not imply the corresponding results on sliding-block coding. Indeed, in the multiterminal case it is, in general, not known how to construct good sliding-block coders from good block coders.