Random Codebook Research Articles

ORBGRAND Is Almost Capacity-Achieving

Decoding via sequentially guessing the error pattern in a received noisy sequence has received attention recently, and ORBGRAND has been proposed as one such decoding algorithm that is capable of utilizing the soft information embedded in the received noisy sequence. An information theoretic study is conducted for ORBGRAND, and it is shown that the achievable rate of ORBGRAND using independent and identically distributed random codebooks almost coincides with the channel capacity, for an additive white Gaussian noise channel under antipodal input. For finite-length codes, improved guessing schemes motivated by the information theoretic study are proposed that attain lower error rates than ORBGRAND, especially in the high signal-to-noise ratio regime.

A Shannon-Theoretic Approach to the Storage–Retrieval Trade-Off in PIR Systems

We consider the storage–retrieval rate trade-off in private information retrieval (PIR) systems using a Shannon-theoretic approach. Our focus is mostly on the canonical two-message two-database case, for which a coding scheme based on random codebook generation and the binning technique is proposed. This coding scheme reveals a hidden connection between PIR and the classic multiple description source coding problem. We first show that when the retrieval rate is kept optimal, the proposed non-linear scheme can achieve better performance over any linear scheme. Moreover, a non-trivial storage-retrieval rate trade-off can be achieved beyond space-sharing between this extreme point and the other optimal extreme point, achieved by the retrieve-everything strategy. We further show that with a method akin to the expurgation technique, one can extract a zero-error PIR code from the random code. Outer bounds are also studied and compared to establish the superiority of the non-linear codes over linear codes.

Guessing Random Additive Noise Decoding With Symbol Reliability Information (SRGRAND)

The design and implementation of error correcting codes has long been informed by two fundamental results: Shannon’s 1948 capacity theorem, which established that long codes use noisy channels most efficiently; and Berlekamp, McEliece, and Van Tilborg’s 1978 theorem on the NP-completeness of decoding linear codes. These results shifted focus away from creating code-independent decoders, but recent low-latency communication applications necessitate relatively short codes, providing motivation to reconsider the development of universal decoders. We introduce a scheme for employing binarized symbol soft information within Guessing Random Additive Noise Decoding, a universal hard detection decoder. We incorporate codebook-independent quantization of soft information to indicate demodulated symbols to be reliable or unreliable. We introduce two decoding algorithms: one identifies a conditional Maximum Likelihood (ML) decoding; the other either reports a conditional ML decoding or an error. For random codebooks, we present error exponents and asymptotic complexity, and show benefits over hard detection. As empirical illustrations, we compare performance with majority logic decoding of Reed-Muller codes, with Berlekamp-Massey decoding of Bose-Chaudhuri-Hocquenghem codes, with CA-SCL decoding of CA-Polar codes, and establish the performance of Random Linear Codes, which require a universal decoder and offer a broader palette of code sizes and rates than traditional codes.

Full key dependent coverless test disguise method by interval-extension-based double authentications

In traditional search-based coverless information hiding methods, there are some problems such as the low hidden capacity, high cost of database creation and search, dense transmission of data, and absence of authentication. To avoid these problems, we propose a full key dependent coverless test disguise method by interval-extension-based double authentications. We transform secret information into a coded sequence by a random codebook sequence that is generated by user keys. The initialized codebook sequence is used to hinder the direct transmission of secret information. Then, we use candidate answers’ orders and question set indexes to express the coded index sequence. In the restoration process, the extracted information can be authenticated efficiently by the legal interval and a comparison. The experimental results and analysis show that our method achieves a high hidden capacity with a simpler test database, avoids dense transmission of data and authenticates extracted information well. Without the correct user keys, the secret information cannot be acquired.

The Perils and Pitfalls of Block Design for EEG Classification Experiments.

A recent paper [31] claims to classify brain processing evoked in subjects watching ImageNet stimuli as measured with EEG and to employ a representation derived from this processing to construct a novel object classifier. That paper, together with a series of subsequent papers [11, 18, 20, 24, 25, 30, 34], claims to achieve successful results on a wide variety of computer-vision tasks, including object classification, transfer learning, and generation of images depicting human perception and thought using brain-derived representations measured through EEG. Our novel experiments and analyses demonstrate that their results crucially depend on the block design that they employ, where all stimuli of a given class are presented together, and fail with a rapid-event design, where stimuli of different classes are randomly intermixed. The block design leads to classification of arbitrary brain states based on block-level temporal correlations that are known to exist in all EEG data, rather than stimulus-related activity. Because every trial in their test sets comes from the same block as many trials in the corresponding training sets, their block design thus leads to classifying arbitrary temporal artifacts of the data instead of stimulus-related activity. This invalidates all subsequent analyses performed on this data in multiple published papers and calls into question all of the reported results. We further show that a novel object classifier constructed with a random codebook performs as well as or better than a novel object classifier constructed with the representation extracted from EEG data, suggesting that the performance of their classifier constructed with a representation extracted from EEG data does not benefit from the brain-derived representation. Together, our results illustrate the far-reaching implications of the temporal autocorrelations that exist in all neuroimaging data for classification experiments. Further, our results calibrate the underlying difficulty of the tasks involved and caution against overly optimistic, but incorrect, claims to the contrary.

Capacity-Achieving Guessing Random Additive Noise Decoding

We introduce a new algorithm for realizing Maximum Likelihood (ML) decoding in discrete channels with or without memory. In it, the receiver rank orders noise sequences from most likely to least likely. Subtracting noise from the received signal in that order, the first instance that results in a member of the code-book is the ML decoding. We name this algorithm GRAND for Guessing Random Additive Noise Decoding. We establish that GRAND is capacity-achieving when used with random code-books. For rates below capacity we identify error exponents, and for rates beyond capacity we identify success exponents. We determine the scheme's complexity in terms of the number of computations the receiver performs. For rates beyond capacity, this reveals thresholds for the number of guesses by which if a member of the code-book is identified it is likely to be the transmitted code-word. We introduce an approximate ML decoding scheme where the receiver abandons the search after a fixed number of queries, an approach we dub GRANDAB, for GRAND with ABandonment. While not an ML decoder, we establish that the algorithm GRANDAB is also capacity-achieving for an appropriate choice of abandonment threshold, and characterize its complexity, error and success exponents. Worked examples are presented for Markovian noise that indicate these decoding schemes substantially out-perform the brute force decoding approach.

New fast multi‐user beam training scheme based on compressed sensing theory for millimetre‐wave communication

Compensating for the considerable propagation loss, millimetre-wave communication systems adopt large antenna arrays and beamforming technologies to obtain high antenna gains. The beamforming training is rather time-consuming especially in multi-user cases. To reduce the training overhead, a new fast multi-user training scheme based on Compressed Sensing theory is proposed. This scheme includes a multi-user training algorithm and a corresponding semi-fixed semi-random codebook design strategy. The base station (BS) simultaneously broadcasts the training pilots to all user equipments (UEs). Then all UEs independently conduct the proposed training algorithm from channel responses. Finally, all UEs send back the training results to the BS. Compared to the exhaustive search algorithm's training overhead ξ = K r K t , the cost of the proposed scheme is only O ( ξ ) = log ⁡ ( K r K t ) , where K r and K t denote the number of beam patterns in the receiver and the transmitter. Simulation results show that the proposed scheme achieves excellent performance in high SNR (signal-to-noise ratio) cases with only 3 − 5 % performance difference from the theoretical bound. And in contrast with other Compressed Sensing schemes with random codebooks, the proposed scheme reduces the sensitivity to transmission SNR and improves approximately 3 dB performance in low SNR cases.

On Random-Coding Union Bounds With and Without Erasures

Upper bounds on the error probability of channel coding are derived for codebooks drawn from random codebook ensembles with various independence and symmetry assumptions. For regular decoding (without an erasure option), the random coding union bound of Polyanskiy et al. is improved by carefully taking ties (equal likelihood scores) into account. It is shown that the improved bound is always better than threshold-decoding based bounds. The framework is extended to the case of decoding with an erasure option, deriving several achievability bounds in the same spirit. In order to exemplify the merits of the approach, the bounds are evaluated for the case of pairwise-independent uniformly-distributed ensembles (e.g., shifted random linear codes).

Semantic-Security Capacity for Wiretap Channels of Type II

The secrecy capacity of the type II wiretap channel (WTC II) with a noisy main channel is currently an open problem. Herein its secrecy-capacity is derived and shown to be equal to its semantic-security (SS) capacity. In this setting, the legitimate users communicate via a discrete-memoryless (DM) channel in the presence of an eavesdropper that has perfect access to a subset of its choosing of the transmitted symbols, constrained to a fixed fraction of the blocklength. The secrecy criterion is achieved simultaneously for all possible eavesdropper subset choices. The SS criterion demands negligible mutual information between the message and the eavesdropper’s observations even when maximized over all message distributions. A key tool for the achievability proof is a novel and stronger version of Wyner’s soft covering lemma. Specifically, a random codebook is shown to achieve the soft-covering phenomenon with high probability. The probability of failure is doubly exponentially small in the blocklength. Since the combined number of messages and subsets grows only exponentially with the blocklength, SS for the WTC II is established by using the union bound and invoking the stronger soft-covering lemma. The direct proof shows that rates up to the weak-secrecy capacity of the classic WTC with a DM erasure channel (EC) to the eavesdropper are achievable. The converse follows by establishing the capacity of this DM wiretap EC as an upper bound for the WTC II. From a broader perspective, the stronger soft-covering lemma constitutes a tool for showing the existence of codebooks that satisfy exponentially many constraints, a beneficial ability for many other applications in information theoretic security.

The Henchman Problem: Measuring Secrecy by the Minimum Distortion in a List

We introduce a new measure of information-theoretic secrecy based on rate-distortion theory and study it in the context of the Shannon cipher system. Whereas rate-distortion theory is traditionally concerned with a single reconstruction sequence, in this paper, we suppose that an eavesdropper produces a list of $2^{nR_{\sf L}}$ reconstruction sequences and measure secrecy by the minimum distortion over the entire list. We show that this setting is equivalent to one in which an eavesdropper must reconstruct a single sequence, but also receives side information about the source sequence and public message from a rate-limited henchman (a helper for an adversary). We characterize the optimal tradeoff of secret key rate, list rate, and eavesdropper distortion. The solution hinges on a problem of independent interest: lossy compression of a codeword uniformly drawn from a random codebook. We also characterize the solution to the lossy communication version of the problem in which distortion is allowed at the legitimate receiver. The analysis in both the settings is greatly aided by a recent technique for proving source coding results with the use of a likelihood encoder.

One-Shot Marton Inner Bound for Classical-Quantum Broadcast Channel

We consider the problem of communication over a classical-quantum broadcast channel with one sender and two receivers. Generalizing the classical inner bounds shown by Marton and the recent quantum asymptotic version shown by Savov and Wilde, we obtain one-shot inner bounds in the quantum setting. Our bounds are stated in terms of hypothesis testing and one-shot max divergences. These results give a full justification of the claims of Savov and Wilde in the classical-quantum asymptotic iid setting; the techniques also yield similar bounds in the information spectrum setting. We obtain these results using a different analysis of the random codebook argument; our method yields a classical one-shot Marton bound with a common message and a classical one-shot mutual covering lemma based on rejection sampling.

$E_{\gamma}$-Resolvability

The conventional channel resolvability refers to the minimum rate needed for an input process to approximate the channel output distribution in total variation distance. In this paper, we study $E_{\gamma }$ -resolvability, in which total variation is replaced by the more general $E_{\gamma }$ distance. A general one-shot achievability bound for the precision of such an approximation is developed. Let $Q_{\sf X|U}$ be a random transformation, $n$ be an integer, and $E\in (0,+\infty )$ . We show that in the asymptotic setting where $\gamma =\exp (nE)$ , a (nonnegative) randomness rate above $\inf _{Q_{\sf U}: D(Q_{\sf X}\|{\pi }_{\sf X})\le E} \{D(Q_{\sf X}\|{\pi }_{\sf X})+I(Q_{\sf U},Q_{\sf X|U})-E\}$ is sufficient to approximate the output distribution ${\pi }_{\sf X}^{\otimes n}$ using the channel $Q_{\sf X|U}^{\otimes n}$ , where $Q_{\sf U}\to Q_{\sf X|U}\to Q_{\sf X}$ , and is also necessary in the case of finite $\mathcal {U}$ and $\mathcal {X}$ . In particular, a randomness rate of $\inf _{Q_{\sf U}}I(Q_{\sf U},Q_{\sf X|U})-E$ is always sufficient. We also study the convergence of the approximation error under the high-probability criteria in the case of random codebooks. Moreover, by developing simple bounds relating $E_{\gamma }$ and other distance measures, we are able to determine the exact linear growth rate of the approximation errors measured in relative entropy and smooth Renyi divergences for a fixed-input randomness rate. The new resolvability result is then used to derive: 1) a one-shot upper bound on the probability of excess distortion in lossy compression, which is exponentially tight in the i.i.d. setting; 2) a one-shot version of the mutual covering lemma; and 3) a lower bound on the size of the eavesdropper list to include the actual message and a lower bound on the eavesdropper false-alarm probability in the wiretap channel problem, which is (asymptotically) ensemble-tight.

An Achievable Rate Region for the Three-User Interference Channel Based on Coset Codes

We consider the problem of communication over a three user discrete memoryless interference channel ($3-$IC). The current known coding techniques for communicating over an arbitrary $3-$IC are based on message splitting, superposition coding and binning using independent and identically distributed (iid) random codebooks. In this work, we propose a new ensemble of codes - partitioned coset codes (PCC) - that possess an appropriate mix of empirical and algebraic closure properties. We develop coding techniques that exploit algebraic closure property of PCC to enable efficient communication over $3-$IC. We analyze the performance of the proposed coding technique to derive an achievable rate region for the general discrete $3-$IC. Additive and non-additive examples are identified for which the derived achievable rate region is the capacity, and moreover, strictly larger than current known largest achievable rate regions based on iid random codebooks.

On the Performance of MIMO Nullforming with Random Vector Quantization Limited Feedback

This paper analyzes the performance of random vector quantization (RVQ) for limited feedback nullforming in multi-input multi-output (MIMO) communication systems with and without receiver coordination. A single-stream scenario is considered in which one or more primary receivers request nulls by providing limited feedback to the transmitter. Without receiver coordination, each primary receiver informs the transmitter of its best beamforming precoding vector. The transmitter then selects a zero-forcing precoding vector orthogonal to all of the beamforming precoding vectors. With receiver coordination, the primary receivers feed back the common precoding vector that minimizes the average interference. In both cases, secondary receivers in the network do not provide feedback and experience channels statistically equivalent to a single-antenna fading channel. Analytical results show that, for a system with K primary receivers and random codebooks with N=2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</sup> precoding vectors, the mean received power at the primary receivers is upper bounded by N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1/K</sup> = 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-B/K</sup> with or without receiver coordination. Exact results are also derived for the K=1 receiver case. Numerical results verify the scaling and also show that systems with receiver coordination outperform those without receiver coordination by a constant gap for large N in terms of average interference.

Codebook-Based Opportunistic Interference Alignment

For the multiple-input multiple-output interfering multiple-access channels (IMACs), opportunistic interference alignment (OIA) using the singular value decomposition (SVD)-based beamforming at each user fundamentally reduces the user scaling condition required to achieve any target DoF, compared to that for the single-input multiple-output IMAC. In this paper, we tackle two practical challenges of the existing SVD-based OIA: 1) the need of full feedforward of the selected users' beamforming weight vectors and 2) a low rate achieved based on the exiting zero-forcing receiver. We first propose a codebook-based OIA, in which the weight vectors are chosen from a predefined codebook with a finite size so that information of the weight vectors can be sent to the belonging BS with limited feedforward. We derive the codebook size required to achieve the same user scaling condition as the SVD-based OIA case for both Grassmannian and random codebooks. Surprisingly, it is shown that the derived codebook size is the same for the two codebook methods. Second, we introduce an enhanced receiver at the base stations (BSs) in pursuit of further improving the achievable rate. Assuming no collaboration between the BSs, the interfering links between a BS and the selected users in neighboring cells are difficult to be acquired at the belonging BS. We propose the use of a simple minimum Euclidean distance receiver operating with no information of the interfering links. With the help of the OIA, we show that this new receiver asymptotically achieves the channel capacity as the number of users increases.

On the Geometry and Quantization of Manifolds of Positive Semi-Definite Matrices

The geometry of different spaces of positive semi-definite matrices buffeted by rank and trace constraints is studied. In addition to revealing their Riemannian structure, we derive the normalized volume of a ball over these spaces. Further, we use the leading coefficient from the ball volume expansion to bound the quantization error incurred with finite-sized sphere-packing codebooks as well as random codebooks to represent sources distributed over general Riemannian manifolds.

A Single Feedback Based Interference Alignment for Three-User MIMO Interference Channels with Limited Feedback

Conventional interference alignment (IA) for a MIMO interference channel (IFC) requires global and perfect channel state information at transmitter (CSIT) to achieve the optimal degrees of freedom (DoF), which prohibits practical implementation. In order to alleviate the global CSIT requirement caused by the coupled relation among all of IA equations, we propose an IA scheme with a single feedback link of each receiver in a limited feedback environment for a three-user MIMO IFC. The main feature of the proposed scheme is that one of users takes out a fraction of its maximum number of data streams to decouple IA equations for three-user MIMO IFC, which results in a single link feedback structure at each receiver. While for the conventional IA each receiver has to feed back to all transmitters for transmitting the maximum number of data streams. With the assumption of a random codebook, we analyze the upper bound of the average throughput loss caused by quantized channel knowledge as a function of feedback bits. Analytic results show that the proposed scheme outperforms the conventional IA scheme in term of the feedback overhead and the sum rate as well.

On the Generalization of Natural Type Selection to Multiple Description Coding

Natural type selection was originally proposed by Zamir and Rose for universal single description coding. In this paper, we generalize this principle to universal multiple description coding (MDC). Two schemes based on random codebooks are proposed: one is of fixed distortion and the other is of fixed weight. Their operational sum-rate-distortion functions are derived, which coincide with the EGC (El Gamal-Cover) sum-rate bound if the parameters of the schemes are optimized. It is also shown that in both schemes the joint type of reconstruction codewords can be used to improve the rate-distortion (R-D) performance. Based on our theoretical results, a practical universal scheme is proposed by leveraging the MDC methods based on low-density generator matrix (LDGM) codes. The performance of this scheme is compared experimentally with the EGC bound, which shows its effectiveness.

Adjacent Channel Interference Cancellation Using Orthogonal Precoding for Fractional Sampling OFDM Receiver

To achieve diversity with a single antenna, fractional sampling has been proposed. The imaging component of a desired signal must be transmitted by increasing the baseband filter bandwidth. This paper proposes and evaluates a system in which the imaging component is transmitted on the adjacent channel to the desired signal. The adjacent channel interference is cancelled by the precoding generated based on the basis of the channel state information of all the terminals. In this paper, the performance with perfect feedback and limited feedback is evaluated. Numerical results obtained through computer simulation show that the proposed scheme with a 6 bit random codebook for QPSK modulation improves the performance compared with that with the conventional scheme in the low $$E_b/N_0$$ region.

Random vector quantisation modelling in automatic speaker verification

Automatic speaker verification (ASV) is a binary classification task. It consists of accepting or rejecting the claimed identity. ASV system has to decide whether a claimed speaker uttered a sentence. This paper proposes an algorithm called impostor random vector quantisation (IRVQ) based on multiples random codebook. IRVQ represents the impostor model also called universal background model (UBM) and we compare it to the second algorithm called partial impostor VQ (IVQ) for vector quantisation (VQ) in speaker verification. The present study demonstrates that the several random selected codebooks representing impostor models give better results and less half total error HTER than impostor IVQ method and baseline system. The performance of these models is evaluated on Arabic speaker verification dataset. However, this improvement also depends on the codebook size. The assumption concerning partitioning impostor acoustic spaces and choosing the best subspace to represent impostor model specific to each speaker is an efficient approach.