Problems In Information Theory Research Articles

Optimization of Time-Ordered Processes in the Finite and Asymptotic Regimes

Many problems in quantum information theory can be formulated as optimizations over the sequential outcomes of dynamical systems subject to unpredictable external influences. Such problems include many-body entanglement detection through adaptive measurements, computing the maximum average score of a preparation game over a continuous set of target states, and limiting the behavior of a (quantum) finite-state automaton. In this work, we introduce tractable relaxations of this class of optimization problems. To illustrate their performance, we use them to: (a) compute the probability that a finite-state automaton outputs a given sequence of bits; (b) develop a new many-body entanglement-detection protocol; and (c) let the computer an adaptive protocol for magic state detection. As we further show, the maximum score of a sequential problem in the limit of infinitely many time steps is in general incomputable. Nonetheless, we provide general heuristics to bound this quantity and show that they provide useful estimates in relevant scenarios. Published by the American Physical Society 2024

Perfect Codes over Non-Prime Power Alphabets: An Approach Based on Diophantine Equations

Perfect error-correcting codes allow for an optimal transmission of information while guaranteeing error correction. For this reason, proving their existence has been a classical problem in both pure mathematics and information theory. Indeed, the classification of the parameters of e-error correcting perfect codes over q-ary alphabets was a very active topic of research in the late 20th century. Consequently, all parameters of perfect e-error-correcting codes were found if e≥3, and it was conjectured that no perfect 2-error-correcting codes exist over any q-ary alphabet, where q>3. In the 1970s, this was proved for q a prime power, for q=2r3s and for only seven other values of q. Almost 50 years later, it is surprising to note that there have been no new results in this regard and the classification of 2-error-correcting codes over non-prime power alphabets remains an open problem. In this paper, we use techniques from the resolution of the generalised Ramanujan–Nagell equation and from modern computational number theory to show that perfect 2-error-correcting codes do not exist for 172 new values of q which are not prime powers, substantially increasing the values of q which are now classified. In addition, we prove that, for any fixed value of q, there can be at most finitely many perfect 2-error-correcting codes over an alphabet of size q.

Efficient Computation of the Quantum Rate-Distortion Function

The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allows us to better understand the properties of the quantum channels which obtain the optimal rate-distortion trade-off, while also allowing for more efficient computation of the quantum rate-distortion function regardless of the numerical algorithm being used. Additionally, we propose an inexact variant of the mirror descent algorithm to compute the quantum rate-distortion function with provable sublinear convergence rates. We show how this mirror descent algorithm is related to Blahut-Arimoto and expectation-maximization methods previously used to solve similar problems in information theory. Using these techniques, we present the first numerical experiments to compute a multi-qubit quantum rate-distortion function, and show that our proposed algorithm solves faster and to higher accuracy when compared to existing methods.

Neural Joint Entropy Estimation.

Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics, and data compression. Over the years, a variety of estimation schemes have been suggested. However, despite significant progress, most methods still struggle when the sample is small, compared to the variable's alphabet size. In this work, we introduce a practical solution to this problem, which extends the work of McAllester and Statos. The proposed scheme uses the generalization abilities of cross-entropy estimation in deep neural networks (DNNs) to introduce improved entropy estimation accuracy. Furthermore, we introduce a family of estimators for related information-theoretic measures, such as conditional entropy and mutual information (MI). We show that these estimators are strongly consistent and demonstrate their performance in a variety of use cases. First, we consider large alphabet entropy estimation. Then, we extend the scope to MI estimation. Next, we apply the proposed scheme to conditional MI estimation, as we focus on independence testing tasks. Finally, we study a transfer entropy (TE) estimation problem. The proposed estimators demonstrate improved performance compared to existing methods in all of these setups.

Пропускная способность и информационная емкость ресурса: пространство, частота, время

Studies of the bandwidth of the resource: space, frequency, time during spatial multiplexing in information transmission, storage and processing systems have been carried out. Bandwidth estimation is relevant in the development of communication systems with spatial multiplexing, as well as in problems of information theory. Limitations in spatial multiplexing are investigated. Theorems for estimating the amount of information passing through a given linear and flat area of space at certain values of the time interval and frequency band are formulated and proved. A theorem for estimating the amount of information passing through a rectangular parallelepiped due to three beams of electromagnetic radiation through three mutually perpendicular faces of a rectangular parallelepiped is formulated and proved. As an example, the amount of information contained in one cubic meter of space filled with electromagnetic radiation of a certain type is estimated. Thus, for frequencies in the range from 3 •108 Hz to 6•1019 Hz, one cubic meter of space contains 3,207•1034 bits of information, which is equivalent to the amount of information contained due to quantum states in a substance weighing 1,444 g at 20 °C. Practical significance: the research results can be useful for the development of information theory and the practice of developing communication systems with spatial multiplexing.

The expressivity of classical and quantum neural networks on entanglement entropy

Analytically continuing the von Neumann entropy from Rényi entropies is a challenging task in quantum field theory. While the n-th Rényi entropy can be computed using the replica method in the path integral representation of quantum field theory, the analytic continuation can only be achieved for some simple systems on a case-by-case basis. In this work, we propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. We begin by studying several examples with known von Neumann entropy, where the input data is generated by representing TrρAn\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\ ext {Tr}}\\rho _A^n$$\\end{document} with a generating function. We adopt KerasTuner to determine the optimal network architecture and hyperparameters with limited data. In addition, we frame a similar problem in terms of quantum machine learning models, where the expressivity of the quantum models for the entanglement entropy as a partial Fourier series is established. Our proposed methods can accurately predict the von Neumann and Rényi entropies numerically, highlighting the potential of deep learning techniques for solving problems in quantum information theory.

Entropy Estimators for Markovian Sequences: A Comparative Analysis.

Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performances vary depending on the system being studied and the available data size. In this work, we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation, and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields.

QARV: Quantization-Aware ResNet VAE for Lossy Image Compression.

This paper addresses the problem of lossy image compression, a fundamental problem in image processing and information theory that is involved in many real-world applications. We start by reviewing the framework of variational autoencoders (VAEs), a powerful class of generative probabilistic models that has a deep connection to lossy compression. Based on VAEs, we develop a new scheme for lossy image compression, which we name quantization-aware ResNet VAE (QARV). Our method incorporates a hierarchical VAE architecture integrated with test-time quantization and quantization-aware training, without which efficient entropy coding would not be possible. In addition, we design the neural network architecture of QARV specifically for fast decoding and propose an adaptive normalization operation for variable-rate compression. Extensive experiments are conducted, and results show that QARV achieves variable-rate compression, high-speed decoding, and better rate-distortion performance than existing baseline methods.

Polytope compatibility—From quantum measurements to magic squares

Several central problems in quantum information theory (such as measurement compatibility and quantum steering) can be rephrased as membership in the minimal matrix convex set corresponding to special polytopes (such as the hypercube or its dual). In this article, we generalize this idea and introduce the notion of polytope compatibility, by considering arbitrary polytopes. We find that semiclassical magic squares correspond to Birkhoff polytope compatibility. In general, we prove that polytope compatibility is in one-to-one correspondence with measurement compatibility, when the measurements have some elements in common and the post-processing of the joint measurement is restricted. Finally, we consider how much tuples of operators with appropriate joint numerical range have to be scaled in the worst case in order to become polytope compatible and give both analytical sufficient conditions and numerical ones based on linear programming.

Smoothing of Binary Codes, Uniform Distributions, and Applications.

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in the Euclidean space. We aim to characterize the cases when the output distribution is close to the uniform distribution on the space, as measured by the Rényi divergence of order α∈(1,∞]. A version of this question is known as the channel resolvability problem in information theory, and it has implications for security guarantees in wiretap channels, error correction, discrepancy, worst-to-average case complexity reductions, and many other problems. Our work quantifies the requirements for asymptotic uniformity (perfect smoothing) and identifies explicit code families that achieve it under the action of the Bernoulli and ball noise operators on the code. We derive expressions for the minimum rate of codes required to attain asymptotically perfect smoothing. In proving our results, we leverage recent results from harmonic analysis of functions on the Hamming space. Another result pertains to the use of code families in Wyner's transmission scheme on the binary wiretap channel. We identify explicit families that guarantee strong secrecy when applied in this scheme, showing that nested Reed-Muller codes can transmit messages reliably and securely over a binary symmetric wiretap channel with a positive rate. Finally, we establish a connection between smoothing and error correction in the binary symmetric channel.

Detecting entanglement harnessing Lindblad structure

The problem of entanglement detection is a long standing problem in quantum information theory. One of the primary procedures of detecting entanglement is to find the suitable positive but non-completely positive maps. Here we try to give a generic prescription to construct a positive map that can be useful for such scenarios. We study a class of positive maps arising from Lindblad structures. We show that two famous positive maps viz. transposition, reduction map and Choi map can be obtained as a special case of a class of positive maps having Lindblad structure. Generalizing the transposition map to a one parameter family we have used it to detect genuine multipartite entanglement. Finally being motivated by the negativity of entanglement, we have defined a similar measure for genuine multipartite entanglement.

Bayesian denoising of structured sources and its implications on learning-based denoising

Abstract Denoising a stationary process $(X_{i})_{i \in \mathbb{Z}}$ corrupted by additive white Gaussian noise $(Z_{i})_{i \in \mathbb{Z}}$ is a classic, well-studied and fundamental problem in information theory and statistical signal processing. However, finding theoretically founded computationally efficient denoising methods applicable to general sources is still an open problem. In the Bayesian set-up where the source distribution is known, a minimum mean square error (MMSE) denoiser estimates $X^{n}$ from noisy measurements $Y^{n}$ as $\hat{X}^{n}=\mathrm{E}[X^{n}|Y^{n}]$. However, for general sources, computing $\mathrm{E}[X^{n}|Y^{n}]$ is computationally very challenging, if not infeasible. In this paper, starting from a Bayesian set-up, a novel denoising method, namely, quantized maximum a posteriori (Q-MAP) denoiser is proposed and its asymptotic performance is analysed. Both for memoryless sources, and for structured first-order Markov sources, it is shown that, asymptotically, as $\sigma _{z}^{2} $ (noise variance) converges to zero, ${1\over \sigma _{z}^{2}} \mathrm{E}[(X_{i}-\hat{X}^{\mathrm{QMAP}}_{i})^{2}]$ converges to the information dimension of the source. For the studied memoryless sources, this limit is known to be optimal. A key advantage of the Q-MAP denoiser, unlike an MMSE denoiser, is that it highlights the key properties of the source distribution that are to be used in its denoising. This key property leads to a new learning-based denoising approach that is applicable to generic structured sources. Using ImageNet database for training, initial simulation results exploring the performance of such a learning-based denoiser in image denoising are presented.

Proving Information Inequalities and Identities With Symbolic Computation

Proving linear inequalities and identities of Shannon’s information measures, possibly with linear constraints on the information measures, is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In particular, an identity <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> = 0 is verified by solving two LPs, one for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> ≥ 0 and one for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> ≤ 0. In this paper, we develop a set of algorithms that can be implemented by symbolic computation. Based on these algorithms, procedures for verifying linear information inequalities and identities are devised. Compared with LP-based algorithms, our procedures can produce analytical proofs that are both human-verifiable and free of numerical errors. Our procedures are also more efficient computationally. For constrained inequalities, by taking advantage of the algebraic structure of the problem, the size of the LP that needs to be solved can be significantly reduced. For identities, instead of solving two LPs, the identity can be verified directly with very little computation.

Projections of orbital measures and quantum marginal problems

This paper studies projections of uniform random elements of (co)adjoint orbits of compact Lie groups. Such projections generalize several widely studied ensembles in random matrix theory, including the randomized Horn’s problem, the randomized Schur’s problem, and the orbital corners process. In this general setting, we prove integral formulae for the probability densities, establish some properties of the densities, and discuss connections to multiplicity problems in representation theory as well as to known results in the symplectic geometry literature. As applications, we show a number of results on marginal problems in quantum information theory and also prove an integral formula for restriction multiplicities.

Criteria for entanglement and separability of discrete quantum states

Entanglement is an important quantum resource, which can be used in quantum teleportation and quantum computation. How to judge and measure entanglement or separability has become a basic problem in quantum information theory. In this paper, by analyzing the properties of generalized ring $\mathbb{Z}[i]^{{2}^{n}}$, a new method is presented to judge the entanglement or separability of any quantum state in the discrete quantum computing model proposed by Gatti and Lacalle. Different from previous criteria based on matrices, it is relatively simple to operate in mathematical calculation. And if a quantum state is separable, it can calculate the separable mathematical expression. Taking $n=2,3$ as examples, the concrete forms of all separable states in the model are presented. It provides a new research perspective for the discrete quantum computing model.

Beating the Probabilistic Lower Bound on q-Perfect Hashing

In this paper, we investigate the achievable rate of perfect hash code. For an integer $$q\ge 2$$ , a perfect q-hash code C is a block code over $$[q]:=\{1,\ldots ,q\}$$ of length n in which every subset $$\{{\textbf{c}}_1,{\textbf{c}}_2,\dots ,{\textbf{c}}_q\}$$ of q elements is separated, i.e., there exists $$i\in [n]$$ such that $$\{\textrm{proj}_i({\textbf{c}}_1),\dots ,\textrm{proj}_i({\textbf{c}}_q)\}=[q]$$ , where $$\textrm{proj}_i({\textbf{c}}_j)$$ denotes the ith position of $${\textbf{c}}_j$$ . Finding the maximum size M(n, q) of perfect q-hash codes of length n, for given q and n, is a fundamental problem in combinatorics, information theory, and computer science. In this paper, we are interested in asymptotic behavior of this problem. Precisely speaking, we will focus on the quantity $$R_q:=\limsup _{n\rightarrow \infty }\frac{\log _2 M(n,q)}{n}$$ . A well-known probabilistic argument shows an existence lower bound on $$R_q$$ , namely $$R_q\ge \frac{1}{q-1}\log _2\left( \frac{1}{1-q!/q^q}\right) $$ (Fredman and Komlós, SIAM J Algebr Discrete Methods 5(1):61–68, 1984; Körner, SIAM J Algebr Discrete Methods 7(4):560–570, 1986). This is still the best-known lower bound till now except for the case $$q=3$$ (Körner and Marton, Eur J Comb 9(6):523–530, 1988). The improved lower bound of $$R_3$$ was discovered in 1988 and there has been no progress on the lower bound of $$R_q$$ for more than 30 years. In this paper we show that this probabilistic lower bound can be improved for q from 4 to 15 and all odd integers between 17 and 25, and all sufficiently large q.

New Mathematical Techniques in Information Theory

Information theory is the richer for a surge of recent advances in relevant mathematical techniques. The workshop fostered an exchange of ideas on new mathematical tools which are typically outside the classical toolbox of information theorists and that are yet useful in solving classical and modern problems in information theory and related areas. The focus was on mathematical techniques that are of a general nature and that could benefit a wide class of problems. A number of broad mathematical areas were identified that held promise with established early successes, and key contributors were invited to make presentations and initiate discussions with an emphasis on emergent topics. The areas were: information measures, measure concentration, hypercontractivity and correlation measures, Shannon theory and extremal combinatorics, advanced tools for proving converse results in coding theorems, and recent techniques for proving Gaussian optimality entailing new characterizations of Gaussian distributions.

The face lattice of the set of reduced density matrices and its coatoms

Many problems of quantum information theory rely on the set of quantum marginals. A precise knowledge of the faces of this convex set is necessary, for example, in the reconstruction of states from their marginals or in the evaluation of complexity measures of many-body systems. Yet, even the two-body marginals of just three qubits were only described in part. Here, we propose an experimental method to search for the coatoms in the lattice of exposed faces of the convex set of quantum marginals. The method is based on sampling from the extreme points of the dual spectrahedron. We provide an algebraic certificate of correctness, employing ground projectors of local Hamiltonians. Using this method, we present an explicit family of coatoms of rank five in the lattice of ground projectors of two-local three-qubit Hamiltonians (the rank is always six for bits). This family describes a family of coatoms in the lattice of exposed faces of the convex set of two-body marginals of three qubits. Besides introducing the experimental method, we show that the support sets of probability distributions that factor are the ground projectors of frustration-free Hamiltonians in the commutative setting. We also discuss nonexposed points of the set of marginals.

Rank-Adaptive Tensor Completion Based on Tucker Decomposition

Tensor completion is a fundamental tool to estimate unknown information from observed data, which is widely used in many areas, including image and video recovery, traffic data completion and the multi-input multi-output problems in information theory. Based on Tucker decomposition, this paper proposes a new algorithm to complete tensors with missing data. In decomposition-based tensor completion methods, underestimation or overestimation of tensor ranks can lead to inaccurate results. To tackle this problem, we design an alternative iterating method that breaks the original problem into several matrix completion subproblems and adaptively adjusts the multilinear rank of the model during optimization procedures. Through numerical experiments on synthetic data and authentic images, we show that the proposed method can effectively estimate the tensor ranks and predict the missing entries.

On the Generalized Quadratic-Quartic Cauchy Functional Equation and its Stability over Non-Archimedean Normed Space

Functional equation plays a very important and interesting role in the area of mathematics, which involves simple algebraic manipulations and through which one can arrive an interesting solution. The theory of functional equations is also used in the development of other areas such as analysis, algebra, Geometry etc., the new methods and techniques are applied in solving problem in Information theory, Finance, Geometry, wireless sensor networks etc., In recent decades, the study of various types of stability of a functional equation such as HUS (Hyers-Ulam stability), HURS (Hyers-Ulam-Rassias stability) and generalized HUS of different types of functional equation and also for mixed type were discussed by many authors in various space. The problem of the stability of different functional equations has been widely studied by many authors, and more interesting results have been proved in the classical case (Archimedean). In recent years, the analogues results of the stability problem of these functional equations were investigated in non-Archimedean space. The aim of this study is to investigate the HUS of a mixed type of general Quadratic-Quartic Cauchy functional equation in non-Archimedean normed space. In this current article, we prove the generalized HUS for the following Quadratic-Quartic Cauchy functional equation over non-Archimedean Normed space.<img src=image/13428783_01.gif>