Problems In Information Theory Research Articles

On approximate real mutually unbiased bases in square dimension

Construction of Mutually Unbiased Bases (MUBs) is a very challenging combinatorial problem in quantum information theory with several long standing open questions in this domain. With certain relaxations, the object Approximate Mutually Unbiased Bases (AMUBs) is defined in this context. In this paper we provide a method to construct upto $(\sqrt {d} + 1)$ many AMUBs in dimension d = q2, where q is a positive integer. The result is particularly important when q ≡ 0 mod 4, as we obtain Approximate Real MUBs (ARMUBs) assuming the cases where a Hadamard matrix of order q exists. In this construction, we also characterize the inner product values between the elements of two different bases. In particular, when d is of the form (4x)2 where x is a prime, we obtain $(\frac {\sqrt {d}}{4}+1)$ ARMUBs such that for any two vectors v1,v2 belonging to different bases, $|{\left \langle \left . v_{1} \right | v_{2} \right \rangle }| \leq \frac {4}{\sqrt {d}}$ .

Tropical probability theory and an application to the entropic cone

In a series of articles, we have been developing a theory of tropical diagrams of probability spaces, expecting it to be useful for information optimization problems in information theory and artificial intelligence. In this article, we give a summary of our work so far and apply the theory to derive a dimension-reduction statement about the shape of the entropic cone.

Reduced-Complexity Optimization of Distributed Quantization Using the Information Bottleneck Principle

This paper addresses the optimization of distributed compression in a sensor network. A direct communication among the sensors is not possible so that noisy measurements of a single relevant signal have to be locally compressed in order to meet the rate constraints of the communication links to a common receiver. This scenario is widely known as the Chief Executive Officer (CEO) problem and represents a long-standing problem in information theory. In recent years significant progress has been achieved and the rate region has been completely characterized for specific distributions of involved processes and distortion measures. While algorithmic solutions of the CEO problem are principally known, their practical implementation quickly becomes challenging due to complexity reasons. In this contribution, an efficient greedy algorithm to determine feasible solutions of the CEO problem is derived using the information bottleneck (IB) approach. Following the Wyner-Ziv coding principle, the quantizers are successively designed using already optimized quantizer mappings as side-information. However, processing this side-information in the optimization algorithm becomes a major bottleneck because the memory complexity grows exponentially with number of sensors. Therefore, a sequential compression scheme leading to a compact representation of the side-information and ensuring moderate memory requirements even for larger networks is introduced. This internal compression is optimized again by means of the IB method. Numerical results demonstrate that the overall loss in terms of relevant mutual information can be made sufficiently small even with a significant compression of the side-information. The performance is compared to separately optimized quantizers and a centralized quantization. Moreover, the influence of the optimization order for asymmetric scenarios is discussed.

Quantum-Enhanced Barcode Decoding and Pattern Recognition

Quantum hypothesis testing is one of the most fundamental problems in quantum information theory, with crucial implications in areas like quantum sensing, where it has been used to prove quantum advantage in a series of binary photonic protocols, e.g., for target detection or memory cell readout. In this work, we generalize this theoretical model to the multi-partite setting of barcode decoding and pattern recognition. We start by defining a digital image as an array or grid of pixels, each pixel corresponding to an ensemble of quantum channels. Specializing each pixel to a black and white alphabet, we naturally define an optical model of barcode. In this scenario, we show that the use of quantum entangled sources, combined with suitable measurements and data processing, greatly outperforms classical coherent-state strategies for the tasks of barcode data decoding and classification of black and white patterns. Moreover, introducing relevant bounds, we show that the problem of pattern recognition is significantly simpler than barcode decoding, as long as the minimum Hamming distance between images from different classes is large enough. Finally, we theoretically demonstrate the advantage of using quantum sensors for pattern recognition with the nearest neighbor classifier, a supervised learning algorithm, and numerically verify this prediction for handwritten digit classification.

Information decomposition based on cooperative game theory

We offer a new approach to the information decomposition problem in information theory: given a 'target' random variable co-distributed with multiple 'source' variables, how can we decompose the mutual information into a sum of non-negative terms that quantify the contributions of each random variable, not only individually but also in combination? We derive our composition from cooperative game theory. It can be seen as assigning a fair share of the mutual information to each combination of the source variables. Our decomposition is based on a different lattice from the usual 'partial information decomposition' (PID) approach, and as a consequence our decomposition has a smaller number of terms: it has analogs of the synergy and unique information terms, but lacks terms corresponding to redundancy. Because of this, it is able to obey equivalents of the axioms known as 'local positivity' and 'identity', which cannot be simultaneously satisfied by a PID measure.

Source Resolvability and Intrinsic Randomness: Two Random Number Generation Problems With Respect to a Subclass of f-Divergences

This paper deals with two typical random number generation problems in information theory. One is the source resolvability problem (resolvability problem for short) and the other is the intrinsic randomness problem. In the literature, optimum achievable rates in these two problems with respect to the variational distance as well as the Kullback-Leibler (KL) divergence have already been analyzed. On the other hand, in this study we consider these two problems with respect to f-divergences. The f-divergence is a general non-negative measure between two probabilistic distributions on the basis of a convex function f. The class of f-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to f-divergences. In this paper, we impose some conditions on the function f so as to simplify the analysis, that is, we consider a subclass of f-divergences. Then, we first derive general formulas of the first-order optimum achievable rates with respect to f-divergences. Next, we particularize our general formulas to several specified functions f. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. The second-order optimum achievable rates and optimistic optimum achievable rates have also been investigated.

(Invited) High Performance Stochastic Memristive Networks for Neurocomputing and Neurooptimization

Stochastic neural networks have become the state-of-the-art approach for solving problems in machine learning, information theory, and statistics. The key operation in such networks is a stochastic dot-product. While there have been many demonstrations of dot-product circuits and, separately, of stochastic neurons, the efficient hardware implementation combining both functionalities is still missing. In my talk I will discuss our recent work on addressing this need. I will first discuss very compact, fast, energy-efficient, and scalable implementation of stochastic dot-product circuits based on passively integrated metal-oxide memristors. The high performance of such circuits is due to mixed-signal implementation, while the efficient stochastic operation is achieved by utilizing circuit’s noise, intrinsic and/or extrinsic to the memory cell array. The dynamic scaling of weights, enabled by analog memory devices, allows for efficient realization of different annealing approaches to improve functionality. I will then discuss several application demonstrations based on passively integrated TiO2-x memristors, including Hopfield networks for solving combinatorial optimization problems, and a Boltzmann machine.

Geometry of Arimoto algorithm

In information theory, the channel capacity, which indicates how efficient a given channel is, plays an important role. The best-used algorithm for evaluating the channel capacity is Arimoto algorithm [3]. This paper aims to reveal an information geometric structure of Arimoto algorithm. In the process of trying to reveal an information geometric structure of Arimoto algorithm, a new algorithm that monotonically increases the Kullback-Leibler divergence is proposed, which is called “the Backward em-algorithm.” Since the Backward em-algorithm is available in many cases where we need to increase the Kullback-Leibler divergence, it has a rich potential for application to many problems of statistics and information theory.

Bounding sets of sequential quantum correlations and device-independent randomness certification

An important problem in quantum information theory is that of bounding sets of correlations that arise from making local measurements on entangled states of arbitrary dimension. Currently, the best-known method to tackle this problem is the NPA hierarchy; an infinite sequence of semidefinite programs that provides increasingly tighter outer approximations to the desired set of correlations. In this work we consider a more general scenario in which one performs sequences of local measurements on an entangled state of arbitrary dimension. We show that a simple adaptation of the original NPA hierarchy provides an analogous hierarchy for this scenario, with comparable resource requirements and convergence properties. We then use the method to tackle some problems in device-independent quantum information. First, we show how one can robustly certify over 2.3 bits of device-independent local randomness from a two-quibt state using a sequence of measurements, going beyond the theoretical maximum of two bits that can be achieved with non-sequential measurements. Finally, we show tight upper bounds to two previously defined tasks in sequential Bell test scenarios.

A gap in the slice rank of k-tensors

The slice-rank method, introduced by Tao as a symmetrized version of the polynomial method of Croot, Lev and Pach and Ellenberg and Gijswijt, has proved to be a useful tool in a variety of combinatorial problems. Explicit tensors have been introduced in different contexts but little is known about the limitations of the method.In this paper, building upon a method presented by Tao and Sawin, it is proved that the asymptotic slice rank of any k-tensor in any field is either 1 or at least k/(k−1)(k−1)/k. This provides evidence that straight-forward application of the method cannot give useful results in certain problems for which non-trivial exponential bounds are already known. An example, actually a motivation for starting this work, is the problem of bounding the size of sets of trifferent sequences, which constitutes a long-standing open problem in information theory and in theoretical computer science.

Rényi formulation of uncertainty relations for POVMs assigned to a quantum design

Information entropies provide powerful and flexible way to express restrictions imposed by the uncertainty principle. This approach seems to be very suitable in application to problems of quantum information theory. It is typical that questions of such a kind involve measurements having one or another specific structure. The latter often allows us to improve entropic bounds that follow from uncertainty relations of sufficiently general scope. Quantum designs have found use in many issues of quantum information theory, whence uncertainty relations for related measurements are of interest. In this paper, we obtain uncertainty relations in terms of min-entropies and Rényi entropies for POVMs assigned to a quantum design. Relations of the Landau–Pollak type are addressed as well. Using examples of quantum designs in two dimensions, the obtained lower bounds are then compared with the previous ones. An impact on entropic steering inequalities is briefly discussed.

Separability discrimination and decomposition of m -partite quantum mixed states

The separability detecting problem of mixed states is one of the fundamental problems in quantum information theory. In the last 20 years, almost all methods are based on the sufficient or necessary conditions for entanglement. However, in this paper, we only need one algorithm to solve the problem. We propose a tensor optimization method to check whether an $m$-partite quantum mixed state is separable or not and give a decomposition for it if it is. We first convert the separability discrimination problem of mixed states to the positive Hermitian decomposition problem of Hermitian tensors. Then, employing the $E$-truncated $K$-moment method, we obtain an optimization model for discriminating separability. Moreover, applying semidefinite relaxation method, we get a hierarchy of semidefinite relaxation optimization models and propose an $E$-truncated $K$-moment and semidefinite relaxations (ETKM-SDR) algorithm for detecting the separability of mixed states. The algorithm can also be used for symmetric and non-symmetric decomposition of separable mixed states. By numerical examples, we find that not all symmetric separable states have symmetric decompositions. The algorithm can be used for studying properties of mixed states in the future.

An Overview of Capacity Results for Synchronization Channels

Synchronization channels, such as the well-known deletion channel, are surprisingly harder to analyze than memoryless channels, and they are a source of many fundamental problems in information theory and theoretical computer science. One of the most basic open problems regarding synchronization channels is the derivation of an exact expression for their capacity. Unfortunately, most of the classic information-theoretic techniques at our disposal fail spectacularly when applied to synchronization channels. Therefore, new approaches must be considered to tackle this problem. This survey gives an account of the great effort made over the past few decades to better understand the (broadly defined) capacity of synchronization channels, including both the main results and the novel techniques underlying them. Besides the usual notion of channel capacity, we also discuss the zero-error capacity of adversarial synchronization channels.

Tropical diagrams of probability spaces

After endowing the space of diagrams of probability spaces with an entropy distance, we study its large-scale geometry by identifying the asymptotic cone as a closed convex cone in a Banach space. We call this cone the tropical cone, and its elements tropical diagrams of probability spaces. Given that the tropical cone has a rich structure, while tropical diagrams are rather flexible objects, we expect the theory of tropical diagrams to be useful for information optimization problems in information theory and artificial intelligence. In a companion article, we give a first application to derive a statement about the entropic cone.

On the Compression of Messages in the Multi-Party Setting

We consider the following communication task in the multi-party setting, which involves a joint random variable $XYZMN$ with the property that $M$ is independent of $YZN$ conditioned on $X$ and $N$ is independent of $XZM$ conditioned on $Y$. Three parties Alice, Bob and Charlie, respectively, observe samples $x,y$ and $z$ from $XYZ$. Alice and Bob communicate messages to Charlie with the goal that Charlie can output a sample from $MN$ having correct correlation with $XYZ$. This task reflects the simultaneous message passing model of communication complexity. Furthermore, it is a generalization of some well studied problems in information theory, such as distributed source coding, source coding with a helper and one sender and one receiver message compression. It is also closely related to the lossy distributed source coding task. Our main result is an achievable communication region for this task in the one-shot setting, through which we obtain a near optimal characterization using auxiliary random variables of bounded size. We employ our achievability result to provide a near-optimal one-shot communication region for the task of lossy distributed source coding, in terms of auxiliary random variables of bounded size. Finally, we show that interaction is necessary to achieve the optimal expected communication cost for our main task.

Constellation Optimization for Phase-Shift Keying Coherent States With Displacement Receiver to Maximize Mutual Information

An important problem in quantum information theory is finding the best possible capacity of the optical communication channel employing suitable codewords, receiver design, and constellation optimization techniques. In this paper, we derive an alternative channel capacity, $C_{G}$ , of phase-shift keying coherent state with a realizable displacement receiver by maximizing mutual information over symbol priors and pre-detection displacement. We find that $C_{G}$ is higher than the capacity achieved by maximizing mutual information over symbol prior but with zero displacement. The overall scheme demonstrates designing an improved, yet easy-to-implement receiver for better communication performance by tuning it at different photon number regime. Further, we present a comparative analysis of $C_{G}$ with existing receiver designs. We extend our study to account for detector imperfections.

On the explicit constructions of certain unitary t-designs

Unitary t-designs are ‘good’ finite subsets of the unitary group that approximate the whole unitary group well. Unitary t-designs have been applied in randomized benchmarking, tomography, quantum cryptography and many other areas of quantum information science. If a unitary t-design itself is a group then it is called a unitary t-group. Although it is known that unitary t-designs in exist for any t and d, the unitary t-groups do not exist for if , as it is shown by Guralnick–Tiep (Guralnick and Tiep 2005 Represent. Theory 9 138–208) and Bannai–Navarro–Rizo–Tiep (Bannai et al 2018 J. Math. Soc. Japan (accepted)). Explicit constructions of exact unitary t-designs in are not easy in general. In particular, explicit constructions of unitary 4-designs in have been an open problem in quantum information theory. We prove that some exact unitary -designs in the unitary group are constructed from unitary t-groups in that satisfy certain specific conditions. Based on this result, we specifically construct exact unitary 3-designs in from the unitary 2-group in and also unitary 4-designs in from the unitary 3-group in numerically. We also discuss some related problems and a few applications to physics problems.

Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

The key operation in stochastic neural networks, which have become the state-of-the-art approach for solving problems in machine learning, information theory, and statistics, is a stochastic dot-product. While there have been many demonstrations of dot-product circuits and, separately, of stochastic neurons, the efficient hardware implementation combining both functionalities is still missing. Here we report compact, fast, energy-efficient, and scalable stochastic dot-product circuits based on either passively integrated metal-oxide memristors or embedded floating-gate memories. The circuit’s high performance is due to mixed-signal implementation, while the efficient stochastic operation is achieved by utilizing circuit’s noise, intrinsic and/or extrinsic to the memory cell array. The dynamic scaling of weights, enabled by analog memory devices, allows for efficient realization of different annealing approaches to improve functionality. The proposed approach is experimentally verified for two representative applications, namely by implementing neural network for solving a four-node graph-partitioning problem, and a Boltzmann machine with 10-input and 8-hidden neurons.

The Independence Number of the Orthogonality Graph in Dimension 2k

We determine the independence number of the orthogonality graph on $2^k$-dimensional hypercubes. This answers a question by Galliard from 2001 which is motivated by a problem in quantum information theory. Our method is a modification of a rank argument due to Frankl who showed the analogous result for $4p^k$-dimensional hypercubes, where $p$ is an odd prime.

Jointly Constrained Semidefinite Bilinear Programming With an Application to Dobrushin Curves

We propose a branch-and-bound algorithm for minimizing a bilinear functional of the form f (X, Y) = tr((X ⊗ Y) Q) +tr(AX) +tr(BY), of pairs of Hermitian matrices (X, Y) restricted by joint semidefinite programming constraints. The functional is parametrized by self-adjoint matrices Q, A and B. This problem generalizes that of a bilinear program, where X and Y belong to polyhedra. The algorithm converges to a global optimum and yields upper and lower bounds on its value in every step. Various problems in quantum information theory can be expressed in this form. As an example application, we compute Dobrushin curves of quantum channels, giving upper bounds on classical coding with energy constraints.