Problems In Information Theory Research Articles

Deterministic Network Model Revisited: An Algebraic Network Coding Approach

The capacity of multiuser networks has been a long-standing problem in information theory. Recently, Avestimehr et al. have proposed a deterministic network model to approximate multiuser wireless networks. This model, known as the ADT network model, takes into account the broadcast nature as well as the multiuser interference inherent in the wireless medium. For the types of connections we consider, we show that the results of Avestimehr et al. under the ADT model can be reinterpreted within the algebraic network coding frame- work introduced by Koetter and Medard. Using this framework, we propose an efficient distributed linear code construction for the deterministic wireless multicast relay network model. Unlike several previous coding schemes, we do not attempt to find flows in the network. Instead, for a layered network, we maintain an invariant where it is required that at each stage of the code construction, certain sets of codewords are linearly independent.

The Entropy Power Inequality and Mrs. Gerber's Lemma for Groups of Order <inline-formula> <tex-math notation="TeX">${2^{n}}$ </tex-math></inline-formula>

Shannon's entropy power inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key role in resolving a number of problems in information theory. It is therefore interesting to examine the existence of a similar inequality for discrete random variables. In this paper, we obtain an entropy power inequality for random variables taking values in a group of order 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> , i.e., for such a group G, we explicitly characterize the function f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</sub> (x, y) giving the minimum entropy of the sum of two independent G-valued random variables with respective entropies x and y. Random variables achieving the extremum in this inequality are thus the analogs of Gaussians in this case, and these are also determined. It turns out that f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</sub> (x, y) is convex in x for fixed y and, by symmetry, convex in y for fixed x. This is a generalization to groups of order 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> of the result known as Mrs. Gerber's Lemma.

Connection between conjunctive capacity and structural properties of graphs

The investigation of the asymptotic behavior of various graph parameters in powers of a graph is motivated by problems in information theory and extremal combinatorics. Considering various parameters and/or various notions of graph powers we can arrive at different notions of graph capacities, of which the Shannon capacity is best known. Here we study a related notion of the so-called conjunctive capacity of a graph G, CAND(G), introduced and studied by Gargano, Körner and Vaccaro. To determine CAND(G) is a convex programming problem. We show that the optimal solution to this problem is unique and describe the structure of the solution in any (simple) graph. We prove that its reciprocal value vcC(G):=1CAND(G) is an optimal solution of the newly introduced problem of Minimum Capacitary Vertex Cover that is closely related to the LP-relaxation of the Minimum Vertex Cover Problem. We also describe its close connection with the binding number/binding set of a graph, and with the strong crown decomposition of graphs.

Quantum Hamiltonian learning using imperfect quantum resources

Identifying an accurate model for the dynamics of a quantum system is a vexing problem that underlies a range of problems in experimental physics and quantum information theory. Recently, a method called quantum Hamiltonian learning has been proposed by the present authors that uses quantum simulation as a resource for modeling an unknown quantum system. This approach can, under certain circumstances, allow such models to be efficiently identified. A major caveat of that work is the assumption of that all elements of the protocol are noise free. Here we show that quantum Hamiltonian learning can tolerate substantial amounts of depolarizing noise and show numerical evidence that it can tolerate noise drawn from other realistic models. We further provide evidence that the learning algorithm will find a model that is maximally close to the true model in cases where the hypothetical model lacks terms present in the true model. Finally, we also provide numerical evidence that the algorithm works for noncommuting models. This work illustrates that quantum Hamiltonian learning can be performed using realistic resources and suggests that even imperfect quantum resources may be valuable for characterizing quantum systems.

The Capacity Region of the Two-Receiver Gaussian Vector Broadcast Channel With Private and Common Messages

A novel method for establishing the optimality of Gaussian auxiliary random variables in multiterminal information theory problems is developed. This method is then employed to show that Marton's inner bound achieves the capacity region of the two-receiver Gaussian vector broadcast channel with private and common messages.

On Marton's Inner Bound and Its Optimality for Classes of Product Broadcast Channels

Marton's inner bound is the tightest known inner bound on the capacity region of the broadcast channel. It is not known, however, if this bound is tight in general. One approach to settle this key open problem in network information theory is to investigate the multiletter extension of Marton's bound, which is known to be tight in general. This approach has become feasible only recently through the development of a new method for bounding cardinalities of auxiliary random variables by Gohari and Anantharam. This paper undertakes this long overdue approach to establish several new results, including 1) establishing the optimality of Marton's bound for new classes of product broadcast channels, 2) showing that the best-known outer bound by Nair and El Gamal is not tight in general, and 3) finding sufficient conditions for a global maximizer of Marton's bound that imply that the 2-letter extension does not increase the achievable rate. Motivated by the new capacity results, we establish a new outer bound on the capacity region of product broadcast channels.

On the capacity of multiple‐input multiple‐output GaussianXchannels

Finding the capacity region of a Gaussian X network is an open problem in information theory. In this study, the authors characterise the capacity regions of 2 × 2 user multiple-input multiple-output Gaussian X channels for very strong interference and generally strong interference. Moreover, the sum capacity is established for noisy interference. Sufficient conditions for Gaussian X networks to have very strong interference, generally strong interference and noisy interference are derived.

MOKSLOTYROS KELIAS Į LIETUVĄ – NAUJA PROF. ONOS VOVERIENĖS KNYGA „MOKSLOTYRA“

The topic of citation, input by country and other scientometrical indicators are widely discussed by Lithuanian scholars, though academic librarians still recall the questions about impact factors and “which are THE journals” to send the articles to, and so to obtain the required scores for the scientific certification. The scientometrical ideas become more popu-lar in Lithuania when “Social functions of science” (1939) by John Desmond Bernal (1901–1970) in 1965 and one year later – a “Little science. Big science” (1963) by Derek John de Solla Price (1922–1983) were translated to Russian language (they were almost unavailable in the original language for scholars in Soviet Union). These world famous monographs were followed by other monographs in scientometrics by soviet scholars, subject of scientometrics discussed at conferences and journal articles. “Scientometrics” was the first article (1969) published in a popular journal Mokslas ir technika by Stasys Poškus. The role of Lithuanian institute of Information science and the board of “Kibernetics” at the Lithuanian Academy of Sciences were outstanding in this field during early 60’ies. Lithuanian scholars were inves-tigating citation impact on science development and information service, information theo-ry, connection between scientometrics and history of science, etc. The scientific school of scientometricians in Lithuania was raised by Ona Voverienė, a famous professor of Vilnius University. O. Voverienė worked as an information officer at the company of chemical industry, investigated information theory, extended the investiga-tion of scientific communities in Lithuania. The science on science policy, main achieve-ments and personal structure of Lithuanian scholarship were presented in a trilogy: “Biblio-mertrija” (Bibliometrics, 1999), “Mokslinės mokyklos Lietuvoje” (Scientific schools in Lith-uanian, with co-authors, 2002) and “Žymieji 20-jojo amžiaus Lietuvos mokslininkai” (Fa-mous Lithuanian scholars in 20th century, 2009). O. Voverienė encouraged a “Bibliometrical seminar” for problems of information theory and bibliometrical investigations (1999–2001 at the Wrobelwski Library of the Lithuanian Academy of Sciences), though professor’s idea to establish a centre for Scientometrics is still unaccomplished. The history of scientometrics (the former term „science on science“ is considered as outdated in the scholarly writings in English language, though in some other languages it is still well accepted) from both historical and regional perspective is presented in the latest monograph „Mokslotyra“* (2013) by professor Ona Voverienė. The contents cover devel-opment of scientometrics in East Europe and Lithuania, theory and methodology are doc-umented using the literature analysis. The biograms of outstanding scientometicians are pre-sented together with Lithuanian specialist. Key words: bibliometrical investigations, information theory, scientometrics.

Basic concepts of entropy, order, organization, information, knowledge, and meaning

The classical concept of entropy is shown not to reflect the essence of the second law of thermodynamics both phenomenologically and theoretically. A new concept of entropy, S n , is given, which offers a particular physical meaning of entropy and provides an experimental determination of the entropy value. This allows one to avoid many attempts at defining the phenomenon of life via entropy characteristics and give definitions of life and an elementary structure of both life itself and devices that simulate life (automata, computers, and robots.). This, in turn, allows one to assign a substantive sense to the mathematical character of corresponding sciences (information theory, cybernetics, system theory, etc.) and define operational basic concepts of information, knowledge, meaning, and control, as well as to formulate operational concepts of order, organization, and self-organization and implement a new approach to problems of biology, biophysics, information theory, cybernetics, and system theory.

Fundamental Bound on the Reliability of Quantum Information Transmission

Information theory tells us that if the rate of sending information across a noisy channel were above the capacity of that channel, then the transmission would necessarily be unreliable. For classical information sent over classical or quantum channels, one could, under certain conditions, make a stronger statement that the reliability of the transmission shall decay exponentially to zero with the number of channel uses, and the proof of this statement typically relies on a certain fundamental bound on the reliability of the transmission. Such a statement or the bound has never been given for sending quantum information. We give this bound and then use it to give the first example where the reliability of sending quantum information at rates above the capacity decays exponentially to zero. We also show that our framework can be used for proving generalized bounds on the reliability.

Quantum entanglement and the communication complexity of the inner product function

We consider the communication complexity of the binary inner product function in a variation of the two-party scenario where the parties have an a priori supply of particles in an entangled quantum state. We prove linear lower bounds for both exact protocols, as well as for protocols that determine the answer with bounded-error probability. Our proofs employ a novel kind of “quantum” reduction from a quantum information theory problem to the problem of computing the inner product. The communication required for the former problem can then be bounded by an application of Holevo’s theorem. We also give a specific example of a probabilistic scenario where entanglement reduces the communication complexity of the inner product function by one bit.

Optimal fidelity for a quantum channel may be attained by nonmaximally entangled states

To establish an entangled state of optimal fidelity between two distant observers when the available quantum channel is noisy, is a central problem in quantum information theory. We consider an instance of this problem for two-qubit systems when only a single use of the channel and local post-processing by trace preserving operations are allowed. We show that the optimal fidelity is obtained only when part of an appropriate non-maximally entangled state is transmitted through the channel. The entanglement of this state can be vanishingly small when the channel becomes very noisy. Moreover, in the optimal case no further local processing is required to enhance the fidelity. We further show that local post-processing can enhance fidelity if and only if the amount of noise is larger than a critical value and entanglement of the transmitted state is bounded from below. A notable consequence of these results is that the ordering of states under an entanglement monotone can be reversed even when the states undergo the same local interaction via a trace-preserving completely positive map.

Experimental device-independent tests of classical and quantum dimensions

A fundamental resource in any communication and computation task is the amount of information that can be transmitted and processed. Information encoded in a classical system is limited by the dimension d_c of the system, i.e., the number of distinguishable states. A system with d_c=2^n classical states can carry n bits of classical information. Information encoded in a quantum system is limited by the dimension d_q of the Hilbert space of the system, i.e., the number of perfectly distinguishable quantum states. A system with d_q=2^n perfectly distinguishable quantum states can carry n qubits of quantum information. Physical systems of higher dimensions may enable more efficient and powerful information processing protocols. The dimension is fundamental in quantum cryptography and random number generation, where the security of many schemes crucially relies on the system's dimension. From a fundamental perspective, the dimension can be used to quantify the non-classicality of correlations, since classical simulation of correlations produced by a quantum system of dimension d_q may require a classical system of dimension d_c >> d_q. For all these reasons, a fundamental problem in information theory is to assess the (classical or quantum) dimension of a physical system in a "device-independent" scenario, i.e., without referring to the system's specifications, which may be under control of a dishonest supplier, eavesdropper or saboteur. In this contribution we report experiments realizing this goal for systems emitted by a black box. Our results indicate that dimension witnesses utilized in the experiments may become a powerful tool for testing systems provided by unreliable sources.

Majorization Theory Approach to the Gaussian Channel Minimum Entropy Conjecture

A long-standing open problem in quantum information theory is to find the classical capacity of an optical communication link, modeled as a Gaussian bosonic channel. It has been conjectured that this capacity is achieved by a random coding of coherent states using an isotropic Gaussian distribution in phase space. We show that proving a Gaussian minimum entropy conjecture for a quantum-limited amplifier is actually sufficient to confirm this capacity conjecture, and we provide a strong argument towards this proof by exploiting a connection between quantum entanglement and majorization theory.

Optimal measures and Markov transition kernels

We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optimal measures belong to a one-parameter exponential family. Measures within such a family have the property of mutual absolute continuity. Here we show that this property characterizes other families of optimal positive measures if a functional representing information has a strictly convex dual. Mutual absolute continuity of optimal probability measures allows us to strictly separate deterministic and non-deterministic Markov transition kernels, which play an important role in theories of decisions, estimation, control, communication and computation. We show that deterministic transitions are strictly sub-optimal, unless information resource with a strictly convex dual is unconstrained. For illustration, we construct an example where, unlike non-deterministic, any deterministic kernel either has negatively infinite expected utility (unbounded expected error) or communicates infinite information.

Information and inference in the wireless physical layer

Wireless networking applications continue to motivate challenging problems in information theory, signal processing, and other fields. This article explores briefly four research areas, primarily involving information theoretic or inferential problems, each of which is motivated by a wireless application-layer issue. In particular, the four applications of secure file transfer, inference, real-time multimedia transmission, and social networking, are used to motivate consideration of four respective research problems involving the wireless physical layer: physical layer security in data networks, distributed inference in sensor networks, finite-blocklength capacity in multimedia networks, and connectivity in small-world networks.

A Coordinate System for Gaussian Networks

This paper investigates network information theory problems where the external noise is Gaussian distributed. In particular, the Gaussian broadcast channel with coherent fading and the Gaussian interference channel are considered. It is shown that in these problems, non-Gaussian code ensembles can achieve higher rates than the Gaussian ones. It is also shown that the strong Shamai-Laroia conjecture on the Gaussian ISI channel does not hold. In order to analyze non-Gaussian code ensembles over Gaussian networks, a geometrical tool using the Hermite polynomials is proposed. This tool provides a coordinate system to analyze a class of non-Gaussian input distributions that are invariant over Gaussian networks.

When a quantum measurement can be implemented locally, and when it cannot

Local operations on subsystems and classical communication between parties (LOCC) constitute the most general protocols available on spatially separated quantum systems. Every LOCC protocol implements a separable generalized measurement -- a complete measurement for which every outcome corresponds to a tensor product of operators on individual subsystems -- but it is known that there exist separable measurements that cannot be implemented by LOCC. A longstanding problem in quantum information theory is to understand the difference between LOCC and the full set of separable measurements. In this paper, we show how to construct an LOCC protocol to implement an arbitrary separable measurement, except that with those measurements for which no LOCC protocol exists, the method shows explicitly that this is the case.

Error Exponents for Joint Source-Channel Coding With Side Information

In this paper, we study upper and lower bounds on the error exponents for joint source-channel coding with decoder side-information. The results in the paper are nontrivial extensions of Csiszar's classical paper “Joint Source-Channel Error Exponent”, Problems of Control and Information Theory, 1980. Unlike the joint source-channel coding result in Csiszar's paper, it is not obvious whether the lower bound and the upper bound are equivalent even if the channel coding error exponent is known. For a class of channels, including symmetric channels, we apply a game-theoretic result to establish the existence of a saddle point and, hence, prove that the lower and upper bounds are the same if the channel coding error exponent is known. More interestingly, we show that encoder side-information does not increase the error exponents in this case.

THE QUANTUM CAPACITY BOUNDS OF A LOSSY GAUSSIAN QUANTUM CHANNEL

Quantum capacity of the lossy Gaussian quantum channel remains an open problem in quantum information theory, although the upper and lower bounds are well-known. We show that for the n-use of the channel, the input of entangled commutative Gaussian state does not improve the lower bound of the capacity. When the total energy is limited, an unfair distribution of the energy among the n-use will improve the lower bound.