The asymptotic equipartition property (AEP) plays an important role in establishing lossless source coding theorems and asymptotic coding theorems through the concepts of typical sets and typical sequences in information theory. In this paper, we shall study the strong law of large numbers and the AEP for hidden Markov tree models. First, we give a strict definition of hidden Markov tree models, and study their key properties and equivalent characterizations. In fact, hidden Markov tree models are closely related to the tree-indexed Markov chains, therefore, we also introduce the definition of tree-indexed Markov chains and their equivalent properties. We also establish a strong law of large numbers for hidden Markov tree models indexed by a Cayley tree. As corollaries, we obtain some strong laws of large numbers for the parameters and the AEP for these models.

When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are typical. Crucially, the typical states usually represent only a small fraction of the total number of possible states, and yet the characterization of the set of typical states -the typical set -alone is sufficient to describe the macroscopic behavior of a given system. Consequently, the concept of typicality, and the associated Asymptotic Equipartition Property allow for a drastic reduction of the degrees of freedom needed for system's statistical description. The mathematical rationale for such a simplification in the description is due to the phenomenon of concentration of measure. The later emerges for equilibrium configurations thanks to very strict constraints on the underlying dynamics, such as weekly interacting and (almost) independent system constituents. The question naturally arises as to whether the concentration of measure and related typicality considerations can be extended and applied to more general complex systems, and if so, what mathematical structure can be expected in the ensuing generalized thermodynamics. In this paper we illustrate the relevance of the concept of typicality in the toy model context of the "thermalized" coin and show how this leads naturally to Shannon entropy. We also show an intriguing connection: The characterization of typical sets in terms of Rényi and Tsallis entropies naturally leads to the free energy and partition function, respectively, and makes their relationship explicit. Finally, we propose potential ways to generalize the concept of typicality to systems where the standard microscopic assumptions do not hold.

Let {Xn}n∈Z be a stationary process with values in a finite set. In this paper, we present a moving average version of the Shannon–McMillan–Breiman theorem; this generalize the corresponding classical results. A sandwich argument reduced the proof to direct applications of the moving strong law of large numbers. The result generalizes the work by Algoet et. al., while relying on a similar sandwich method. It is worth noting that, in some kind of significance, the indices an and ϕ(n) are symmetrical, i.e., for any integer n, if the growth rate of (an)n∈Z is slow enough, all conclusions in this article still hold true.

We propose a generalization of the asymptotic equipartition property to discrete sources with an ambiguous alphabet, and prove that it holds for irreducible stationary Markov sources with an arbitrary distinguishability relation. Our definition is based on the limiting behavior of graph parameters appearing in a recent dual characterization of the Shannon capacity, evaluated at subgraphs of strong powers of the confusability graph induced on high-probability subsets. As a special case, our results give an information-theoretic interpretation of the graph entropy rate of such sources.

The asymptotic equipartition property (AEP) plays an important role in establishing lossless source coding theorems and asymptotic coding theorems through the concepts of typical sets and typical sequences in information theory. In this paper, we study the generalized asymptotic equipartition property in the form of moving average for N bifurcating Markov chains indexed by an N-branch Cayley tree, which is a special case of Markov Urandom fields. Firstly, we construct a class of random variables containing a parameter with means of 1, and establish a strong limit theorem for the moving average of multivariate functions of such chains using the Borel–Cantelli lemma. Secondly, we present the strong law of large numbers for the frequencies of occurrence of states of the moving average, as well as the generalized asymptotic equipartition property for N bifurcating Markov chains indexed by an N-branch Cayley tree. As corollaries, we also generalize some known results.

Information theoretic perspective on sample complexity

In this paper, we consider passive RIS-assisted multi-user communication between wireless nodes to improve the blocked line-of-sight (LOS) link performance. The wireless nodes are assumed to be equipped with Massive Multiple-Input Multiple-Output antennas, hybrid precoder, combiner, and low-resolution analog-to-digital converters (ADCs). We first derive the expression for the Cramer-Rao lower bound (CRLB) of the Mean Squared Error (MSE) of the received and combined signal at the intended receiver under interference. By appropriate design of the hybrid precoder, combiner, and RIS phase settings, it can be shown that the MSE achieves the CRLB. We further show that minimizing the MSE w.r.t. the phase settings of the RIS is equivalent to maximizing the throughput and energy efficiency of the system. We then propose a novel Information-Directed Branch-and-Prune (IDBP) algorithm to derive the phase settings of the RIS. We, for the first time in the literature, use an information-theoretic measure to decide on the pruning rules in a tree-search algorithm to arrive at the RIS phase-setting solution, which is vastly different compared to the traditional branch-and-bound algorithm that uses bounds of the cost function to define the pruning rules. In addition, we provide the theoretical guarantees of the near-optimality of the RIS phase-setting solution thus obtained using the Asymptotic Equipartition property. This also ensures near-optimal throughput and MSE performance.

The quantum relative entropy is known to play a key role in determining the asymptotic convertibility of quantum states in general resource-theoretic settings, often constituting the unique monotone that is relevant in the asymptotic regime. We show that this is no longer the case when one allows stochastic protocols that may only succeed with some probability, in which case the quantum relative entropy is insufficient to characterize the rates of asymptotic state transformations, and a new entropic quantity based on a regularization of the Hilbert projective metric comes into play. Such a scenario is motivated by a setting where the cost associated with transformations of quantum states, typically taken to be the number of copies of a given state, is instead identified with the size of the quantum memory needed to realize the protocol. Our approach allows for constructing transformation protocols that achieve strictly higher rates than those imposed by the relative entropy. Focusing on the task of resource distillation, we give broadly applicable strong converse bounds on the asymptotic rates of probabilistic distillation protocols, and show them to be tight in relevant settings such as entanglement distillation with nonentangling operations. This generalizes and extends previously known limitations that are only applicable to deterministic protocols. Our methods are based on recent results for probabilistic one-shot transformations as well as a new asymptotic equipartition property for the projective relative entropy.

Abstract A novel Markovian network evolution model is introduced and analysed by means of information theory. It will be proved that the model, called network evolution chain, is a stationary and ergodic stochastic process. Therefore, the asymptotic equipartition property can be applied to it. The model’s entropy rate and typical sequences are also explored. Extracting particular information from the network and methods to simulate network evolution in the continuous time domain are discussed. Additionally, the Erdős–Rényi network evolution chain is introduced as a subset of our model with the additional property of its stationary distribution matching the Erdős–Rényi random graph model. The stationary distributions of nodes and graphs are calculated for this subset alongside its entropy rate. The simulation results at the end of the article back up the proved theorems and calculated values.

We suggest to construct infinite stochastic binary sequences by associating one of the two symbols of the sequence with the renewal times of an underlying renewal process. Focusing on stationary binary sequences corresponding to delayed renewal processes, we investigate correlations and the ability of the model to implement a prescribed autocovariance structure, showing that a large variety of subexponential decay of correlations can be accounted for. In particular, robustness and efficiency of the method are tested by generating binary sequences with polynomial and stretched-exponential decay of correlations. Moreover, to justify the maximum entropy principle for model selection, an asymptotic equipartition property for typical sequences that naturally leads to the Shannon entropy of the waiting time distribution is demonstrated. To support the comparison of the theory with data, a law of large numbers and a central limit theorem are established for the time average of general observables.

Information-theoretic ideas have provided numerous insights in the progress of fundamental physics, especially in our pursuit of quantum gravity. In particular, the holographic entanglement entropy is a very useful tool in studying AdS/CFT, and its efficacy is manifested in the recent black hole page curve calculation. On the other hand, the one-shot information-theoretic entropies, such as the smooth min/max-entropies, are less discussed in AdS/CFT. They are however more fundamental entropy measures from the quantum information perspective and should also play pivotal roles in holography. We combine the technical methods from both quantum information and quantum gravity to put this idea on firm grounds. In particular, we study the quantum extremal surface (QES) prescription that was recently revised to highlight the significance of one-shot entropies in characterizing the QES phase transition. Motivated by the asymptotic equipartition property (AEP), we derive the refined quantum extremal surface prescription for fixed-area states via a novel AEP replica trick, demonstrating the synergy between quantum information and quantum gravity. We further prove that, when restricted to pure bulk marginal states, such corrections do not occur for the higher Rényi entropies of a boundary subregion in fixed-area states, meaning they always have sharp QES transitions. Our path integral derivation suggests that the refinement applies beyond AdS/CFT, and we confirm it in a black hole toy model by showing that the Page curve, for a black hole in a superposition of two radiation stages, receives a large correction that is consistent with the refined QES prescription.

The article obtains large deviation asymptotic for sub-critical communication networks modelled as signal-interference-noise-ratio(SINR) random networks. To achieve this, we define the empirical power measure and the empirical connectivity measure, as well as prove joint large deviation principles(LDPs) for the two empirical measures on two different scales. Using the joint LDPs, we prove an asymptotic equipartition property(AEP) for wireless telecommunication networks modelled as the subcritical SINR random networks. Further, we prove a local large deviation principle(LLDP) for the sub-critical SINR random network. From the LLDPs, we prove the large deviation principle, and a classical McMillan Theorem for the stochastic SINR model processes. Note that, the LDPs for the empirical measures of this stochastic SINR random network model were derived on spaces of measures equipped with the τ- topology, and the LLDPs were deduced in the space of SINR model process without any topological limitations. We motivate the study by describing a possible anomaly detection test for SINR random networks.

In this article, we obtain large deviation asymptotic for supercritical social or communication networks modelled as signal-interference-noise ratio (SINR) graphs. To do this, we define the empirical power measure and the empirical connectivity measure, and prove joint large deviation principles (LDPs) for the two empirical measures on two different scales, i.e. λ and λ 2 aλ , where λ is the intensity measure of the Poisson point process (PPP) which defines the SINR random network and aλ real-valued sequence such that λ 2 aλ → ∞ as λ → ∞Using these joint LDPs we prove an asymptotic equipartition property for the super-critical networks modelled as the SINR random networks. Furthermore, we prove a local large deviation principle (LLDP) for the SINR random network. From the LLDP we prove a large deviation principle, and a classical McMillian theorem for the SINR random network processes. Note that, for a given empirical power measure and typical empirical connectivity measure, qπ ⊗ π, we can deduce from the LLDP a bound on the cardinality of the space of SINR networks to be approximately equal to , where the connectivity probability of the network, , satisfies and qπ ⊗ π is the typical behavior of the empirical connectivity measure. Observe, the LDPs for the empirical measures of SINR random networks were obtained on spaces of measures equipped with the τ-topology, and the LLDPs were obtained in the space of SINR network process without any topological restrictions.

The von Neumann entropy of a quantum state is a central concept in physics and information theory, having a number of compelling physical interpretations. There is a certain perspective that the most fundamental notion in quantum mechanics is that of a quantum channel, as quantum states, unitary evolutions, measurements, and discarding of quantum systems can each be regarded as certain kinds of quantum channels. Thus, an important goal is to define a consistent and meaningful notion of the entropy of a quantum channel. Motivated by the fact that the entropy of a state $\rho$ can be formulated as the difference of the number of physical qubits and the "relative entropy distance" between $\rho$ and the maximally mixed state, here we define the entropy of a channel $\mathcal{N}$ as the difference of the number of physical qubits of the channel output with the "relative entropy distance" between $\mathcal{N}$ and the completely depolarizing channel. We prove that this definition satisfies all of the axioms, recently put forward in [Gour, IEEE Trans. Inf. Theory 65, 5880 (2019)], required for a channel entropy function. The task of quantum channel merging, in which the goal is for the receiver to merge his share of the channel with the environment's share, gives a compelling operational interpretation of the entropy of a channel. The entropy of a channel can be negative for certain channels, but this negativity has an operational interpretation in terms of the channel merging protocol. We define Renyi and min-entropies of a channel and prove that they satisfy the axioms required for a channel entropy function. Among other results, we also prove that a smoothed version of the min-entropy of a channel satisfies the asymptotic equipartition property.

We show that the typical coordinate-wise encoding of multivariate ergodic source into prescribed alphabets has the entropy profile close to the convolution of the entropy profile of the source and the modular polymatroid that is determined by the cardinalities of the output alphabets. We show that the proportion of the exceptional encodings that are not close to the convolution goes to zero doubly exponentially. The result holds for a class of multivariate sources that satisfy asymptotic equipartition property described via the mean fluctuation of the information functions. This class covers asymptotically mean stationary processes with ergodic mean, ergodic processes, irreducible Markov chains with an arbitrary initial distribution. We also proved that typical encodings yield the asymptotic equipartition property for the output variables. These asymptotic results are based on an explicit lower bound of the proportion of encodings that transform a multivariate random variable into a variable with the entropy profile close to the suitable convolution.

Quantum key distribution (QKD) allows for secure communications safe against attacks by quantum computers. QKD protocols are performed by sending a sizeable, but finite, number of quantum signals between the distant parties involved. Many QKD experiments, however, predict their achievable key rates using asymptotic formulas, which assume the transmission of an infinite number of signals, partly because QKD proofs with finite transmissions (and finite-key lengths) can be difficult. Here we develop a robust numerical approach for calculating the key rates for QKD protocols in the finite-key regime in terms of two semi-definite programs (SDPs). The first uses the relation between conditional smooth min-entropy and quantum relative entropy through the quantum asymptotic equipartition property, and the second uses the relation between the smooth min-entropy and quantum fidelity. The numerical programs are formulated under the assumption of collective attacks from the eavesdropper and can be promoted to withstand coherent attacks using the postselection technique. We then solve these SDPs using convex optimization solvers and obtain numerical calculations of finite-key rates for several protocols difficult to analyze analytically, such as BB84 with unequal detector efficiencies, B92, and twin-field QKD. Our numerical approach democratizes the composable security proofs for QKD protocols where the derived keys can be used as an input to another cryptosystem.

Given devices space D, an intensity measure λm ϵ (0, ∞), a transition kernel Q from the space D to positive real numbers ℝ+, a path-loss function (which depends on the Euclidean distance between the devices and a positive constant a), we define a Marked Poisson Point process (MPPP). For a given MPPP and technical constants , we define a Marked Signal-to- Interference and Noise Ratio (SINR) graph, and associate with it two empirical measures; the empirical marked measure and the empirical connectivity measure. For a class of marked SINR graphs, we prove a joint large deviation principle(LDP) for these empirical measures, with speed λ in the τ-topology. From the joint large deviation principle for the empirical marked measure and the empirical connectivity measure, we obtain an Asymptotic Equipartition Property(AEP) for network structured data modelled as a marked SINR graph. Specifically, we show that for large dense marked SINR graph one requires approximately about λ 2 H(Q × Q)/log 2 bits to transmit the information contained in the network with high probability, where H(Q × Q) is a properly defined entropy for the exponential transition kernel with parameter c. Further, we prove a local large deviation principle (LLDP) for the class of marked SINR graphs on D, where , with speed λ from a spectral potential point. From the LLDP we derive a conditional LDP for the marked SINR graphs. Note that, while the joint LDP is established in the τ-topology, the LLDP assume no topological restriction on the space of marked SINR graphs. Observe also that all our rate functions are expressed in terms of the relative entropy or the kullback action or divergence function of the marked SINR on the devices space D.

We ask the question whether entropy accumulates, in the sense that the operationally relevant total uncertainty about an n-partite system A = (A_1, ldots A_n) corresponds to the sum of the entropies of its parts A_i. The Asymptotic Equipartition Property implies that this is indeed the case to first order in n—under the assumption that the parts A_i are identical and independent of each other. Here we show that entropy accumulation occurs more generally, i.e., without an independence assumption, provided one quantifies the uncertainty about the individual systems A_i by the von Neumann entropy of suitably chosen conditional states. The analysis of a large system can hence be reduced to the study of its parts. This is relevant for applications. In device-independent cryptography, for instance, the approach yields essentially optimal security bounds valid for general attacks, as shown by Arnon-Friedman et al. (SIAM J Comput 48(1):181–225, 2019).

In this paper we report a continuous-variable quantum key distribution protocol using multimode coherent states generated on subcarrier frequencies of the optical spectrum. We propose a coherent detection scheme where power from a carrier wave is used as a local oscillator. We compose a mathematical model of the proposed scheme and perform its security analysis in the finite-size regime using fully quantum asymptotic equipartition property technique. We calculate a lower bound on the secret key rate for the system under the assumption that the quantum channel noise is negligible compared to detector dark counts, and an eavesdropper is restricted to collective attacks. Our calculation shows that the current realistic system implementation would allow distributing secret keys over channels with losses up to 9 dB.

We study the general framework of quantum channel simulation, that is, the ability of a quantum channel to simulate another one using different classes of codes. First, we show that the minimum error of simulation and the one-shot quantum simulation cost under no-signalling assisted codes are given by semidefinite programs. Second, we introduce the channel's smooth max-information, which can be seen as a one-shot generalization of the mutual information of a quantum channel. We provide an exact operational interpretation of the channel's smooth max-information as the one-shot quantum simulation cost under no-signalling assisted codes, which significantly simplifies the study of channel simulation and provides insights and bounds for the case under entanglement-assisted codes. Third, we derive the asymptotic equipartition property of the channel's smooth max-information; i.e., it converges to the quantum mutual information of the channel in the independent and identically distributed asymptotic limit. This implies the quantum reverse Shannon theorem in the presence of no-signalling correlations. Finally, we explore the simulation cost of various quantum channels.