Output Entropy Research Articles

Classical information capacity of a class of quantum channels

We consider the additivity of the minimal output entropy and the classical information capacity of a class of quantum channels. For this class of channels, the norm of the output is maximized for the output being a normalized projection. We prove the additivity of the minimal output Renyi entropies with entropic parameters α ∊ [0, 2], generalizing an argument by Alicki and Fannes, and present a number of examples in detail. In order to relate these results to the classical information capacity, we introduce a weak form of covariance of a channel. We then identify various instances of weakly covariant channels for which we can infer the additivity of the classical information capacity. Both additivity results apply to the case of an arbitrary number of different channels. Finally, we relate the obtained results to instances of bi-partite quantum states for which the entanglement cost can be calculated.

MINIMUM OUTPUT ENTROPY OF A GAUSSIAN BOSONIC CHANNEL

The entropy at the output of a Bosonic channel is analyzed when coherent fields are randomly added to the signal. Coherent-state inputs are conjectured to provide the minimum output entropy. Supporting physical and mathematical evidence is provided.

ADDITIVITY CONJECTURE AND COVARIANT CHANNELS

In this survey paper we discuss the relation between the minimal output entropy and the χ-capacity for irreducibly covariant quantum channels implying equivalence of the additivity property for both quantities for such channels. The structure of the Weyl-covariant channels is described in detail.

Qubit channels which require four inputs to achieve capacity: Implications for additivity conjectures

An example is given of a qubit quantum channel which requires four inputs to maximize the Holevo capacity. The example is one of a family of channels which are related to 3-state channels. The capacity of the product channel is studied and numerical evidence presented which strongly suggests additivity. The numerical evidence also supports a conjecture about the concavity of output entropy as a function of entanglement parameters. However, an example is presented which shows that for some channels this conjecture does not hold for all input states. A numerical algorithm for finding the capacity and optimal inputs is presented and its relation to a relative entropy optimization discussed.

Note on Multiple Additivity of Minimal Renyi Entropy Output of the Werner-Holevo Channels

We give an elementary self-contained proof that the minimal Renyi entropy output of arbitrary products of channels [Formula: see text] is additive.

Minimum output entropy of bosonic channels: A conjecture

The von Neumann entropy at the output of a bosonic channel with thermal noise is analyzed. Coherent-state inputs are conjectured to minimize this output entropy. Physical and mathematical evidence in support of the conjecture is provided. A stronger conjecture--that output states resulting from coherent-state inputs majorize the output states from other inputs--is also discussed.

Minimum Rényi and Wehrl entropies at the output of bosonic channels

The minimum Renyi and Wehrl output entropies are found for bosonic channels in which the signal photons are either randomly displaced by a Gaussian distribution (classical-noise channel), or in which they are coupled to a thermal environment through lossy propagation (thermal-noise channel). It is shown that the Renyi output entropies of integer orders z>1 and the output Wehrl entropy are minimized when the channel input is a coherent state.

Minimising entropy and mean tracking control for affine nonlinear and non-Gaussian dynamic stochastic systems

The entropy concept in stochastic systems is used to formulate a control algorithm which minimises the closed-loop randomness for a class of nonlinear dynamic stochastic systems and places the output mean value as close as possible to a given value. Since the entropy measures the randomness of stochastic systems in a more general sense than that of the variance measure for Gaussian random variables, the use of entropy here can produce control algorithms which minimise uncertainties for the closed-loop stochastic systems subjected to any bounded random inputs (generally non-Gaussian). The output probability density function of the system is approximated by the recently developed linear B-spline decoupling model, and the dynamic part of the system links the coefficients of the B-spline expansion with a deterministic control input by a nonlinear affine model. To minimise the randomness of the closed-loop system, the entropy of the output probability density function is included in the proposed performance function. By minimising this performance function, a controller is obtained through a first-order approximation of the ‘logarithm’ function involved in the output entropy calculations. An illustrative example is used to show the use of the control algorithm, and encouraging results have been obtained.

Efficient library characterization for high-level power estimation

This paper describes LP-DSM, which is an algorithm used for efficient library characterization in high-level power estimation. LP-DSM characterizes the power consumption of building blocks using the entropy of primary inputs and primary outputs. The experimental results showed that over a wide range of benchmark circuits implemented using full custom design in 0.35-/spl mu/m 3.3 V CMOS process the statistical performance (mean and maximum error) of LP-DSM is comparable or sometimes better than most of the published algorithms. Moreover, it was found that LP-DSM has the lowest prediction sum of squares, which makes it an efficient tool for power prediction. Furthermore, the complexity of the LP-DSM is linear in relation to the number of primary inputs (O(NI)), whereas state of the art published library characterization algorithms have a complexity of O(NI/sup 2/).

Equivalence of Additivity Questions in Quantum Information Theory

We reduce the number of open additivity problems in quantum information theory by showing that four of them are equivalent. We show that the conjectures of additivity of the minimum output entropy of a quantum channel, additivity of the Holevo expression for the classical capacity of a quantum channel, additivity of the entanglement of formation, and strong superadditivity of the entanglement of formation, are either all true or all false.

Equivalence of Additivity Questions in Quantum Information Theory

We reduce the number of open additivity problems in quantum information theory by showing that four of them are equivalent. Namely, we show that the conjectures of additivity of the minimum output entropy of a quantum channel, additivity of the Holevo expression for the classical capacity of a quantum channel, additivity of the entanglement of formation, and strong superadditivity of the entanglement of formation, are either all true or all false.

Do optimal entropy-constrained quantizers have a finite or infinite number of codewords?

An entropy-constrained quantizer Q is optimal if it minimizes the expected distortion D(Q) subject to a constraint on the output entropy H(Q). We use the Lagrangian formulation to show the existence and study the structure of optimal entropy-constrained quantizers that achieve a point on the lower convex hull of the operational distortion-rate function D/sub h/(R) = inf/sub Q/{D(Q) : H(Q) /spl les/ R}. In general, an optimal entropy-constrained quantizer may have a countably infinite number of codewords. Our main results show that if the tail of the source distribution is sufficiently light (resp., heavy) with respect to the distortion measure, the Lagrangian-optimal entropy-constrained quantizer has a finite (resp., infinite) number of codewords. In particular, for the squared error distortion measure, if the tail of the source distribution is lighter than the tail of a Gaussian distribution, then the Lagrangian-optimal quantizer has only a finite number of codewords, while if the tail is heavier than that of the Gaussian, the Lagrangian-optimal quantizer has an infinite number of codewords.

Symbolic dynamics of a chaotic neuron model

Chaotic neurons change their internal state according to a bimodal map, and when they communicate with other neurons their internal state is transformed into one of two separate outputs, firing or resting. We address and investigate the topological entropy of the two-valued output of the chaotic neuron from two different viewpoints: the dependence upon the parameters of the neurons, and the relationship to their threshold. From the former viewpoint, we clarify the mechanism that changes the shift space corresponding to the time series of the neuronal output. From the latter viewpoint, we examine the effect of small fluctuations on the threshold of the chaotic neuron.

Blind signal processing by complex domain adaptive spline neural networks

In this paper, neural networks based on an adaptive nonlinear function suitable for both blind complex time domain signal separation and blind frequency domain signal deconvolution, are presented. This activation function, whose shape is modified during learning, is based on a couple of spline functions, one for the real and one for the imaginary part of the input. The shape control points are adaptively changed using gradient-based techniques. B-splines are used, because they allow to impose only simple constraints on the control parameters in order to ensure a monotonously increasing characteristic. This new adaptive function is then applied to the outputs of a one-layer neural network in order to separate complex signals from mixtures by maximizing the entropy of the function outputs. We derive a simple form of the adaptation algorithm and present some experimental results that demonstrate the effectiveness of the proposed method.

Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem

The entanglement-assisted classical capacity of a noisy quantum channel (C/sub E/) is the amount of information per channel use that can be sent over the channel in the limit of many uses of the channel, assuming that the sender and receiver have access to the resource of shared quantum entanglement, which may be used up by the communication protocol. We show that the capacity C/sub E/ is given by an expression parallel to that for the capacity of a purely classical channel: i.e., the maximum, over channel inputs /spl rho/, of the entropy of the channel input plus the entropy of the channel output minus their joint entropy, the latter being defined as the entropy of an entangled purification of /spl rho/ after half of it has passed through the channel. We calculate entanglement-assisted capacities for two interesting quantum channels, the qubit amplitude damping channel and the bosonic channel with amplification/attenuation and Gaussian noise. We discuss how many independent parameters are required to completely characterize the asymptotic behavior of a general quantum channel, alone or in the presence of ancillary resources such as prior entanglement. In the classical analog of entanglement-assisted communication - communication over a discrete memoryless channel (DMC) between parties who share prior random information - we show that one parameter is sufficient, i.e., that in the presence of prior shared random information, all DMCs of equal capacity can simulate one another with unit asymptotic efficiency.

Additivity of the classical capacity of entanglement-breaking quantum channels

We show that for the tensor product of an entanglement-breaking quantum channel with an arbitrary quantum channel, both the minimum entropy of an output of the channel and the Holevo–Schumacher–Westmoreland capacity are additive. In addition, for the tensor product of two arbitrary quantum channels, we give a bound involving entanglement of formation for the amount of subadditivity (for minimum entropy output) or superadditivity (for classical capacity) that can occur.

Joint entropy maximization in kernel-based topographic maps.

A new learning algorithm for kernel-based topographic map formation is introduced. The kernel parameters are adjusted individually so as to maximize the joint entropy of the kernel outputs. This is done by maximizing the differential entropies of the individual kernel outputs, given that the map's output redundancy, due to the kernel overlap, needs to be minimized. The latter is achieved by minimizing the mutual information between the kernel outputs. As a kernel, the (radial) incomplete gamma distribution is taken since, for a gaussian input density, the differential entropy of the kernel output will be maximal. Since the theoretically optimal joint entropy performance can be derived for the case of nonoverlapping gaussian mixture densities, a new clustering algorithm is suggested that uses this optimum as its "null" distribution. Finally, it is shown that the learning algorithm is similar to one that performs stochastic gradient descent on the Kullback-Leibler divergence for a heteroskedastic gaussian mixture density model.

Comparison of maximum entropy and minimal mutual information in a nonlinear setting

In blind source separation (BSS), two different separation techniques are mainly used: minimal mutual information (MMI), where minimization of the mutual output information yields an independent random vector, and maximum entropy (ME), where the output entropy is maximized. However, it is yet unclear why ME should solve the separation problem, i.e. result in an independent vector. Yang and Amari have given a partial confirmation for ME in the linear case in [18], where they prove that under the assumption of vanishing expectation of the sources ME does not change the solutions of MMI except for scaling and permutation. In this paper, we generalize Yang and Amari's approach to nonlinear BSS problems, where random vectors are mixed by output functions of layered neural networks. We show that certain solution points of MMI are kept fixed by ME if no scaling in all layers is allowed. In general, ME, however, might also change the scaling in the non-output network layers, hence, leaving the MMI solution points. Therefore, we conclude this paper by suggesting that in nonlinear ME algorithms, the norm of all weight matrix rows of each non-output layer should be kept fixed in later epochs during network training.

Minimum entropy control of non-Gaussian dynamic stochastic systems

This paper presents a new method to minimize the closed loop randomness for general dynamic stochastic systems using the entropy concept. The system is assumed to be subjected to any bounded random inputs. Using the recently developed linear B-spline model for the shape control of the system output probability density function, a control input is formulated which minimizes the output entropy of the closed-loop system. Since the entropy is the measure of randomness for a given random variable, this controller can thus reduce the uncertainty of the closed-loop system. A sufficient condition is established to guarantee the local stability of the closed-loop system. It is shown that this minimum entropy control concept generates a minimum variance control when the stochastic system is represented by an ARMAX model which is subjected to Gaussian noises. An illustrative example is utilized to demonstrate the use of the control algorithm, and satisfactory results are obtained.

Black Holes Are One-Dimensional

The holographic principle has revealed that physical systems in 3-D space, black holes included, are basically two-dimensional as far as their information content is concerned. This conclusion is complemented by one sketched here: as far as entropy or information flow is concerned, a black hole behaves as a one-dimensional channel. We define a channel in flat spacetime in thermodynamic terms, and contrast it with common entropy emitting systems. A black hole is more like the former: its entropy output is related to the emitted power as it would be for a one-dimensional channel, and disposal of an information stream down a black hole is limited by the power invested in the same way as for a one-dimensional channel.