Entropy Theorem Research Articles

Algebraic entropy for amenable semigroup actions

We introduce two notions of algebraic entropy for actions of cancellative right amenable semigroups S on discrete abelian groups A by endomorphisms; they extend the classical algebraic entropy for endomorphisms of abelian groups, corresponding to the case S=N. We investigate the fundamental properties of the algebraic entropy and compute it in several examples, paying special attention to the case when S is an amenable group.For actions of cancellative right amenable monoids on torsion abelian groups, we prove the so called Addition Theorem. In the same setting, we see that a Bridge Theorem connects the algebraic entropy with the topological entropy of the dual action by means of Pontryagin duality, so that we derive an Addition Theorem for the topological entropy of actions of cancellative left amenable monoids on totally disconnected compact abelian groups.

Application of Maximum Entropy Theorem in Channel Estimation

Scholars have been highly concerned with the blind channel estimation of multi-carrier modulation transmission systems, and the decoding process of block codes is usually closely related to channel parameters. Therefore, the accurate estimation of channel parameters is an important method to ensure communication accuracy. According to the transmission characteristics of multi-carrier modulated transmission channels, this paper uses the maximum likelihood theorem and the maximum entropy theorem to put forward a blind channel estimation method based on the proposed received signals. In order to verify the effect of this method, this research utilizes the BP decoding algorithm, which is used in LDPC coding, to calculate the symbol error rate and analyze the estimated effect of the transmission system. The simulation results show that the proposed method can estimate channel parameters effectively, and that the symbol error rate of LDPC decoding is almost similar to that of non-blind channel estimation. This implies that the proposed channel estimation method can effectively improve both channel utilization and transmission efficiency.

Binarization Trees and Random Number Generation

An $m$ -extracting procedure produces unbiased random bits from a loaded dice with $m$ faces, and its output rate, the average number of output per input is bounded by the Shannon entropy of the source. This information-theoretic upper bound can be achieved only asymptotically as the input size increases, by certain extracting procedures that we call asymptotically optimal. Although a computationally efficient asymptotically optimal 2-extracting procedure has been known for a while, its counterparts for $m$ -ary input, $m>2$ , was found only recently, and they are still relatively complicated to describe. A binarization takes inputs from an $m$ -faced dice and produce bit sequences to be fed into a binary extracting procedure to obtain random bits. Thus, binary extracting procedures give rise to an $m$ -extracting procedure via a binarization. A binarization is to be called complete, if it preserves the asymptotic optimality, and such a procedure has been proposed by Zhou and Bruck. We show that a complete binarization naturally arises from a binary tree with $m$ leaves. Therefore, there exist complete binarizations in abundance and Zhou-Bruck scheme is an instance of them. We now have a relatively simple way to obtain an asymptotically optimal and computationally efficient $m$ -extracting procedure, from a binary one, because these binarizations are both conceptually and computationally simple. The well-known leaf entropy theorem and a closely related structure lemma play important roles in the arguments.

On Furstenberg's intersection conjecture, self-similar measures, and the $L^q$ norms of convolutions

We study a class of measures on the real line with a kind of self-similar structure, which we call dynamically driven self-similar measures, and contain proper self-similar measures such as Bernoulli convolutions as special cases. Our main result gives an expression for the $L^q$-dimensions of such dynamically driven self-similar measures, under certain conditions. As an application, we settle Furstenberg's long-standing conjecture on the dimension of the intersections of $\times p$ and $\times q$-invariant sets. Among several other applications, we also show that Bernoulli convolutions have an $L^q$ density for all finite $q$, outside of a zero-dimensional set of exceptions. The proof of the main result is inspired by M. Hochman's approach to the dimensions of self-similar measures and his inverse theorem for entropy. Our method can be seen as an extension of Hochman's theory from entropy to $L^q$ norms, and likewise relies on an inverse theorem for the decay of $L^q$ norms of discrete measures under convolution. This central piece of our approach may be of independent interest, and is an application of well-known methods and results in additive combinatorics: the asymmetric version of the Balog-Szemeredi-Gowers Theorem due to Tao-Vu, and some constructions of Bourgain.

The Maximum Deng Entropy

Deng entropy has been proposed to measure the uncertainty degree of basic probability assignment in evidence theory. In this paper, the condition of the maximum of Deng entropy is discussed. According to the proposed theorem of the maximum Deng entropy, we obtain the analytic solution of the maximum Deng entropy, which yields that the most information volume of Deng entropy is bigger than that of the previous belief entropy functions. Some numerical examples are used to illustrate the basic probability assignment with the maximum Deng entropy.

Dimension growth for iterated sumsets

We study dimensions of sumsets and iterated sumsets and provide natural conditions which guarantee that a set F subseteq mathbb {R} satisfies overline{dim }_{mathrm {B}}F+F > overline{dim }_{mathrm {B}}F or even dim _{mathrm {H}}n F rightarrow 1. Our results apply to, for example, all uniformly perfect sets, which include Ahlfors–David regular sets. Our proofs rely on Hochman’s inverse theorem for entropy and the Assouad and lower dimensions play a critical role. We give several applications of our results including an Erdős–Volkmann type theorem for semigroups and new lower bounds for the box dimensions of distance sets for sets with small dimension.

On Hudetz entropy localization

In Rahimi and Riazi (2014) [22], a type of entropy of continuous maps on compact metric spaces is defined via integrating a local entropy map. This average entropy is represented in terms of the entropy in the sense of Dumitrescu, but the equality of these two types of entropy is left as an open problem in [22]. In this paper, we modify the problem stated in [22], by replacing the Dumitrescu entropy by the Hudetz entropy, since in our case, the two previous types of entropy take an infinite value and the problem stated in [22] reduces to a trivial equality. Then we conclude the paper by proving the equality of the modified average entropy and the Hudetz entropy. The key tool is a version of Jaccob's theorem for the Hudetz entropy.

Logical Entropy: Introduction to Classical and Quantum Logical Information Theory.

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about distinctions, differences and distinguishability and is formalized using the distinctions (“dits”) of a partition (a pair of points distinguished by the partition). All the definitions of simple, joint, conditional and mutual entropy of Shannon information theory are derived by a uniform transformation from the corresponding definitions at the logical level. The purpose of this paper is to give the direct generalization to quantum logical information theory that similarly focuses on the pairs of eigenstates distinguished by an observable, i.e., qudits of an observable. The fundamental theorem for quantum logical entropy and measurement establishes a direct quantitative connection between the increase in quantum logical entropy due to a projective measurement and the eigenstates (cohered together in the pure superposition state being measured) that are distinguished by the measurement (decohered in the post-measurement mixed state). Both the classical and quantum versions of logical entropy have simple interpretations as “two-draw” probabilities for distinctions. The conclusion is that quantum logical entropy is the simple and natural notion of information for quantum information theory focusing on the distinguishing of quantum states.

Hierarchical Structure of Generalized Thermodynamic and Informational Entropy.

The present research aimed at discussing the thermodynamic and informational aspects of entropy concept to propose a unitary perspective of its definitions as an inherent property of any system in any state. The dualism and the relation between physical nature of information and the informational content of physical states of matter and phenomena play a fundamental role in the description of multi-scale systems characterized by hierarchical configurations. A method is proposed to generalize thermodynamic and informational entropy property and characterize the hierarchical structure of its canonical definition at macroscopic and microscopic levels of a system described in the domain of classical and quantum physics. The conceptual schema is based on dualisms and symmetries inherent to the geometric and kinematic configurations and interactions occurring in many-particle and few-particle thermodynamic systems. The hierarchical configuration of particles and sub-particles, representing the constitutive elements of physical systems, breaks down into levels characterized by particle masses subdivision, implying positions and velocities degrees of freedom multiplication. This hierarchy accommodates the allocation of phenomena and processes from higher to lower levels in the respect of the equipartition theorem of energy. However, the opposite and reversible process, from lower to higher level, is impossible by virtue of the Second Law, expressed as impossibility of Perpetual Motion Machine of the Second Kind (PMM2) remaining valid at all hierarchical levels, and the non-existence of Maxwell’s demon. Based on the generalized definition of entropy property, the hierarchical structure of entropy contribution and production balance, determined by degrees of freedom and constraints of systems configuration, is established. Moreover, as a consequence of the Second Law, the non-equipartition theorem of entropy is enunciated, which would be complementary to the equipartition theorem of energy derived from the First Law.

Crow Search based Multi-objective Optimization of Irreversible Air Refrigerators

This study proposes the optimum performance of the irreversible air refrigerators through recently developed metaheuristic algorithm called Crow search algorithm by means of finite time thermodynamics. Finite time thermodynamics is based on choosing the optimum pathways for any kind of thermodynamic system in order to reach the maximum efficiency of the thermodynamic cycle. Handful of objectives for assessing the performance of the irreversible air refrigerators such as coefficient of performance (COP), exergetic efficiency (η II ) , ecological coefficient of performance (ECOP), thermoeconomic optimization (F), and thermoecologic optimization functions (ECF) have been successfully applied on the system. Three optimization scenarios have been studied for the multi objective optimization of irreversible air refrigerators. First scenario evaluates the concurrent optimization of objectives including exergetic efficiency (η II ), coefficient of performance (COP), and ecological coefficient of performance (ECOP). In second scenario, coefficient of performance (COP), thermoeconomic parameter (F), and thermoecological coefficient of performance (ECOP) have been simultaneously maximized to retain optimum working point of the cycle. Third case studies the simultaneous optimization of the imposed objectives such as second law efficiency (η II ), coefficient of performance (COP), and thermoecological function (ECF). Widely known decision-making theorems of LINMAP, TOPSIS, and Shannon’s entropy theorem have been applied on the Pareto curve constructed by the non-dominated solutions to decide the most favorable solution on the frontier.

A NOTE ON ENTROPY OF AUTO-EQUIVALENCES: LOWER BOUND AND THE CASE OF ORBIFOLD PROJECTIVE LINES

Entropy of categorical dynamics is defined by Dimitrov–Haiden–Katzarkov–Kontsevich. Motivated by the fundamental theorem of the topological entropy due to Gromov–Yomdin, it is natural to ask an equality between the entropy and the spectral radius of induced morphisms on the numerical Grothendieck group. In this paper, we add two results on this equality: the lower bound in a general setting and the equality for orbifold projective lines.

Introduction to Quantum Logical Information Theory: Talk

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about distinctions, differences, and distinguishability, and is formalized using the distinctions (“dits”) of a partition (a pair of points distinguished by the partition). All the definitions of simple, joint, conditional, and mutual entropy of Shannon information theory are derived by a uniform transformation from the corresponding definitions at the logical level. The purpose of this talk is to outline the direct generalization to quantum logical information theory that similarly focuses on the pairs of eigenstates distinguished by an observable, i.e., “qudits” of an observable. The fundamental theorem for quantum logical entropy and measurement establishes a direct quantitative connection between the increase in quantum logical entropy due to a projective measurement and the eigenstates (cohered together in the pure superposition state being measured) that are distinguished by the measurement (decohered in the postmeasurement mixed state). Both the classical and quantum versions of logical entropy have simple interpretations as “two-draw” probabilities for distinctions. The conclusion is that quantum logical entropy is the simple and natural notion of information for quantum information theory focusing on the distinguishing of quantum states.

An Entropy Rate Theorem for a Hidden Inhomogeneous Markov Chain

Objective: The main object of our study is to extend some entropy rate theorems to a Hidden Inhomogeneous Markov Chain (HIMC) and establish an entropy rate theorem under some mild conditions. Introduction: A hidden inhomogeneous Markov chain contains two different stochastic processes; one is an inhomogeneous Markov chain whose states are hidden and the other is a stochastic process whose states are observable. Materials and Methods: The proof of theorem requires some ergodic properties of an inhomogeneous Markov chain, and the flexible application of the properties of norm and the bounded conditions of series are also indispensable. Results: This paper presents an entropy rate theorem for an HIMC under some mild conditions and two corollaries for a hidden Markov chain and an inhomogeneous Markov chain. Conclusion: Under some mild conditions, the entropy rates of an inhomogeneous Markov chains, a hidden Markov chain and an HIMC are similar and easy to calculate.

A Single Formulation for Uncertainty Propagation in Turbomachinery: SAMBA PC

This work newly proposes an uncertainty quantification (UQ) method named sparse approximation of moment-based arbitrary polynomial chaos (SAMBA PC) that offers a single solution to many current problems in turbomachinery applications. At the moment, every specific case is characterized by a variety of different input types such as histograms (from experimental data), normal probability density functions (PDFs) (design rules) or fat tailed PDFs (for rare events). Thus, the application of UQ requires the adaptation of ad hoc methods for each individual case. A second problem is that parametric PDFs have to be determined for all inputs. This is difficult if only few samples are available. In gas turbines, however, the collection of statistical information is difficult, expensive, and having scarce information is the norm. A third critical limitation is that if using nonintrusive polynomial chaos (NIPC) methods, the number of required simulations grows exponentially with increasing numbers of input uncertainties: the so-called “curse of dimensionality.” It is shown that the fitting of parametric PDFs to small data sets can lead to large bias and the direct use of the available data is more accurate. This is done by propagating uncertainty through several test functions and the computational fluid dynamics (CFD) simulation of a diffuser, highlighting the impact of different PDF fittings on the output. From the results, it is concluded that the direct propagation of the experimental data set is preferable to the fit of parametric distributions if data is scarce. Thus, the suggested method offers an alternative to the maximum entropy theorem to handle scarce data. SAMBA simplifies the mathematical procedure for many different input types by basing the polynomial expansion on moments. Its moment-based expansion automatically takes care of arbitrary combinations of different input data. It is also numerically efficient compared to other UQ implementations. The relationship between the number of random variables and number of simulation is linear (only 21 simulations for ten input random variables are required). It is shown in this paper that SAMBA's algorithm can propagate a high number of input distributions through a set of nonlinear analytic test functions. Doing this, the code needs a very small number of simulations and preserve a 5% error margin. SAMBA's flexibility to handle different forms of input distributions and a high number of input variables is shown on a low-pressure turbine (LPT) blade-based on H2 profile. The relative importance of manufacturing errors in different location of the blade is analyzed.

Introduction to Quantum Logical Information Theory

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about distinctions, differences, and distinguishability, and is formalized as the distinctions of a partition (a pair of points distinguished by the partition). All the definitions of simple, joint, conditional, and mutual entropy of Shannon information theory are derived by a uniform transformation from the corresponding definitions at the logical level. The purpose of this paper is to give the direct generalization to quantum logical information theory that similarly focuses on the pairs of eigenstates distinguished by an observable, i.e., qubits of an observable. The fundamental theorem for quantum logical entropy and measurement establishes a direct quantitative connection between the increase in quantum logical entropy due to a projective measurement and the eigenstates (cohered together in the pure superposition state being measured) that are distinguished by the measurement (decohered in the post-measurement mixed state). Both the classical and quantum versions of logical entropy have simple interpretations as two-draw probabilities. The conclusion is that quantum logical entropy is the simple and natural notion of information for a quantum information theory focusing on the distinguishing of quantum states.

On a generalization of the maximum entropy Theorem of Burg

In this article we introduce some matrix manipulations that allow us to obtain a version of the original Christoffel-Darboux formula, which is of interest in many applications of linear algebra. Using these developments matrix and Jensen’s inequality, we obtain the main result of this proposal, which is the generalization of the maximum entropy theorem of Burg for multivariate processes.

A concept of local metric entropy for finite-time nonautonomous dynamical systems

We introduce a concept of entropy for difference and differential equations which is a local-in-space and transient-in-time version of the classical concept of metric entropy. Based on that, a finite-time (or transient) version of Pesin’s entropy theorem and also an explicit formula of finite-time entropy for 2-D systems are derived.

A new topological entropy-based approach for measuring similarities among piecewise linear functions

In this paper we present a novel methodology based on a topological entropy, the so-called persistent entropy, for addressing the comparison between discrete piecewise linear functions. The comparison is certified by the stability theorem for persistent entropy that is presented here. The theorem is used in the implementation of a new algorithm. The algorithm transforms a discrete piecewise linear function into a filtered simplicial complex that is analyzed via persistent homology and persistent entropy. Persistent entropy is used as a discriminant feature for solving the supervised classification problem of real long-length noisy signals of DC electrical motors. The quality of classification is stated in terms of the area under receiver operating characteristic curve (AUC=93.87%).

Some properties of the growth and of the algebraic entropy of group endomorphisms

Abstract We study the growth of group endomorphisms, a generalization of the classical notion of growth of finitely generated groups, which is strictly related to algebraic entropy. We prove that the inner automorphisms of a group have the same growth type and the same algebraic entropy as the identity automorphism. Moreover, we show that endomorphisms of locally finite groups cannot have intermediate growth. We also find an example showing that the Addition Theorem for algebraic entropy does not hold for endomorphisms of arbitrary groups.

Entropy theorems in classical mechanics, general relativity, and the gravitational two-body problem

In classical Hamiltonian theories, entropy may be understood either as a statistical property of canonical systems, or as a mechanical property, that is, as a monotonic function of the phase space along trajectories. In classical mechanics, there are theorems which have been proposed for proving the non-existence of entropy in the latter sense. We explicate, clarify and extend the proofs of these theorems to some standard matter (scalar and electromagnetic) field theories in curved spacetime, and then we show why these proofs fail in general relativity; due to properties of the gravitational Hamiltonian and phase space measures, the second law of thermodynamics holds. As a concrete application, we focus on the consequences of these results for the gravitational two-body problem, and in particular, we prove the non-compactness of the phase space of perturbed Schwarzschild-Droste spacetimes. We thus identify the lack of recurring orbits in phase space as a distinct sign of dissipation and hence entropy production.