Is There a World Behind Shannon? Entropies for Complex Systems

Abstract

In their seminal works, Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, \(S=-\sum \nolimits _i p_i \log p_i\). In many physical systems one of these axioms may be violated. For non-ergodic systems the so called separation axiom (Shannon-Khinchin axiom 4) is not valid. We show that whenever this axiom is violated the entropy takes a more general form, \(S_{c,d}\propto \sum _i ^W \Gamma (d+1, 1- c \log p_i)\), where \(c\) and \(d\) are scaling exponents and \(\Gamma (a,b)\) is the incomplete gamma function. These exponents \((c,d)\) define equivalence classes for all!, interacting and non interacting, systems and unambiguously characterize any statistical system in its thermodynamic limit. The proof is possible because of two newly discovered scaling laws which any entropic form has to fulfill, if the first three Shannon-Khinchin axioms hold [1]. \((c,d)\) can be used to define equivalence classes of statistical systems. A series of known entropies can be classified in terms of these equivalence classes. We show that the corresponding distribution functions are special forms of Lambert-\(\mathcal{W}\) exponentials containing—as special cases—Boltzmann, stretched exponential, and Tsallis distributions (power-laws). We go on by showing how the dependence of phase space volume \(W(N)\) of a classical system on its size \(N\), uniquely determines its extensive entropy, and in particular that the requirement of extensivity fixes the exponents \((c,d)\), [2]. We give a concise criterion when this entropy is not of Boltzmann-Gibbs type but has to assume a generalized (non-additive) form. We showed that generalized entropies can only exist when the dynamically (statistically) relevant fraction of degrees of freedom in the system vanishes in the thermodynamic limit [2]. These are systems where the bulk of the degrees of freedom is frozen and is practically statistically inactive. Systems governed by generalized entropies are therefore systems whose phase space volume effectively collapses to a lower-dimensional ‘surface’. We explicitly illustrated the situation for binomial processes and argue that generalized entropies could be relevant for self organized critical systems such as sand piles, for spin systems which form meta-structures such as vortices, domains, instantons, etc., and for problems associated with anomalous diffusion [2]. In this contribution we largely follow the lines of thought presented in [1–3].