Abstract We generalize the convex duality symmetry in Gibbs’ statistical ensemble formulation, between the Gibbs entropy φV,N(E) as a function of mean internal energy E and Massieu’s free entropy ΨV,N(β) as a function of inverse temperature β. The duality in terms of Legendre-Fenchel transform tells us that Gibbs’ thermodynamic entropy is to the law of large numbers (LLN) for arithmetic sample mean values what Shannon’s information entropy is to the LLN for empirical counting frequencies in independent and identically distributed data. Proceeding with the same mathematical logic, we identify the energy of the state {ui } as the conjugate variable to the counting of statistical occurrence {mi } and find a Hamilton-Jacobi equation for the Shannon entropy analogous to an equation of state in thermodynamics. An eigenvalue problem that is reminiscent of certain features in quantum mechanics arises in the entropy theory of statistical counting frequencies of Markov correlated data.
Read full abstract