Thermodynamic-like transformations in information theory

Abstract

This article explores the application of thermodynamic and statistical thermodynamic formalism to information theory problems. In particular, the applicability of the transformation theory of thermodynamics is investigated. After a brief tutorial discussion of thermodynamic and statistical thermodynamic methods and concepts, their information theory analogues are developed. Besides information theory entropy, the information theory counterparts of temperature, chemical potential, Helmholtz free energy, etc., are developed and related to conventional information theory concepts such as channel capacity, matching of source and channel, etc. Information theory theorems are proved via the statistical thermodynamic analogue method; and, finally, several problems are formulated and solved using thermodynamic-like transformations. This article is aimed chiefly at bridging the interface between the two disciplines, and is intended to be provocative. Therefore, no attempt has been made to have it be all inclusive.