Entropy as a Fixed Point

Abstract

We present general ideas about the complexity of objects and how complexity can be used to define the information in objects. In essence, the idea is that while complexity is relative to a given class of processes, information is process independent: information is complexity relative to the class of all conceivable processes. We test these ideas on the complexity of classical states. A domain is used to specify the class of processes, and both qualitative and quantitative notions of complexity for classical states emerge. The resulting theory can be used to give new proofs of fundamental results from classical information theory, to give a new characterization of entropy, to derive lower bounds on algorithmic complexity and even to establish new connections between physics and computation. All of this is a consequence of the setting which gives rise to the fixed point theorem: The least fixed point of the copying operator above complexity is information.KeywordsClassical StateShannon EntropyDomain TheoryScott TopologyClassical Information TheoryThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.