Information theoretic methods of spatial model building: A guide to the unbiased estimation of the form of probability distributions†

Abstract

The methods of maximum entropy and minimum information are employed in order to provide unbiased a priori estimates of the form of probability distributions, given limited prior knowledge. These a priori estimates, in turn, can be cast in the form of theoretical models of the phenomena of interest; models to be tested against real world data. As such, the methods of maximum entropy and minimum information have relevance to any scientific or social-scientific discipline wherein the mathematical modelling of probability distributions is a concern. At the same time, however, the methods are mathematically difficult and conceptually demanding. This paper therefore presents a pedagogic introduction to the methods of maximum entropy and minimum information in the hope of making them more accessible to those social scientists who might find them relevant to their work. Using a simple example, the paper works through the two methods, one at a time, adding more and more information to the estimation problem as the discussion proceeds. The focus herein is a geographic one, yet the example employed is easily transferable to alternative contexts.