Christopher S. WallaceStatistical and Inductive Inference by Minimum Message Length. Springer (2005). ISBN 038723795X. £46.00. 432 pp. Hardbound.

Abstract

This book is about the Minimum Message Length (MML) Principle, an information-theoretic approach to induction, hypothesis testing, model selection and statistical inference. MML, which can be seen as a mathematically precise version of Occam's Razor, asserts that the ‘best’ explanation of the observed data is the shortest. The book is essentially the manuscript left behind by Professor Chris Wallace when he died on August 7, 2004. Professor Wallace was a remarkably versatile scientist: originally trained as a physicist, he has made significant contributions to various areas of computer science including computer architecture, arithmetic and simulation. Simultaneously, he is the main originator of the MML Principle, which has implications for applied and theoretical statistics as well as for the philosophy of science. MML was developed between 1968 and 2004 in a series of papers by Wallace and several coworkers such as D. Boulton, P. Freeman and D. Dowe. These papers mostly concern practical applications of MML to problems of statistics and machine learning, as well as the mathematical development of the underlying ideas. Since these papers were often quite short and technical, there was much need of an extensive introduction to, and comprehensive overview of, the MML idea. As its first contribution, this book provides such a, most welcome, introduction. Yet its aim and scope are much wider: as its second contribution, the book presents MML as a general theory of inductive inference, and there is extensive discussion of its philosophical foundations and implications. Thus, the book will appeal to a broad audience: the technical part of the book is mainly of interest to researchers in applied machine learning, data mining and statistics who wish to learn about a non-main stream but practically successful, generally applicable statistical method. The philosophical part is interesting for researchers in these fields, as well as philosophers of science, who will enjoy Wallace's original ideas on the foundations of statistical and inductive inference.