Chapter 1 - Introduction to Bayesian Inference

Abstract

This chapter provides an introduction to Bayesian approach to statistics. Usefulness of Bayesian approach derives in large measure from its simplicity. Its simplicity allows the investigation of far more complex models than can be handled by the tools in the classical toolbox. The essential difference between Bayesian and frequentist philosophies is in the use of the term “probability.” Frequentists restrict the application of the term probability to summaries of hypothetical replicate data sets, whereas Bayesians use probability to describe all unknown quantities. The approach Bayes developed extends in a straightforward fashion to analysis of far more complex data and models, including cases where no frequentist method exists. It is particularly appropriate for hierarchical models. The approach also provides mathematically sensible methods of analysis without the need for asymptotic approximations, and a precise inferential system even when dealing with small sample sizes and limited data. Further Bayesian inference is a self-consistent and qualitatively simple system of reasoning. All unknown quantities—be they parameters, latent variables, or predictions—are treated as random variables. Existing knowledge about unknown quantities is summarized and explicit mathematical expression by means of probability distributions is given. Consequently, Bayesian inference provides a formal mechanism for incorporating and updating prior knowledge, and a proper accounting for all sources of uncertainty.