Building Better Theories: Prediction Intervals as a Tool for Theory Testing and Improvement

Abstract

The precision with which a theory predicts behavior speaks both to the quality of that theory and to its potential utility in real-world applications. Unfortunately, predictive precision is frequently overlooked when evaluating theories in social psychology. Here we call attention to the benefits of placing more emphasis on predictive precision. We define what we mean by predictive precision, consider its role in theory evaluation from a philosophy of science perspective, and suggest a method for quantifying it: by constructing prediction intervals. A prediction interval is the range of values within which an empirical observation (e.g., a sample mean) is expected to fall some specified percentage (e.g., 95%) of the time. The size of a prediction interval reflects the degree of fluctuation in the empirical observations expected by theory, and so the precision of its prediction. Prediction intervals are useful because they simplify theory testing and provide a metric by which theory improvement—and so scientific progress—can be gauged. Prediction intervals are easily created when theories are expressed formally as agent-based models. We illustrate the process of creating and using prediction intervals in three detailed examples, each involving a different agent-based model concerned with the behavior of small interacting groups.