Chapter 8 - Model Selection and Assessment

Abstract

In this chapter, we focus on two basic inference tasks: model selection (which model or models should be favored), and model assessment (do the data appear to be consistent with a particular model). Researchers commonly want to compare competing models to determine “the best” model for their data. For example, this might include comparisons of models with and without a behavior response or models with and without time-dependent encounter rates. To assist readers in making such comparisons, we review some basic strategies of model selection using both likelihood methods (as implemented in the the R package secr) and Bayesian analysis. Specifically, we review a number of methods for “variable selection” problems, when our set of models consists of distinct covariate effects and they represent constraints of some larger model. For classical analysis based on likelihood, model selection by Akaike Information Criterion (AIC) is the standard approach. For Bayesian analysis there are a number of different methods—we cover the use of the deviance information criterion (DIC), the Kuo and Mallick indicator variable selection approach, and for model adequacy, we cover the Bayesian p-value method for assessing goodness-of-fit. We also discuss sensitivity to state space resolution and extent and how to quantify lack-of-fit. As a demonstration of the various approaches that are outlined, we work with data from the wolverine camera trapping study to investigate sex specificity of model parameters and whether there is a behavioral response to encounter. We evaluate whether certain models for encounter probability appear to be adequate descriptions of the data, and we evaluate the uniformity assumption for the underlying point process.