A review and comparison of four commonly used Bayesian and maximum likelihood model selection tools

Abstract

Many tools have become available for biologists for evaluating competing ecological models – models may be judged based on the fit to data alone (e.g. likelihood), or more formal statistical criteria may be used. Because of the implied assumptions of each tool, model selection criteria should be chosen a priori for the problem at hand, – a model that is considered ‘good’ in its explanatory power may not be the best choice for a problem that requires prediction. In this paper, I review the behavior and assumptions of the four most commonly used statistical criteria (Akaike's Information Criterion, AIC; Schwarz or Bayesian Information Criterion, BIC; Deviance Information Criterion, DIC; Bayes factors). Second, I illustrate differences in these model selection tools by applying the four criteria to thousands of simulated abundance trajectories. With the simulation model known, I examine whether each of the criteria are useful in selecting models to evaluate simple questions, such as whether time series support evidence of density dependent population growth. Across simulations, the maximum likelihood criteria consistently favored simpler population models when compared to Bayesian criteria. Among the Bayesian criteria, the Bayes factor favored the correct simulation model more frequently than the Deviance Information Criterion. There was considerable uncertainty in the ability of the Bayes factor to discriminate between models, this tool selected the simulation model slightly more frequently than other approaches.