Information-theoretic Model Selection Criteria Research Articles

Complexity: Frontiers in Data‐Driven Methods for Understanding, Prediction, and Control of Complex Systems 2022 on the Development of Information Theoretic Model Selection Criteria for the Analysis of Experimental Data

It can be argued that the identification of sound mathematical models is the ultimate goal of any scientific endeavour. On the other hand, particularly in the investigation of complex systems and nonlinear phenomena, discriminating between alternative models can be a very challenging task. Quite sophisticated model selection criteria are available but their deployment in practice can be problematic. In this work, the Akaike Information Criterion is reformulated with the help of purely information theoretic quantities, namely, the Gibbs‐Shannon entropy and the Mutual Information. Systematic numerical tests have proven the improved performances of the proposed upgrades, including increased robustness against noise and the presence of outliers. The same modifications can be implemented to rewrite also Bayesian statistical criteria, such as the Schwartz indicator, in terms of information‐theoretic quantities, proving the generality of the approach and the validity of the underlying assumptions.

Model selection uncertainty and multimodel inference in partial least squares structural equation modeling (PLS-SEM)

Comparing alternative explanations for behavioral phenomena is central to the process of scientific inquiry. Recent research has emphasized the efficacy of Information Theoretic model selection criteria in partial least squares structural equation modeling (PLS-SEM), which has gained massive dissemination in a variety of fields. However, selecting one model over others based on model selection criteria may lead to a false sense of confidence as differences in the criteria values are often small. To overcome this limitation researchers have proposed Akaike weights, whose efficacy however, has not been assessed in the PLS-SEM context yet. Addressing this gap in research, we analyze the efficacy of Akaike weights in PLS-SEM-based model comparison tasks. We find that Akaike weights derived from BIC and GM are well suited for separating incorrectly specified from correctly specified models, and that Akaike weights based on AIC are useful for creating model-averaged predictions under conditions of model selection uncertainty.

Toward a contemporary quantitative model of punishment.

A direct-suppression, or subtractive, model of punishment has been supported as the qualitatively and quantitatively superior matching law-based punishment model (Critchfield, Paletz, MacAleese, & Newland, 2003; de Villiers, 1980; Farley, 1980). However, this conclusion was made without testing the model against its predecessors, including the original (Herrnstein, 1961) and generalized (Baum, 1974) matching laws, which have different numbers of parameters. To rectify this issue, we reanalyzed a set of data collected by Critchfield et al. (2003) using information theoretic model selection criteria. We found that the most advanced version of the direct-suppression model (Critchfield et al., 2003) does not convincingly outperform the generalized matching law, an account that does not include punishment rates in its prediction of behavior allocation. We hypothesize that this failure to outperform the generalized matching law is due to significant theoretical shortcomings in model development. To address these shortcomings, we present a list of requirements that all punishment models should satisfy. The requirements include formal statements of flexibility, efficiency, and adherence to theory. We compare all past punishment models to the items on this list through algebraic arguments and model selection criteria. None of the models presented in the literature thus far meets all of the requirements.

Attribution in the presence of a long-memory climate response

Abstract. Multiple, linear regression is employed to attribute variability in the global surface temperature to various forcing components and prominent internal climatic modes. The purpose of the study is to asses how sensitive attribution is to long-range memory (LRM) in the model for the temperature response. The model response to a given forcing component is its fingerprint and is different for a zero response time (ZRT) model and one with an LRM response. The fingerprints are used as predictors in the regression scheme to express the response as a linear combination of footprints. For the instrumental period 1880–2010 CE (Common Era) the LRM response model explains 89 % of the total variance and is also favoured by information-theoretic model selection criteria. The anthropogenic footprint is relatively insensitive to LRM scaling in the response and explains almost all global warming after 1970 CE. The solar footprint is weakly enhanced by the LRM response, while the volcanic footprint is reduced by a factor of 2. The natural climate variability on multidecadal timescales has no systematic trend and is dominated by the footprint of the Atlantic Multidecadal Oscillation. The 2000–2010 CE hiatus is explained as a natural variation. A corresponding analysis for the last millennium is performed, using a Northern Hemisphere temperature reconstruction. The Little Ice Age (LIA) is explained as mainly due to volcanic cooling or as a long-memory response to a strong radiative disequilibrium during the Medieval Warm Anomaly, and it is not attributed to the low solar activity during the Maunder Minimum.

What drives long‐distance dispersal? A test of theoretical predictions

Long-distance dispersal (LDD) may contribute disproportionately to species persistence in fragmented landscapes, non-native invasions, and range shifts in response to climate change. However, direct data on LDD are extremely limited, leaving us with little understanding of why it occurs. I used six years of mark-recapture data on the stream salamander Gyrinophilus porphyriticus to test theoretical predictions of how variation in habitat quality affects LDD. Frequency of LDD was quantified using the kurtosis of yearly movement distributions from recaptured animals in a 1-km headwater stream. Temporal and spatial variation in habitat quality were quantified with spatially explicit data on the body condition and dispersion of individuals throughout the study stream. Using information-theoretic model selection criteria, I found that LDD increased during periods of low average body condition and low spatial variation in body condition. Consistent with basic theory, my results indicate that temporal variation in habitat quality is critical to initiating dispersal, and that LDD increases when animals must move farther to encounter higher-quality habitat. This suggests that information on how habitat quality varies in time and space can be useful for predicting LDD. More broadly, this study highlights the value of direct data on animal movement for testing dispersal theory.

Performance of Model Selection Criteria in Bayesian Threshold VAR (TVAR) Models

This article presents a new Bayesian modeling and information-theoretic model selection criteria for threshold vector autoregressive (TVAR) models. The analytical framework of Bayesian modeling for threshold VAR models are developed. Markov Chain Monte Carlo (MCMC) simulation and importance/rejection sampling methods are used to estimate the parameters of the model and to obtain posterior samples. We propose reliable modeling procedures using Bayes factor, and the information-theoretic model selection criteria such as, Akaike's (1973) Information Criterion (AIC), Schwarz (1978) Bayesian Criterion (SBC), Information Complexity (ICOMP) Criterion of Bozdogan (1990, 1994, 2000), Extended Consistent (AIC) with Fisher Information (CAICF E ), and the new Bayesian Model Selection (BMS) Criterion of Bozdogan and Ueno (2000). We study the performance of these criteria under different design of the simulation protocol with varying sample sizes in TVAR models. Our results show that these criteria perform well in small sample as well as large samples to avoid heavy computational burden in conventional procedures.

ENCOUNTER, SURVIVAL, AND MOVEMENT PROBABILITIES FROM AN ATLANTIC PUFFIN (FRATERCULA ARCTICA) METAPOPULATION

Several weaknesses in our understanding of long-lived animal populations have persisted, mainly due to a prevalence of studies of a single local population at the expense of multisite studies. We performed a multisite capture–mark–resight analysis using 2050 Atlantic Puffins (Fratercula arctica) banded as chicks on four islands (colonies) over 24 years in the Gulf of Maine, USA and Canada. Within program MARK, encounter, apparent survival, pre-breeding movement (PBM; annual movements between colonies prior to breeding), and natal dispersal (ND) probabilities were modeled as functions of age, colony, and several covariates. Information-theoretic model selection criteria and estimated model effect sizes were used to identify important effects and select models to estimate parameters. Encounter probabilities were extremely variable (0.10–0.95) and declined annually starting six years after bands were applied, due to changes in resighting effort, and band wear, respectively. Colony-dependent survival probabil...

Detecting determinism in time series: the method of surrogate data

We review a relatively new statistical test that may be applied to determine whether an observed time series is inconsistent with a specific class of dynamical systems. These surrogate data methods may test an observed time series against the hypotheses of: i) independent and identically distributed noise; ii) linearly filtered noise; and iii) a monotonic nonlinear transformation of linearly filtered noise. A recently suggested fourth algorithm for testing the hypothesis of a periodic orbit with uncorrelated noise is also described. We propose several novel applications of these methods for various engineering problems, including: identifying a deterministic (message) signal in a noisy time series; and separating deterministic and stochastic components. When employed to separate deterministic and noise components, we show that the application of surrogate methods to the residuals of nonlinear models is equivalent to fitting that model subject to an information theoretic model selection criteria.