Conditional Akaike Information Criterion Research Articles

Measuring complexity for hierarchical models using effective degrees of freedom

AbstractHierarchical models can express ecological dynamics using a combination of fixed and random effects, and measurement of their complexity (effective degrees of freedom, EDF) requires estimating how much random effects are shrunk toward a shared mean. Estimating EDF is helpful to (1) penalize complexity during model selection and (2) to improve understanding of model behavior. I applied the conditional Akaike Information Criterion (cAIC) to estimate EDF from the finite‐difference approximation to the gradient of model predictions with respect to each datum. I confirmed that this has similar behavior to widely used Bayesian criteria, and I illustrated ecological applications using three case studies. The first compared model parsimony with or without time‐varying parameters when predicting density‐dependent survival, where cAIC favors time‐varying demographic parameters more than conventional Akaike Information Criterion. The second estimates EDF in a phylogenetic structural equation model, and identifies a larger EDF when predicting longevity than mortality rates in fishes. The third compares EDF for a species distribution model fitted for 20 bird species and identifies those species requiring more model complexity. These highlight the ecological and statistical insight from comparing EDF among experimental units, models, and data partitions, using an approach that can be broadly adopted for nonlinear ecological models.

Construction of CPUE standardization model and its simulation testing for chub mackerel (Scomber japonicus) in the Northwest Pacific Ocean

Standardized catch per unit effort (CPUE) data not only yield precise and biologically meaningful abundance indices but also provide crucial insights into the spatio-temporal distribution of fisheries resources, critical for their sustainable utilization and management. In this study, we integrated chub mackerel fishery statistics from the Northwest Pacific Ocean (NPO) and marine remote sensing environmental data, and constructed CPUE standardization models for chub mackerel based on the generalized linear mixed model (GLMM) and spatio-temporal GLMM (Vector Autoregressive Spatio-Temporal, VAST) and evaluated their performance. GLMM is a statistical technique extending GLM with random effects for complex datasets with non-independent observations and correlations or hierarchies. The VAST model is a statistical model for multivariate time series data with observations at different spatial locations, displaying both temporal autocorrelation and spatial dependence. Additionally, influence analysis and simulation testing were employed to quantify the effects of explanatory variables on the differences between nominal CPUE and standardized CPUE, and to assess the estimation accuracy of different models in CPUE standardization, respectively. Finally, the distribution pattern of chub mackerel in the NPO was analyzed by estimating centers of gravity (COGs). The results indicated that: 1) The GLMM model containing all explanatory variables was considered the best model with the smallest conditional Akaike Information Criterion (CAIC), while the VAST model, with only the covariate SST, performed the best. 2) Interactions of year and spatial effects in GLMM and the spatio-temporal variable in VAST, exerted a notable influence on annual standardized CPUE and should be accounted for in the model. 3) Based on simulation testing results, the VAST model exhibited lower model error (root mean square error, RMSE) and model bias, outperforming the GLMM model in CPUE standardization. 4) From 2014 to 2018, chub mackerel exhibited high biomass density widely distributed throughout almost the entire NPO, which shifted to the central-north region from 2019 to 2021. There was no significant northing and easting shift in COGs for the population (P > 0.05). The findings of this research enhance our understanding of the variations in resource abundance indices and spatio-temporal distribution patterns of chub mackerel in the NPO.

Variable selection using conditional AIC for linear mixed models with data-driven transformations

When data analysts use linear mixed models, they usually encounter two practical problems: (a) the true model is unknown and (b) the Gaussian assumptions of the errors do not hold. While these problems commonly appear together, researchers tend to treat them individually by (a) finding an optimal model based on the conditional Akaike information criterion (cAIC) and (b) applying transformations on the dependent variable. However, the optimal model depends on the transformation and vice versa. In this paper, we aim to solve both problems simultaneously. In particular, we propose an adjusted cAIC by using the Jacobian of the particular transformation such that various model candidates with differently transformed data can be compared. From a computational perspective, we propose a step-wise selection approach based on the introduced adjusted cAIC. Model-based simulations are used to compare the proposed selection approach to alternative approaches. Finally, the introduced approach is applied to Mexican data to estimate poverty and inequality indicators for 81 municipalities.

Evaluation of the influence of spatial treatments on catch-per-unit-effort standardization: A fishery application and simulation study of Pacific saury in the Northwestern Pacific Ocean

Fishery-dependent catch-per-unit-effort (CPUE) data often exhibit spatial heterogeneity over space and time, which means that the spatial treatment in statistical models used to standardize CPUE is critically important. We evaluated several spatial treatments to standardize CPUE data using Generalized Linear Mixed Models (GLMMs). Results include a real-world application and a simulation based on the Taiwanese stick-held dip net fishery for Pacific saury in the Northwestern Pacific Ocean. We compared the performance of three spatially stratified approaches in GLMMs, (i) Ad hoc; (ii) Binary (binary recursive area partitioning based on model selection criteria); and (iii) Spatial clustering (partitioning of grids into discrete strata based on the spatial proximity and average CPUE in each grid), to a spatio-temporal GLMM (VAST). An influence analysis was constructed to quantify discrepancies between unstandardized and standardized indices that assisted in identifying the annual influence of explanatory variables in GLMMs. We developed a simulation to corroborate the results from the case study and evaluated the four spatial treatments using data generated from two contrasted, random and preferential, sampling scenarios. Results from the real-world application indicated that VAST was statistically superior to the other approaches, based on conditional deviance explained, conditional Akaike Information Criterion, and five-fold cross-validations. The influence analysis indicated that the interaction of year and spatial effect or spatio-temporal variable had a major influence on the standardized CPUE. Both simulation scenarios showed that VAST performed the best, with the lowest model error (measured by root mean square error) and bias, for estimating relative abundance indices. Although the spatial clustering approach created a flexible shape for the area strata, the simulation results under preferential samplings showed that clustering with a stronger emphasis placed on average CPUE could lead to bias in estimated abundance indices. However, spatial clustering that balanced average CPUE with spatial proximity could be a reasonable alternative if it is not possible to apply a spatio-temporal approach. The importance of conducting influence analysis and the greater performance of a spatio-temporal approach are highlighted.

Estimation and selection for spatial confounding regression models

The spatial random effects model is popular in analyzing spatially referenced data sets. The model includes spatially observed covariates and unobserved spatial random effects. If spatial confounding between covariates and random effects is ignored, parameter estimation and spatial prediction would be inaccurate. In this research, we focus on discussing the estimation of regression coefficients and the selection of covariates for spatial regression when existing unmeasured confounders. First, we introduce an adjusted estimation method for regression coefficients and the consequent spatial predictor in the presence of spatial confounding. From a prediction point of view, we then propose a generalized conditional Akaike information criterion to select a subset of covariates, resulting in satisfactory variable selection and spatial prediction. Statistical inferences of the proposed methodology are justified theoretically and numerically.

Asymptotic Equivalence between Cross-Validations and Akaike Information Criteria in Mixed-Effects Models

For model selection in mixed effects models, Vaida and Blan chard (2005) demonstrated that the marginal Akaike information criterion is appropriate as to the questions regarding the population and the conditional Akaike information criterion is appropriate as to the questions regarding the particular clusters in the data. This article shows that the marginal Akaike information criterion is asymptotically equivalent to the leave-one-cluster-out cross-validation and the conditional Akaike information criterion is asymptotically equivalent to the leave-one-observation-out cross-validation.

New model-averaged estimators of concordance correlation coefficients: simulation and application to longitudinal overdispersed Poisson data

The concordance correlation coefficient (CCC) is a common tool to assess agreement among multiple observers for continuous and discrete responses. However, previous results in the statistical literature have shown that the CCC estimators may suffer from a bias problem under a misspecified model for normal data. In order to avoid fitting data with a misspecified model, thus yielding biased CCC estimates for longitudinal overdispersed Poisson data, this research proposes new model-averaged estimators of CCC by combining the estimators of the variance components (VC) approach with model selection via corrected conditional Akaike information criterion (CAICC) and corrected conditional Bayesian information criterion (CBICC) measures under extended overdispersed three-way Poisson mixed-effects models. In simulation studies, the performance of the proposed model-averaged estimators is compared with the VC estimators with and without model selection via CAICC and CBICC and other existing model-averaged estimators for longitudinal Poisson and overdispersed Poisson data sets. An application of corticospinal diffusion tensor tractography study is presented for illustration. It can be concluded that the proposed model-averaged approach is a reliable procedure yielding small mean square errors and nominal 95% coverage rates. Therefore, the new model-averaged estimator is more robust to model misspecification than other competitors.

Conditional Model Selection in Mixed-Effects Models with cAIC4

Model selection in mixed models based on the conditional distribution is appropriate for many practical applications and has been a focus of recent statistical research. In this paper we introduce the R package cAIC4 that allows for the computation of the conditional Akaike information criterion (cAIC). Computation of the conditional AIC needs to take into account the uncertainty of the random effects variance and is therefore not straightforward. We introduce a fast and stable implementation for the calculation of the cAIC for (generalized) linear mixed models estimated with lme4 and (generalized) additive mixed models estimated with gamm4. Furthermore, cAIC4 offers a stepwise function that allows for an automated stepwise selection scheme for mixed models based on the cAIC. Examples of many possible applications are presented to illustrate the practical impact and easy handling of the package.

Spatial zero-inflated negative binomial regression models: Application for estimating frequencies of rear-end crashes on Thai highways

Objective: Rear-end crashes are a type of road traffic accident that occurs frequently. Currently, the application of advanced statistical models to predict the frequency of accident numbers has increased because such models enable accuracy in predictions. The study focuses on the application of these statistical models to determine the relationship between explanatory variables and the frequency of rear-end crashes. Method: Data used are rear-end collisions occurring on highways throughout Thailand for the years 2011–2018. The number of rear-end collisions was distributed according to road segments with similar physical characteristics. Spatial correlation was utilized by varying according to the jurisdiction of the Department of Highways. Four models, namely, Poisson regression model, negative binomial model, zero-inflated negative binomial model, and spatial zero-inflated negative binomial (SZINB) model were developed. Results: When compared with the conditional Akaike Information Criterion (cAIC), SIZNB was found to be most suitable for data. Regarding random effect results, the effect of the significance was constant for the variables conditional state and zero state, which covered segment length, number of lanes, and traffic volume. Conclusion: This study can serve as a starting point for researchers interested in applying the spatial model to the analysis of rear-end crashes.

Modeling adaptive response profiles in a vaccine clinical trial

BackgroundVaccine clinical studies typically provide time-resolved data on adaptive response read-outs in response to the administration of that particular vaccine to a cohort of individuals. However, modeling such data is challenged by the properties of these time-resolved profiles such as non-linearity, scarcity of measurement points, scheduling of the vaccine at multiple time points. Linear Mixed Models (LMM) are often used for the analysis of longitudinal data but their use in these time-resolved immunological data is not common yet. Apart from the modeling challenges mentioned earlier, selection of the optimal model by using information-criterion-based measures is far from being straight-forward. The aim of this study is to provide guidelines for the application and selection of LMMs that deal with the challenging characteristics of the typical data sets in the field of vaccine clinical studies.MethodsWe used antibody measurements in response to Hepatitis-B vaccine with five different adjuvant formulations for demonstration purposes. We built piecewise-linear, piecewise-quadratic and cubic models with transformations of the axes with pre-selected or optimized knot locations where time is a numerical variable. We also investigated models where time is categorical and random effects are shared intercepts between different measurement points. We compared all models by using Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Deviance Information Criterion (DIC), variations of conditional AIC and by visual inspection of the model fit in the light of prior biological information.ResultsThere are various ways of dealing with the challenges of the data which have their own advantages and disadvantages. We explain these in detail here. Traditional information-criteria-based measures work well for the coarse selection of the model structure and complexity, however are not efficient at fine tuning of the complexity level of the random effects.ConclusionsWe show that common statistical measures for optimal model complexity are not sufficient. Rather, explicitly accounting for model purpose and biological interpretation is needed to arrive at relevant models.Trial RegistrationClinical trial registration number for this study: NCT00805389, date of registration: December 9, 2008 (pro-active registration).

Nested Hierarchical Functional Data Modeling and Inference for the Analysis of Functional Plant Phenotypes

ABSTRACTIn a plant science Root Image Study, the process of seedling roots bending in response to gravity is recorded using digital cameras, and the bending rates are modeled as functional plant phenotype data. The functional phenotypes are collected from seeds representing a large variety of genotypes and have a three-level nested hierarchical structure, with seeds nested in groups nested in genotypes. The seeds are imaged on different days of the lunar cycle, and an important scientific question is whether there are lunar effects on root bending. We allow the mean function of the bending rate to depend on the lunar day and model the phenotypic variation between genotypes, groups of seeds imaged together, and individual seeds by hierarchical functional random effects. We estimate the covariance functions of the functional random effects by a fast penalized tensor product spline approach, perform multi-level functional principal component analysis (FPCA) using the best linear unbiased predictor of the principal component scores, and improve the efficiency of mean estimation by iterative decorrelation. We choose the number of principal components using a conditional Akaike information criterion and test the lunar day effect using generalized likelihood ratio test statistics based on the marginal and conditional likelihoods. We also propose a permutation procedure to evaluate the null distribution of the test statistics. Our simulation studies show that our model selection criterion selects the correct number of principal components with remarkably high frequency, and the likelihood-based tests based on FPCA have higher power than a test based on working independence. Supplementary materials for this article are available online.

A composite spatial predictor via local criteria under a misspecified model

Spatial prediction and variable selection for the study area are both important issues in geostatistics. If spatially varying means exist among different subareas, globally fitting a spatial regression model for observations over the study area may be not suitable. To alleviate deviations from spatial model assumptions, this paper proposes a methodology to locally select variables for each subarea based on a locally empirical conditional Akaike information criterion. In this situation, the global spatial dependence of observations is considered and the local characteristics of each subarea are also identified. It results in a composite spatial predictor which provides a more accurate spatial prediction for the response variables of interest in terms of the mean squared prediction errors. Further, the corresponding prediction variance is also evaluated based on a resampling method. Statistical inferences of the proposed methodology are justified both theoretically and numerically. Finally, an application of a mercury data set for lakes in Maine, USA is analyzed for illustration.

Use of Continuous Exposure Variables when Examining Dose-Dependent Pharmacological Effects - Application to the Association between Exposure to Higher Statin Doses and the Incidence of Diabetes.

Many observational studies have found an association between the exposure to statins and the increased risk of diabetes, mostly through the use of intent-to-treat (ITT) like exposure measure (EM). ITT like EM may not adequately reflect the mechanism of action by which statins could cause diabetes. To determine if continuous EMs can more accurately reflect the mechanism of action by which statins and incidence of diabetes would be associated than ITT like EM. We obtained a cohort of 404,129 diabetes-free incident statin users from the Québec public drug insurance plan. Patients dispensed with a drug used in the treatment of diabetes or diagnosed with diabetes within 2-years follow-up were defined as cases. Controls were randomly matched to each case on the index date. Three EMs were tested. EM 1: exposure to a high versus low dose statin at baseline (ITT like); EM 2: cumulative standardized statin dose (cSSD) at the index date; and EM 3: cSSD in the 180 days prior to the index date. The optimal EM was selected based upon each model's Akaike's information criterion (AIC). Conditional logistic regressions were used to calculate conditional OR and model AIC. All three EMs identified an increased risk of diabetes among patients exposed to higher statin doses. Model AIC identified EM 3 as the best EM for this association. Our results indicate that higher statin doses increase the risk of diabetes but favour a cumulative reversible diabetogenic effect of statins.

A note on the comparison of mixed-effects models for longitudinal studies

Abstract This paper concerns methods of choosing appropriate models for longitudinal studies. Attention is paid to three criteria: the marginal Akaike Information Criterion (mAIC), the conditional Akaike Information Criterion (cAIC), and the corrected conditional Akaike Information Criterion (ccAIC). We consider these criteria based on an example concerning the effect of storage time and addition of flaxseed (Linum usitatissimum L.) preparations (i.e. ground flaxseeds, defatted flaxseed meal and flaxseed ethanolic extract) on changes in lipid oxidation and fatty acid composition during the storage of liver pâté with partial substitution of fat with flax oil.

Estimation of genetic parameters for early growth trait of turbot (Scophthalmus maximusL.) using molecular relatedness

In this study, a data set of 1140 progenies from 19 families that were composed of 13 half-sib and six full-sib families was used in the estimation of the genetic parameters of weight for juvenile turbot at 100 days post hatch. Sixty progenies were randomly selected from each family, and 20 of these individuals were genotyped at 12 microsatellite loci. Two sorts of relatedness, including pedigree relatedness (PR) from a complete pedigree and average molecular relatedness (AMR) from molecular markers, were compared to explore the feasibility of AMR in selective breeding when only parents were known. Two different animal models, Model 1 and Model 2 which were without and with the addition of maternal and common environmental effects, respectively, were used to estimate the variance component and breeding values for weight using restricted maximum likelihood (REML) method. Thereafter, cross-validation was applied to investigate predictive ability of model and accuracy of breeding values. Pearson correlation analysis showed that AMR was highly correlated (0.91) with PR. According to conditional Akaike Information Criterion, the best models for AMR and PR methods were Model 2 and Model 1 respectively. Heritability estimates from AMR and PR methods based on their best models were 0.19 (±0.06) and 0.66 (±0.17). Cross-validation showed that AMR was comparable to PR in terms of predictive abilities and accuracies of breeding values based on the same model. This study therefore suggests that the AMR can be applied as alternative of PR when only parents were known.

Comparison of concordance correlation coefficient via variance components, generalized estimating equations and weighted approaches with model selection

Variance components (VC) and generalized estimating equations (GEE) are two approaches for estimating concordance correlation coefficients (CCC) adjusting for covariates, and allowing dependency between replicated samples. However, under VC and GEE, a model including all potential explanatory variables may lead to biased parameter estimates. To overcome this problem, the estimation of CCC using VC and GEE approaches, as well as applying the conditional Akaike information criterion (CAIC) and the quasi-likelihood under the independence model criterion (QIC) measures for model selection is applied. The weighted approach which is the most efficient estimator of CCC obtained by combining the estimators from VC and GEE is also proposed. Simulation studies are conducted to compare the performance of the VC and the GEE, both with and without model-selection via CAIC and QIC, respectively, and the weighted approaches for dependent continuous data. Two applications are illustrated: an assessment of conformity between two optometric devices and an evaluation of agreement in degree of myopia for dizygotic twins. To conclude, the CAIC and QIC model-selection procedures embedded in VC and GEE approaches, respectively, can provide more satisfactory results than VC and GEE involving all possible covariates. Furthermore, the weighted approach is a reliable and stable procedure with the smallest mean square errors and nominal 95% coverage rates in estimating CCC.

Modified conditional AIC in linear mixed models

In linear mixed models, the conditional Akaike Information Criterion (cAIC) is a procedure for variable selection in light of the prediction of specific clusters or random effects. This is useful in problems involving prediction of random effects such as small area estimation, and much attention has been received since suggested by Vaida and Blanchard (2005). A weak point of cAIC is that it is derived as an unbiased estimator of conditional Akaike Information (cAI) in the overspecified case, namely in the case that candidate models include the true model. This results in larger biases in the underspecified case that the true model is not included in candidate models. In this paper, we derive the modified cAIC (McAIC) to cover both the underspecified and overspecified cases, and investigate properties of McAIC. It is numerically shown that McAIC has less biases and less prediction errors than cAIC.

A unifying approach to the estimation of the conditional Akaike information in generalized linear mixed models

The conditional Akaike information criterion, AIC, has been frequently used for model selection in linear mixed models. We develop a general framework for the calculation of the conditional AIC for different exponential family distributions. This unified framework incorporates the conditional AIC for the Gaussian case, gives a new justification for Poisson distributed data and yields a new conditional AIC for exponentially distributed responses but cannot be applied to the binomial and gamma distributions. The proposed conditional Akaike information criteria are unbiased for finite samples, do not rely on a particular estimation method and do not assume that the variance-covariance matrix of the random effects is known. The theoretical results are investigated in a simulation study. The practical use of the method is illustrated by application to a data set on tree growth.

Performance evaluation of recent information criteria for selecting multilevel models in Behavioral and Social Sciences

This study was designed to find the best strategy for selecting the correct multilevel model among several alternatives taking into account variables such as intraclass correlation, number of groups (m), group size (n), or others as parameter values and intercept-slope covariance. First, we examine this question in a simulation study and second, to illustrate the behavior of the criteria and to explore the generalizability of the findings, a previously published educational dataset is analyzed. The results showed that none of the selection criteria behaved correctly under all the conditions or was consistently better than the others. The intraclass correlation somewhat affects the performance of all selection criteria, but the extent of this influence is relatively minor compared to sample size, parameter values, and correlation between random effects. A large number of groups appears more important than a large number of individuals per group in selecting the best model (m≥50 and n≥20 is suggested). Finally, model selection tools such as Akaike's Information Criterion (AIC) or the conditional AIC are recommend when it is assumed that random effects are correlated, whereas use of the Schwarz's Bayesian Information Criterion or the consistent AIC are advantageous for uncorrelated random effects.

Model comparison and estimation of genetic parameters for body weight in commercial broilers

Maniatis, G., Demiris, N., Kranis, A., Banos, G. and Kominakis, A. 2013. Model comparison and estimation of genetic parameters for body weight in commercial broilers. Can. J. Anim. Sci. 93: 67–77. The availability of powerful computing and advances in algorithmic efficiency allow for the consideration of increasingly complex models. Consequently, the development and application of appropriate statistical procedures for model evaluation is becoming increasingly important. This paper is concerned with the application of an alternative model determination criterion (conditional Akaike Information Criterion, cAIC) in a large dataset comprising 203 323 body weights of broilers, pertaining to 7 (BW7) and 35 (BW35) days of age. Seven univariate and seven bivariate models were applied. Direct genetic, maternal genetic and maternal environmental (c2) effects were estimated via REML. The model evaluation criteria included conditional Akaike Information Criterion (cAIC), Bayesian Information Criterion (BIC) and the standard Akaike Information Criterion (henceforth marginal; mAIC). According to cAIC the best-fitting model included direct genetic, maternal genetic and c2 effects. Maternal heritabilities were low (0.10 and 0.03) compared to the direct heritabilities (0.17 and 0.21), while c2 was 0.05 and 0.04 for BW7 and BW35, respectively. BIC and mAIC favoured a model that additionally included a direct-maternal genetic covariance, resulting in highly negative direct-maternal genetic correlations (−0.47 and −0.64 for BW7 and BW35, respectively) and higher direct heritabilities (0.25 and 0.28 for BW7 and BW35, respectively). Results suggest that cAIC can select different animal models than mAIC and BIC with different biological properties.