Model Selection Indices Research Articles

Discrimination of rival isotherm equations for aqueous contaminant removal systems

Two different model selection indices, the Akaike information criterion (AIC) and the coefficient of determination (<TEX>$R^2$</TEX>), are used to discriminate competing isotherm equations for aqueous pollutant removal systems. The former takes into account model accuracy and complexity while the latter considers model accuracy only. The five types of isotherm shape in the Brunauer-Deming-Deming-Teller (BDDT) classification are considered. Sorption equilibrium data taken from the literature were correlated using isotherm equations with fitting parameters ranging from two to five. For the isotherm shapes of types I (favorable) and III (unfavorable), the AIC favors two-parameter equations which can easily track these simple isotherm shapes with high accuracy. The <TEX>$R^2$</TEX> indicator by contrast recommends isotherm equations with more than two parameters which can provide marginally better fits than two-parameter equations. To correlate the more intricate shapes of types II (multilayer), IV (two-plateau) and V (S-shaped) isotherms, both indices favor isotherm equations with more than two parameters.

Combining Diffusion Models and Macroeconomic Indicators with a Modified Genetic Programming Method: Implementation in Forecasting the Number of Mobile Telecommunications Subscribers in OECD Countries

This paper proposes a modified Genetic Programming method for forecasting the mobile telecommunications subscribers’ population. The method constitutes an expansion of the hybrid Genetic Programming (hGP) method improved by the introduction of diffusion models for technological forecasting purposes in the initial population, such as the Logistic, Gompertz, and Bass, as well as the Bi-Logistic and LogInLog. In addition, the aforementioned functions and models expand the function set of hGP. The application of the method in combination with macroeconomic indicators such as Gross Domestic Product per Capita (GDPpC) and Consumer Prices Index (CPI) leads to the creation of forecasting models and scenarios for medium- and long-term level of predictability. The forecasting module of the program has also been improved with the multi-levelled use of the statistical indices as fitness functions and model selection indices. The implementation of the modified-hGP in the datasets of mobile subscribers in the Organisation for Economic Cooperation and Development (OECD) countries shows very satisfactory forecasting performance.

A Skew-Normal Mixture Regression Model

A challenge associated with traditional mixture regression models (MRMs), which rest on the assumption of normally distributed errors, is determining the number of unobserved groups. Specifically, even slight deviations from normality can lead to the detection of spurious classes. The current work aims to (a) examine how sensitive the commonly used model selection indices are in class enumeration of MRMs with nonnormal errors, (b) investigate whether a skew-normal MRM can accommodate nonnormality, and (c) illustrate the potential of this model with a real data analysis. Simulation results indicate that model information criteria are not useful for class determination in MRMs unless errors follow a perfect normal distribution. The skew-normal MRM can accurately identify the number of latent classes in the presence of normal or mildly skewed errors, but fails to do so in severely skewed conditions. Furthermore, across the experimental conditions it is seen that some parameter estimates provided by the skew-normal MRM become more biased as skewness increases whereas others remain unbiased. Discussion of these results in the context of the applicability of skew-normal MRMs is provided.

MPTinR: Analysis of multinomial processing tree models in R

We introduce MPTinR, a software package developed for the analysis of multinomial processing tree (MPT) models. MPT models represent a prominent class of cognitive measurement models for categorical data with applications in a wide variety of fields. MPTinR is the first software for the analysis of MPT models in the statistical programming language R, providing a modeling framework that is more flexible than standalone software packages. MPTinR also introduces important features such as (1) the ability to calculate the Fisher information approximation measure of model complexity for MPT models, (2) the ability to fit models for categorical data outside the MPT model class, such as signal detection models, (3) a function for model selection across a set of nested and nonnested candidate models (using several model selection indices), and (4) multicore fitting. MPTinR is available from the Comprehensive R Archive Network at http://cran.r-project.org/web/packages/MPTinR/ .

Model selection information criteria in latent class models with missing data and contingency question

Latent class analysis (LCA) has been found to have important applications in social and behavioural sciences for modelling categorical response variables, and non-response is typical when collecting data. In this study, the non-response mainly included ‘contingency questions’ and real ‘missing data’. The primary objective of this study was to evaluate the effects of some potential factors on model selection indices in LCA with non-response data. We simulated missing data with contingency question and evaluated the accuracy rates of eight information criteria for selecting the correct models. The results showed that the main factors are latent class proportions, conditional probabilities, sample size, the number of items, the missing data rate and the contingency data rate. Interactions of the conditional probabilities with class proportions, sample size and the number of items are also significant. From our simulation results, the impact of missing data and contingency questions can be amended by increasing the sample size or the number of items.

Model Selection Indices for Polytomous Items

Model Selection Indices for Polytomous Items

Model Selection Indices for Polytomous Items

This study examines the utility of four indices for use in model selection with nested and nonnested polytomous item response theory (IRT) models: a cross-validation index and three information-based indices. Four commonly used polytomous IRT models are considered: the graded response model, the generalized partial credit model, the partial credit model, and the rating scale model. In a simulation study, comparisons among the four indices suggest that model selection is dependent to some extent on the particular conditions simulated. Overall, the Bayesian information criterion index appears to be most accurate in selecting the correct polytomous IRT model. Results are presented from analysis of a real data set to illustrate the use of the four indices for selecting an appropriate model.

Model Selection Methods for Mixture Dichotomous IRT Models

This study examines model selection indices for use with dichotomous mixture item response theory (IRT) models. Five indices are considered: Akaike's information coefficient (AIC), Bayesian information coefficient (BIC), deviance information coefficient (DIC), pseudo-Bayes factor (PsBF), and posterior predictive model checks (PPMC). The five indices provide somewhat different recommendations for a set of real data. Results from a simulation study indicate that BIC selects the correct (i.e., the generating) model well under most conditions simulated and for all three of the dichotomous mixture IRT models considered. PsBF is almost as effective. AIC and PPMC tend to select the more complex model under some conditions. DIC is least effective for this use.

Hierarchical models in ecology: confidence intervals, hypothesis testing, and model selection using data cloning

Hierarchical statistical models are increasingly being used to describe complex ecological processes. The data cloning (DC) method is a new general technique that uses Markov chain Monte Carlo (MCMC) algorithms to compute maximum likelihood (ML) estimates along with their asymptotic variance estimates for hierarchical models. Despite its generality, the method has two inferential limitations. First, it only provides Wald-type confidence intervals, known to be inaccurate in small samples. Second, it only yields ML parameter estimates, but not the maximized likelihood values used for profile likelihood intervals, likelihood ratio hypothesis tests, and information-theoretic model selection. Here we describe how to overcome these inferential limitations with a computationally efficient method for calculating likelihood ratios via data cloning. The ability to calculate likelihood ratios allows one to do hypothesis tests, construct accurate confidence intervals and undertake information-based model selection with hierarchical models in a frequentist context. To demonstrate the use of these tools with complex ecological models, we reanalyze part of Gause's classic Paramecium data with state-space population models containing both environmental noise and sampling error. The analysis results include improved confidence intervals for parameters, a hypothesis test of laboratory replication, and a comparison of the Beverton-Holt and the Ricker growth forms based on a model selection index.

A comparison of model selection indices for nested latent class models

Latent Class analysis has been used to study the hierarchical relationship among sets of categorical variables. Researchers routinely use chi-squared statistics as model-selection criteria. Due to the limitation of chi-squared statistics, it is desirable to develop other model selection indices. In this study, we compared the performance of chi-squared statistics with three information criteria, Akaike's AIC, Schwarz's BIC and Bozdogan's CAIC. The factors actually manipulated in this study were types of latent class model and conditional response probabilities including intrusion and omission error rates for certain models.

A comparison of model selection indices for nested latent class models

Latent Class analysis has been used to study the hierarchical relationship among sets of categorical variables. Researchers routinely use chi-squared statistics as model-selection criteria. Due to the limitation of chi-squared statistics, it is desirable to develop other model selection indices. In this study, we compared the performance of chi-squared statistics with three information criteria, Akaike's AIC, Schwarz's BIC and Bozdogan's CAIC. The factors actually manipulated in this study were types of latent class model and conditional response probabilities including intrusion and omission error rates for certain models. In terms of the performance of the six models, BIC and CAIC were most accurate for the Independence, Proctor and Goodman models, AIC was most accurate for the Extended Intrusion-Omission Error model, while chi-squared difference statistics showed their best performance in some settings of the Extended Proctor model. As for the Intrusion-Omission Error model and some situations in the Extend Proctor model, the performance of the five indices depended on sample sizes and error rates. AIC was preferred in Intrusion-Omission Error model when sample size was small and BIC and CAIC were preferred when sample size was medium. In Proctor model, AIC was more accurate than chi-squared difference statistics when error rate was small. A similar effect occurred in Hierarchy I in Extended Proctor model. With a small error rate, AIC was superior to chi-squared difference statistics but worse with a large error rate. None of the five criteria performed well in Extended Proctor and Extended Intrusion-Omission Error models. In conclusion, the information criteria had superior performance than both chi-squared statistics. Among the information criteria, BIC and CAIC were more appropriate than AIC for the Independence and Proctor models while AIC was more accurate for Extended Proctor model.

Joint Effects of Density Dependence and Rainfall on Abundance of San Joaquin Kit Fox

We analyzed time-series abundances of San Joaquin kit fox estimated during 1983-95 on the Naval Petroleum Reserves in California (NPRC). For the analysis, we modified a model of density-dependent, stochastic population growth to include the lagged effects of a weather covariate (vegetation growing season rainfall). Without the covariate in the model, a statistical test failed to detect significant density dependence in fluctuating kit fox abundances. However, when the covariate was added, strong density dependence was detected. According to an information-theoretic model-selection index, the model with both density dependence and rainfall is far superior to the models that result from deleting one or more of these factors. The 2-year time lag in the response of kit fox abundances to changes in rainfall is consistent with biological expectations of how rainfall affects habitat carrying capacity for kit fox. An additional covariate, a coyote abundance index, failed to improve the model. A population viability analysis (PVA) performed with the combined density dependence-rainfall model suggests that the San Joaquin kit fox on NPRC could face a risk of up to 52% of falling to low levels within 20 years.