Model Selection Statistics Research Articles

The utility of the Cognitive Reserve Index questionnaire in chronic traumatic brain injury

Objective: This study examined the relationship between cognitive reserve measured with the Cognitive Reserve Index questionnaire (CRIq) and cognitive and functional outcomes in a chronic traumatic brain injury (TBI) cohort compared to a non-TBI cohort. The utility of the CRIq was compared to common proxies of cognitive reserve (premorbid IQ and years of education) in TBI and non-TBI cohorts. Method: Participants were 105 individuals with moderate-severe TBI (10–33 years post injury) and 91 participants without TBI. Cognitive outcome was examined across four cognitive factors; verbal memory, visual ability and memory, executive attention, and episodic memory. Functional outcome was measured using the Glasgow Outcome Scale Extended. The CRIq total score and three subscale scores (education, work, leisure) were examined. Results: In the TBI cohort, associations were identified between two CRIq subscales and cognitive factors (CRIq education and verbal memory; CRIq work and executive attention). There were no associations between CRIq leisure and cognitive outcomes, or between CRIq and functional outcome. Model selection statistics suggested premorbid IQ and years of education provided a better fit than the CRIq for the relationship between cognitive reserve with two cognitive factors and functional outcome, with neither model providing an improved fit for the remaining two cognitive factors. This finding was broadly consistent in the non-TBI cohort. Conclusion: Cognitive reserve contributes significantly to long-term clinical outcomes following moderate-severe TBI. The relationship between cognitive reserve and long-term cognitive and functional outcomes following TBI is best characterised with traditional proxies of cognitive reserve, mainly premorbid IQ, rather than the CRIq.

Dispfit: An R package to estimate species dispersal kernels

Dispersal of organisms is a ubiquitous aspect of the natural world, with wide implications across scales and organization levels. Interest in dispersal has risen sharply over the past 30 years, mostly due to the multiple and rapid global changes ecosystems face. Among the various aspects that may characterize a dispersion event, dispersal distance is considered a key descriptor in a wide variety of studies across taxonomic groups. Typically, dispersal distances are defined in the form of dispersal kernels describing the dispersal distance distribution according to probability density functions. Although numerous methods providing dispersal data exist, there is still a lack of intuitive and comprehensive approaches and tools to estimate dispersal kernels from such data. Here we present the dispfit package, an R software application developed to fill this gap. dispfit fits and compares different families of parameterized functions to describe and predict dispersal distances. It includes 9 well-known and commonly used distributions, computes goodness-of-fit and model selection statistics, and estimate each distribution's parameters, along with their first four moments (mean, standard deviation, skewness, and kurtosis). We describe the main functions included in dispfit and provide an example to illustrate the workflow of the typical analyses performed within the package. We believe that dispfit will critically contribute to improving the modelling of species' dispersal distances, thus enhancing the understanding of the ecological and evolutionary processes involving dispersal movement.

Dissipative unified dark fluid: Observational constraints

We adopt a standard FRW cosmology with a unified scenario, where the usual dark matter and dark energy sectors are replaced by a single dissipative unified dark fluid (DUDF). The equation of state of such fluid can asymptote between two power laws. As a result, it enables fluid to have a smooth transition from dust at early times to dark energy at late times. The dissipation is represented by a bulk viscosity with a constant coefficient, whereas shear viscosity is excluded due to the isotropy of the universe. We performed a likelihood analysis using recent observational datasets from local H0 measurements, Type Ia supernovae, observational Hubble data, baryon acoustic oscillations, and cosmic microwave background to put cosmological constraints on the model. The special case of the non-dissipative unified dark fluid (UDF) is also studied, while a similar analysis is implemented to the ΛCDM model for comparison. Our analysis revealed that between the three analyzed models, the DUDF has the lower χmin2-value. Based on model selection statistics in the form of the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), we compared different models to select the favored one due to the observational data used. Our results revealed that the UDF model has the minimum AIC, whence, due to the AIC, it is the most favorable model for the data. ΔAIC values of models are, then, measured to this model. This difference indicated that the DUDF model is substantial on the level of empirical support. On the other hand, the BIC results still prefer the ΛCDM model. Our dark fluid also alleviates the tension in the Hubble parameter. We got an H0 value of 70.02 kms−1Mpc−1 for the DUDF model and 70.25 kms−1Mpc−1 for the UDF model. These results alleviate the tension to ∼2.2σ for the DUDF model and ∼2.1σ for the UDF model. In addition, we studied the evolution of the deceleration parameter, the effective equation of state parameter, and the density parameter. We also estimated a value for the viscosity of the cosmic fluid. We found that our unified fluid doesn't deviate from the standard ΛCDM model at early times, with the ability to play the role of the cosmological constant by accelerating the universe at late times.

Incidence, Risk Factors, and Outcomes of Severe Hypoxemia After Cardiac Surgery.

BackgroundHypoxemia is common in patients undergoing cardiac surgery, however, few studies about severe hypoxemia (SH) after cardiac surgery exist. The objectives of this study were to clarify the incidence, risk factors, and outcomes of SH after cardiac surgery.MethodsPatients undergoing cardiac surgery from 2016 to 2019 in a single center were enrolled and were divided into two groups based on whether postoperative SH developed. Independent risk factors for SH were identified by univariate and multivariate analysis. Model selection statistics were applied to help determine the most parsimonious final model.ResultsSevere hypoxemia developed in 222 of the 5,323 included patients (4.2%), was associated with poorer clinical outcomes. Six independent risk factors for SH after cardiac surgery were identified by multivariate analysis, such as surgical types, white blood cell (WBC) count, body mass index (BMI), serum albumin, cardiopulmonary bypass (CPB) time, and intraoperative transfusion of red blood cells (RBCs). After comprehensively considering the discrimination, calibration, and simplicity, the most appropriate and parsimonious model was finally established using four predictors, such as WBC count, BMI, CPB time, and intraoperative transfusion of RBCs. A nomogram and a web-based risk calculator based on the final model were constructed to facilitate clinical practice. Patients were stratified into three risk groups based on the nomogram and clinical practice.ConclusionSevere hypoxemia was common after cardiac surgery and was associated with poorer clinical outcomes. A parsimonious final model with good discrimination, calibration, and clinical utility was constructed, which may be helpful for personalized risk assessment and targeted intervention.

Estimating a global MAIDADS demand system considering demography, climate and norms

Based on data mainly from the International Comparison Program for 156 countries, we conduct a global cross-sectional estimation of an extended rank-3 MAIDADS demand system for nineteen commodity groups including agri-food detail for integration in a Computable General Equilibrium model. We render both marginal budget shares and commitment terms depending on the implicit utility level and consider age shares on the population, the Gini-Coefficient, the share of Islamic population, a sea access indicator and mean temperatures as further explanatory variables. We find that especially demographic factors, the share of Islamic population and mean temperature considerably improve model selection statistics and the fit of commodity groups with a low fit in a variant where prices and income only are used. Graphics of the estimated Engel curves, with details for agro-food commodity groups, highlight income dynamics of budget shares.

Null Distribution of the Test Statistic for Model Selection via Marginal Screening: Implications for Multivariate Regression Analysis

Marginal screening (MS) is the computationally simple and commonly used for the dimension reduction procedures. In it, a linear model is constructed for several top predictors, chosen according to the absolute value of marginal correlations with the dependent variable. Importantly, when kpredictors out of mprimary covariates are selected, the standard regression analysis may yield false-positive results if m>> k(Freedman's paradox). In this work, we provide analytical expressions describing null distribution of the test statistics for model selection via MS. Using the theory of order statistics, we show that under MS, the common F-statistic is distributed as a mean of ktop variables out of mindependent random variables having a 21χdistribution. Based on this finding, we estimated critical p-values for multiple regression models after MS, comparisons with which of those obtained in real studies will help researchers to avoid false-positive result. Analytical solutions obtained in the work are implemented in a free Excel spreadsheet program.

Selection of the number of components for finite mixtures of linear mixed models

Over the last decades, linear models have been studied by the scientific community as an important tool of statistical modelling in a great variety of phenomena. However, in many situations the data are grouped according to factors, so the introduction of random effects is required in order to consider the correlation between observations from the same individual, in which case linear mixed models are used. In addition, it is often observed that the data comes from a heterogeneous population, giving rise to situations where the estimation of a single linear model is not sufficient. Therefore, it is necessary to use models that incorporate this unobserved heterogeneity, as is the case of mixture models. Thus, mixtures of linear mixed models allow modelling the heterogeneity among the individuals and, at the same time, to account for correlations between observations from the same individual. Choosing the number of components for mixture models has long been considered as an important but difficult research problem. There is wide variety of literature available on the performance of model selection statistics for determining the number of components in mixture models. In this article, we study the problem of determining the number of components in mixtures of linear mixed models, investigating the performance of various model selection methods. In order to evaluate the methodologies developed, we carry out a simulation study and we illustrate these methodologies using a real data set.

A New Statistic for Selecting the Smoothing Parameter for Polynomial Loglinear Equating Under the Random Groups Design

AbstractSmoothing is designed to yield smoother equating results that can reduce random equating error without introducing very much systematic error. The main objective of this study is to propose a new statistic and to compare its performance to the performance of the Akaike information criterion and likelihood ratio chi‐square difference statistics in selecting the smoothing parameter for polynomial loglinear equating under the random groups design. These model selection statistics were compared for four sample sizes (500, 1,000, 2,000, and 3,000) and eight simulated equating conditions, including both conditions where equating is not needed and conditions where equating is needed. The results suggest that all model selection statistics tend to improve the equating accuracy by reducing the total equating error. The new statistic tended to have less overall error than the other two methods.

Edge Exclusion Tests for Graphical Model Selection: Complex Gaussian Vectors and Time Series

We consider the problem of inferring the conditional independence graph (CIG) of a stationary multivariate Gaussian time series. A $p$ -variate Gaussian time series graphical model associated with an undirected graph with $p$ vertices is defined as the family of time series that obey the conditional independence restrictions implied by the edge set of the graph. In some existing methods, partial coherence has been used as a test statistic for graphical model selection. To test inclusion/exclusion of a given edge in the graph, the test is applied at distinct frequencies, requiring multiple tests and leading to a loss in test power. The nonparametric method of Matsuda uses the Kullback-Leibler divergence measure to define a test statistic which does not need multiple testing to test between two competing models. In this paper we propose a generalized likelihood ratio test (GLRT) based edge exclusion test statistic that also does not need multiple testing. It is computationally significantly faster than the method of Matsuda, and simulations show that we achieve comparable power levels. Our proposed approach is based on a novel formulation of a GLRT based edge exclusion test for $p$ -variate complex Gaussian graphical models (CGGMs); this result is also of independent interest. The computational complexity of the proposed statistic is $\mathcal {O}(p^3)$ compared to $\mathcal {O}(p^5)$ for the existing result. We also apply our time series graphical model selection method to a foreign exchange data set consisting of monthly trends of foreign exchange rates of 10 countries.

Elephant‐mediated compositional changes in riparian canopy trees over more than two decades in northern Botswana

AbstractQuestionsHow has the composition and diversity of canopy tree species in a riparian woodland changed over time? How are the compositional changes related to impact of elephants? Does the composition of juvenile plants indicate that the woodland retains the potential to recover its former composition?LocationNorthern Botswana adjoining the Linyanti River.MethodsWe assessed the species composition of the riparian woodland in 2007/2008 along belt transects, recording living and dead individuals in different size classes plus signs of elephant damage and the presence of juvenile plants. We related this current composition to the composition recorded in a previous survey in 1991/1992 and reconstructed the earlier composition by combining living and dead trees recorded in 1991/1992. We established the association between mortality and impact agent, severity, year and size class using model selection statistics. Changes in species diversity were assessed using the Shannon diversity index.ResultsThe composition of canopy trees changed from the initial dominance of two Acacia spp. towards the current situation with these two species forming <5% of the woodland canopy. Dead trees were strongly associated with severe damage inflicted by elephants, including bark stripping and felling. As the acacia trees declined, elephant impacts shifted onto other canopy tree species. The woodland canopy became progressively more open because recruitment from juvenile and sapling stages to replace trees that had died was also suppressed. Nevertheless, the tree species that had decreased in abundance in the canopy remained abundant as juvenile plants.ConclusionSubstantial changes in woodland composition can occur in the presence of high elephant concentrations because of the selective damage that elephants impose on particular tree species and size classes. The loss of functionally important species may not be reflected by changes in compositional diversity measures.

An R2 statistic for covariance model selection in the linear mixed model

ABSTRACTThe linear mixed model, sometimes referred to as the multi-level model, is one of the most widely used tools for analyses involving clustered data. Various definitions of have been proposed for the linear mixed model, but several limitations prevail. Presently, there is no method to compute for the linear mixed model that accommodates an interpretation based on variance partitioning, a method to quantify uncertainty and produce confidence limits for the statistic, and a capacity to use the statistic to conduct covariance model selection in a manner similar to information criteria. In this article, we introduce such an statistic. The proposed measures the proportion of generalized variance explained by fixed effects in the linear mixed model. Simulated and real longitudinal data are used to illustrate the statistical properties of the proposed and its capacity to be applied to covariance model selection.

Emerge – an empirical model for the formation of galaxies since z ∼ 10

We present EMERGE, an Empirical ModEl for the foRmation of GalaxiEs, describing the evolution of individual galaxies in large volumes from $z\sim10$ to the present day. We assign a star formation rate to each dark matter halo based on its growth rate, which specifies how much baryonic material becomes available, and the instantaneous baryon conversion efficiency, which determines how efficiently this material is converted to stars, thereby capturing the baryonic physics. Satellites are quenched following the delayed-then-rapid model, and they are tidally disrupted once their subhalo has lost a significant fraction of its mass. The model is constrained with observed data extending out to high redshift. The empirical relations are very flexible, and the model complexity is increased only if required by the data, assessed by several model selection statistics. We find that for the same final halo mass galaxies can have very different star formation histories. Nevertheless, the average star formation and accretion rates are in good agreement with models following an abundance matching strategy. Galaxies that are quenched at $z=0$ typically have a higher peak star formation rate compared to their star-forming counterparts. The accretion of stars can dominate the total mass of massive galaxies, but is insignificant for low-mass systems, independent of star-formation activity. EMERGE predicts stellar-to-halo mass ratios for individual galaxies and introduces scatter self-consistently. We find that at fixed halo mass, passive galaxies have a higher stellar mass on average. The intra-cluster-mass in massive haloes can be up to 8 times larger than the mass of the central galaxy. Clustering for star-forming and quenched galaxies is in good agreement with observational constraints, indicating a realistic assignment of galaxies to haloes.

Bayesian dynamic modeling of time series of dengue disease case counts.

The aim of this study is to model the association between weekly time series of dengue case counts and meteorological variables, in a high-incidence city of Colombia, applying Bayesian hierarchical dynamic generalized linear models over the period January 2008 to August 2015. Additionally, we evaluate the model’s short-term performance for predicting dengue cases. The methodology shows dynamic Poisson log link models including constant or time-varying coefficients for the meteorological variables. Calendar effects were modeled using constant or first- or second-order random walk time-varying coefficients. The meteorological variables were modeled using constant coefficients and first-order random walk time-varying coefficients. We applied Markov Chain Monte Carlo simulations for parameter estimation, and deviance information criterion statistic (DIC) for model selection. We assessed the short-term predictive performance of the selected final model, at several time points within the study period using the mean absolute percentage error. The results showed the best model including first-order random walk time-varying coefficients for calendar trend and first-order random walk time-varying coefficients for the meteorological variables. Besides the computational challenges, interpreting the results implies a complete analysis of the time series of dengue with respect to the parameter estimates of the meteorological effects. We found small values of the mean absolute percentage errors at one or two weeks out-of-sample predictions for most prediction points, associated with low volatility periods in the dengue counts. We discuss the advantages and limitations of the dynamic Poisson models for studying the association between time series of dengue disease and meteorological variables. The key conclusion of the study is that dynamic Poisson models account for the dynamic nature of the variables involved in the modeling of time series of dengue disease, producing useful models for decision-making in public health.

Model-based branching point detection in single-cell data by K-branches clustering.

MotivationThe identification of heterogeneities in cell populations by utilizing single-cell technologies such as single-cell RNA-Seq, enables inference of cellular development and lineage trees. Several methods have been proposed for such inference from high-dimensional single-cell data. They typically assign each cell to a branch in a differentiation trajectory. However, they commonly assume specific geometries such as tree-like developmental hierarchies and lack statistically sound methods to decide on the number of branching events.ResultsWe present K-Branches, a solution to the above problem by locally fitting half-lines to single-cell data, introducing a clustering algorithm similar to K-Means. These halflines are proxies for branches in the differentiation trajectory of cells. We propose a modified version of the GAP statistic for model selection, in order to decide on the number of lines that best describe the data locally. In this manner, we identify the location and number of subgroups of cells that are associated with branching events and full differentiation, respectively. We evaluate the performance of our method on single-cell RNA-Seq data describing the differentiation of myeloid progenitors during hematopoiesis, single-cell qPCR data of mouse blastocyst development, single-cell qPCR data of human myeloid monocytic leukemia and artificial data.Availability and implementationAn R implementation of K-Branches is freely available at https://github.com/theislab/kbranches.Supplementary information Supplementary data are available at Bioinformatics online.

Conditional conceptual predictive statistic for mixed model selection

In linear mixed models, making use of the prediction of the random effects, we propose the conditional Conceptual Predictive Statistic for mixed model selection based on a conditional Gauss discrepancy. We define the conditional Gauss discrepancy for measuring the distance between the true model and the candidate model under the conditional mean of response variables. When the variance components are known, the conditional serves as an unbiased estimator for the expected transformed conditional Gauss discrepancy; when the variance components are unknown, the conditional serves as an asymptotically unbiased estimator for the expected transformed conditional Gauss discrepancy. The best linear unbiased predictor (BLUP) is employed for the estimation of the random effects. The simulation results demonstrate that when the true model includes significant fixed effects, the conditional criteria perform effectively in selecting the most appropriate model. The penalty term in the computed by the estimated effective degrees of freedom yields a very good approximation to the penalty term between the target discrepancy and the goodness-of-fit term.

Clarifying hierarchical age–period–cohort models: A rejoinder to Bell and Jones

Previously, Reither et al. (2015) demonstrated that hierarchical age–period–cohort (HAPC) models perform well when basic assumptions are satisfied. To contest this finding, Bell and Jones (2015) invent a data generating process (DGP) that borrows age, period and cohort effects from different equations in Reither et al. (2015). When HAPC models applied to data simulated from this DGP fail to recover the patterning of APC effects, B&J reiterate their view that these models provide “misleading evidence dressed up as science.” Despite such strong words, B&J show no curiosity about their own simulated data—and therefore once again misapply HAPC models to data that violate important assumptions. In this response, we illustrate how a careful analyst could have used simple descriptive plots and model selection statistics to verify that (a) period effects are not present in these data, and (b) age and cohort effects are conflated. By accounting for the characteristics of B&J's artificial data structure, we successfully recover the “true” DGP through an appropriately specified model. We conclude that B&Js main contribution to science is to remind analysts that APC models will fail in the presence of exact algebraic effects (i.e., effects with no random/stochastic components), and when collinear temporal dimensions are included without taking special care in the modeling process. The expanded list of coauthors on this commentary represents an emerging consensus among APC scholars that B&J's essential strategy—testing HAPC models with data simulated from contrived DGPs that violate important assumptions—is not a productive way to advance the discussion about innovative APC methods in epidemiology and the social sciences.

A Bayesian approach to selecting hyperelastic constitutive models of soft tissue

Abstract Hyperelastic constitutive models of soft tissue mechanical behavior are extensively used in applications like computer-aided surgery, injury modeling, etc. While numerous constitutive models have been proposed in the literature, an objective method is needed to select a parsimonious model that represents the experimental data well and has good predictive capability. This is an important problem given the large variability in the data inherent to soft tissue mechanical testing. In this work, we discuss a Bayesian approach to this problem based on Bayes factors. We propose a holistic framework for model selection, wherein we consider four different factors to reliably choose a parsimonious model from the candidate set of models. These are the qualitative fit of the model to the experimental data, evidence values, maximum likelihood values, and the landscape of the likelihood function. We consider three hyperelastic constitutive models that are widely used in soft tissue mechanics: Mooney–Rivlin, Ogden and exponential. Three sets of mechanical testing data from the literature for agarose hydrogel, bovine liver tissue, porcine brain tissue are used to calculate the model selection statistics. A nested sampling approach is used to evaluate the evidence integrals. In our results, we highlight the robustness of the proposed Bayesian approach to model selection compared to the likelihood ratio, and discuss the use of the four factors to draw a complete picture of the model selection problem.

Calibration of channelized subsurface flow models using nested sampling and soft probabilities

A new method for calibration of channelized subsurface flow models is presented. The proposed method relies on the nested sampling algorithm and on adaptive construction of soft probability maps. Nested sampling (NS) is a Bayesian sampling algorithm for estimating the Bayesian evidence and obtaining samples from the posterior distribution of the unknown fields. NS utilizes a set of samples (active set) that evolves to high-likelihood regions. The sample evolution process is achieved by iteratively replacing the sample with the lowest likelihood within the active set by a new sample from the prior but with higher likelihood value (constrained sampling). For channelized models, drawing samples from the prior model based on a training image and only accepting the samples satisfying the likelihood constraint is computationally inefficient due to low acceptance rates. We develop an efficient constrained sampling step utilizing soft probability maps in addition to the training image (using the Tau model) to obtain samples from the prior satisfying the likelihood constraint. The soft probability map is constructed by averaging the samples within the active set and is shown to significantly increase the acceptance rate of the nested sampling algorithm. The proposed algorithm is applied for calibration of several channelized subsurface flow models. In addition, the NS algorithm is applied for prior model selection of a channelized model with different training images obtained by changing the orientation angles of a reference training image. The results show that selecting the prior model based on the data mismatch can be misleading. This highlights the need to evaluate the Bayesian evidence (estimated by the nested sampling algorithm) as a more reliable prior model selection statistics, especially when the amount of calibration data is limited.

Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies.

BackgroundSeveral types of statistical methods are currently available for the meta-analysis of studies on diagnostic test accuracy. One of these methods is the Bivariate Model which involves a simultaneous analysis of the sensitivity and specificity from a set of studies. In this paper, we review the characteristics of the Bivariate Model and demonstrate how it can be extended with a discrete latent variable. The resulting clustering of studies yields additional insight into the accuracy of the test of interest.MethodsA Latent Class Bivariate Model is proposed. This model captures the between-study variability in sensitivity and specificity by assuming that studies belong to one of a small number of latent classes. This yields both an easier to interpret and a more precise description of the heterogeneity between studies. Latent classes may not only differ with respect to the average sensitivity and specificity, but also with respect to the correlation between sensitivity and specificity.ResultsThe Latent Class Bivariate Model identifies clusters of studies with their own estimates of sensitivity and specificity. Our simulation study demonstrated excellent parameter recovery and good performance of the model selection statistics typically used in latent class analysis. Application in a real data example on coronary artery disease showed that the inclusion of latent classes yields interesting additional information.ConclusionsOur proposed new meta-analysis method can lead to a better fit of the data set of interest, less biased estimates and more reliable confidence intervals for sensitivities and specificities. But even more important, it may serve as an exploratory tool for subsequent sub-group meta-analyses.

AIC type statistics for discretely observed ergodic diffusion processes

We consider the model selection problem for ergodic diffusion processes based on sampled data. The adaptive estimators for parameters of drift and diffusion coefficients are used in order to construct Akaike’s information criterion (AIC) type model selection statistics. Asymptotic properties of our proposed criteria are given for three kinds of the adaptive estimators.