Linear Model Selection Research Articles

Executive Functions Are Associated with Fall Risk but not Balance in Chronic Cerebrovascular Disease.

Background: Older people’s deficits in executive functions (EF) have been shown to lead to higher fall risk, postural sway, and reduced speed. Crucially, EF impairments are even more pronounced in individuals with chronic cerebrovascular disease (CVD), namely vascular cognitive impairment. Methods: In this retrospective cross-sectional study, we used a complete neuropsychological battery, including the Trail Making Test (TMT) and physical measures, such as the Morse fall and EQUI scales, to assess 66 individuals with chronic CVD. Linear regressions, Bayesian analyses, and model selection were performed to see the impact of EF, global cognition, and vascular parkinsonism/hemiplegia on physical measures (fall risk and balance). Results: The TMT part B and BA correlated (r = 0.44 and r = 0.45) with Morse fall scale. Only EF significantly explained fall risk, whereas global cognition and vascular parkinsonism/hemiplegia did not. These findings were confirmed by Bayesian evidence and parsimony model selection. Balance was not significantly correlated with any of the neuropsychological tests. Conclusions: This is the first study investigating the relationship between cognitive and physical measures in a sample of older people with chronic CVD. The results are consistent with previous findings that link EF with fall risk in CVD.

Modal regression based robust empirical likelihood and variable selection for partial linear models with longitudinal data

Modal regression based robust empirical likelihood and variable selection for partial linear models with longitudinal data

Variable selection in high-dimensional linear model with possibly asymmetric errors

In many application areas, the problem of the automatic variable selection in a linear model with asymmetric errors is encountered, when the number of explanatory variables diverges with the sample size. For this high-dimensional model, the penalized least squares method is not appropriate and the quantile framework makes the inference more difficult because of the non differentiability of the loss function. An estimation method by penalizing the expectile process with an adaptive LASSO penalty is proposed and studied. Two cases are considered: first with the number of model parameters is assumed to be much smaller than the sample size and afterwards it could be of the same order; the two cases being distinct by the adaptive penalties considered. For each case, the rate convergence is obtained and the oracle properties of the adaptive LASSO expectile estimator are established. The proposed estimators are evaluated through Monte Carlo simulations and compared with the adaptive LASSO quantile estimator. The proposed estimation method is also applied to real data in genetics.

Which bridge estimator is the best for variable selection?

We study the problem of variable selection for linear models under the high-dimensional asymptotic setting, where the number of observations $n$ grows at the same rate as the number of predictors $p$. We consider two-stage variable selection techniques (TVS) in which the first stage uses bridge estimators to obtain an estimate of the regression coefficients, and the second stage simply thresholds this estimate to select the “important” predictors. The asymptotic false discovery proportion ($\operatorname{AFDP}$) and true positive proportion (ATPP) of these TVS are evaluated. We prove that for a fixed ATPP, in order to obtain a smaller $\operatorname{AFDP}$, one should pick a bridge estimator with smaller asymptotic mean square error in the first stage of TVS. Based on such principled discovery, we present a sharp comparison of different TVS, via an in-depth investigation of the estimation properties of bridge estimators. Rather than “orderwise” error bounds with loose constants, our analysis focuses on precise error characterization. Various interesting signal-to-noise ratio and sparsity settings are studied. Our results offer new and thorough insights into high-dimensional variable selection. For instance, we prove that a TVS with Ridge in its first stage outperforms TVS with other bridge estimators in large noise settings; two-stage LASSO becomes inferior when the signal is rare and weak. As a by-product, we show that two-stage methods outperform some standard variable selection techniques, such as $\operatorname{LASSO}$ and Sure Independence Screening, under certain conditions.

Fitting sparse linear models under the sufficient and necessary condition for model identification

We propose an enhanced support detection and root finding approach (ESDAR) to variable selection in sparse linear models. ESDAR is motivated from the KKT conditions for the ℓ0 penalized regression. In ESDAR, we introduce a step size to balance the primal and dual variables in determining the support of the solution. We establish a sharp oracle error bound and an oracle support recovery property for the solution sequence generated by ESDAR under the weakest possible condition on the design matrix that is sufficient and necessary for the model to be identifiable. The conditions for the oracle results we obtained are weaker than those for Lasso and concave selection methods including SCAD and MCP in the literature.

Consistent Group Selection with Bayesian High Dimensional Modeling

In many applications with high dimensional covariates, the covariates are naturally structured into different groups which can be used to perform efficient statistical inference. We propose a Bayesian hierarchical model with a spike and slab prior specification to perform group selection in high dimensional linear regression models. While several penalization methods and more recently, some Bayesian approaches are proposed for group selection, theoretical properties of Bayesian approaches have not been studied extensively. In this paper, we provide novel theoretical results for group selection consistency under spike and slab priors which demonstrate that the proposed Bayesian approach has advantages compared to penalization approaches. Our theoretical results accommodate flexible conditions on the design matrix and can be applied to commonly used statistical models such as nonparametric additive models for which very limited theoretical results are available for the Bayesian methods. A shotgun stochastic search algorithm is adopted for the implementation of our proposed approach. We illustrate through simulation studies that the proposed method has better performance for group selection compared to a variety of existing methods.

Variable selection in elliptical linear mixed model

Variable selection in elliptical Linear Mixed Models (LMMs) with a shrinkage penalty function (SPF) is the main scope of this study. SPFs are applied for parameter estimation and variable selection simultaneously. The smoothly clipped absolute deviation penalty (SCAD) is one of the SPFs and it is adapted into the elliptical LMM in this study. The proposed idea is highly applicable to a variety of models which are set up with different distributions such as normal, student-t, Pearson VII, power exponential and so on. Simulation studies and real data example with one of the elliptical distributions show that if the variable selection is also a concern, it is worthwhile to carry on the variable selection and the parameter estimation simultaneously in the elliptical LMM.

Variable selection with P-splines in functional linear regression: Application in graft-versus-host disease.

This paper focuses on the problems of estimation and variable selection in the functional linear regression model (FLM) with functional response and scalar covariates. To this end, two different types of regularization (L1 and L2 ) are considered in this paper. On the one hand, a sample approach for functional LASSO in terms of basis representation of the sample values of the response variable is proposed. On the other hand, we propose a penalized version of the FLM by introducing a P-spline penalty in the least squares fitting criterion. But our aim is to propose P-splines as a powerful tool simultaneously for variable selection and functional parameters estimation. In that sense, the importance of smoothing the response variable before fitting the model is also studied. In summary, penalized (L1 and L2 ) and nonpenalized regression are combined with a presmoothing of the response variable sample curves, based on regression splines or P-splines, providing a total of six approaches to be compared in two simulation schemes. Finally, the most competitive approach is applied to a real data set based on the graft-versus-host disease, which is one of the most frequent complications (30% -50%) in allogeneic hematopoietic stem-celltransplantation.

Landscape factors driving the spread of the invasive grass,Hymenachne amplexicaulis, among wetlands in a Florida subtropical grazing land

AbstractWetlands embedded in agroecosystems provide vital ecosystem services (i.e., freeze protection, water retention, nutrient cycling, biodiversity support). However, they are particularly susceptible to invasion by nonnative species. West Indian marsh grass [Hymenachne amplexicaulis(Rudge) Nees] is a major wetland invader in Florida. Despite the documented consequences ofH. amplexicaulisinvasions, the landscape factors influencing the spread of this species are poorly understood. In this study, we asked whether landscape factors associated with wetland isolation, connectivity, and land management influence the presence ofH. amplexicaulisamong wetlands embedded in pastures. We recorded the presence or absence ofH.amplexicaulisin 158 seasonal wetlands embedded in different pasture types (semi-natural vs. intensively managed). Wetland area, isolation from neighboring wetlands, isolation from the nearest source ditch, and connectivity were determined using a geographic information system (GIS). We related landscape factors toH. amplexicaulisusing generalized linear models and model selection based on the second-order Akaike information criterion.Hymenachne amplexicauliswas first detected at the study site in the early 2000s. By 2018, we observed this species in 66% of the surveyed wetlands. The likelihood of observingH. amplexicauliswas higher in wetlands embedded in semi-natural pastures and higher in less isolated wetlands, especially when connected to a ditch. These results indicate thatH. amplexicaulisspreads both overland (during seasonal flooding) and via the ditch network. Future work is needed to understand whether seeds or stolons are the primary invasion propagule and whether the species forms a persistent seed bank that could slow down restoration efforts. Additionally, further research is required to understand the ecological impact of this highly invasive plant in Florida wetlands.

Focused model selection for linear mixed models with an application to whale ecology

A central point of disagreement, in certain long-standing discussions about a particular whaling dataset in the Scientific Committee of the International Whaling Commission, has directly involved model selection issues for linear mixed effect models. The biological question under discussion is associated with a clearly defined parameter of primary interest, a focus parameter, which makes model selection with the Focused Information Criterion (FIC) more appropriate than other selection methods. Since the existing FIC methodology has not covered the case of linear mixed effects models, this article sets up the required framework and develops the necessary formulae for the relevant FIC. Our new criterion requires the asymptotic distribution of estimators derived for a given candidate linear mixed model but with behaviour examined under a wider linear mixed model. These results, needed here to build our FIC, also have independent interest.

The Effects of Gender on Radiation Dose during Fenestrated Endovascular Aneurysm Repair

Female gender is considered a risk factor for worse perioperative outcomes after fenestrated endovascular aneurysm repair (FEVAR). We hypothesized that women would have more unfavorable anatomy, increasing case complexity and leading to higher radiation doses. Our aim was to evaluate the effect of gender on radiation dose during FEVARs. This single-center retrospective study was performed from 1/2015 to 2/2018. For patient data, linear model and stepwise variable selection algorithm were used. All dose measurements were log transformed before analysis. Significance level for parameter estimates and corresponding 95% confidence intervals were all transformed back using an exponential function. P-value of <0.05 was considered statistically significant. All analyses were performed in SAS 9.4 (SAS Institute Inc., Cary, NC). A total of 169 FEVARs (45 women) were performed on a Philips Allura Xper FD 20 fluoroscopy system equipped with clarity technology. There was no difference in body mass index (BMI) or operative time between genders, P=0.9. The median reference air kerma for women was significantly lower than that for men (1,672mGy vs. 2,496mGy), P<0.001. Women had on average a 28% total dose reduction after controlling for BMI, number of vessels fenestrated, operative time, and type of device, P<0.001. The median fluorography and fluoroscopy doses for women were significantly lower than those for men (973mGy vs. 1,401mGy and 659mGy vs. 1,008mGy), resulting in a 24% fluorography dose reduction and a 38% fluoroscopy dose reduction for women, P<0.001. FEVARs can be performed successfully in women with comparatively lower radiation doses.

An iterative algorithm for joint covariate and random effect selection in mixed effects models.

We consider joint selection of fixed and random effects in general mixed-effects models. The interpretation of estimated mixed-effects models is challenging since changing the structure of one set of effects can lead to different choices of important covariates in the model. We propose a stepwise selection algorithm to perform simultaneous selection of the fixed and random effects. It is based on Bayesian Information criteria whose penalties are adapted to mixed-effects models. The proposed procedure performs model selection in both linear and nonlinear models. It should be used in the low-dimension setting where the number of ovariates and the number of random effects are moderate with respect to the total number of observations. The performance of the algorithm is assessed via a simulation study, which includes also a comparative study with alternatives when available in the literature. The use of the method is illustrated in the clinical study of an antibiotic agent kinetics.

Small mammals in fragments of Atlantic Forest: species richness answering to field methods and environment

AbstractSmall mammals can be used as environmental indicators and have been intensively studied in fragmented landscapes of Atlantic Forest, with a wide range of field methods. Our aim in this study was two-fold: we tested for the effects of methods and for the effects of the main environmental variables on observed small mammal richness in fragments of Atlantic Forest. We gathered information on small mammal richness, methods and environmental variables from 122 fragments of Atlantic Forest through literature review. These data were analysed using linear models and model selection based on AIC values along with a regression tree analysis. We found that studies will record more species with bigger trapping effort, using pitfall traps and sampling all forest strata. We also confirmed two important ecological assumptions: fragments at lower latitudes and bigger fragments were the ones with higher species richness. Methodological and environmental variables were analysed together on a regression tree, where trapping effort was the most important variable, surpassing any environmental effect. Considering that a significant number of the studies on Atlantic Forest fragments did not use pitfall traps or sample all forest strata, their results on forest fragmentation were affected by sampling bias.

Functional markers to predict forest ecosystem properties along a rural‐to‐urban gradient

AbstractQuestionsWith increasing focus on urban sustainability, new tools are needed to manage urban woodlands for resilience and ecosystem service provision. Functional traits can provide quantitative and testable links between urban plant communities and specific ecosystem properties (functional markers). We ask whether commonly described multivariate patterns of trait association with urbanization (trait syndromes) capture changes in ecosystem properties associated with urbanization. Given that environmental heterogeneity can generate weak or non‐linear trait–ecosystem property relationships, we ask whether linear methods can yield functional markers with significant power for different ecosystem properties.LocationMontreal metropolitan area (Canada).MethodsWe documented the functional composition of 43 woodlands along an urbanization gradient and measured proxies of three ecosystem properties: plant colonization, soil water infiltration and organic matter decomposition. We use redundancy analysis to identify traits associated with urbanization, and multiple linear regression and model selection to identify response and effect traits that best predict actual differences in ecosystem properties. We compare the resulting linear model with a non‐linear equivalent.ResultsTraits associated with urbanization (urban syndrome) were inconsistently selected as best predictors of ecosystems properties (functional markers). Although predictive power varied between ecosystem properties, all three could be significantly predicted from community‐weighted traits (functional markers), with both response and effect traits contributing to the final model. When we fitted equivalent non‐linear models, we found that traits had largely non‐linear relationships with ecosystem properties.ConclusionsOur results demonstrate that community‐weighted traits of urban woodlands can yield functional markers that capture ecosystem properties, but these are inconsistently identified by “trait syndrome” approaches. In linear combinations, such functional markers provide a testable and generalizable way to quantify ecosystem properties in urban woodlands. Capturing such properties is one important step toward management of woodlands for their continued ability to provide ecosystem services into the future.

Robust Bayesian model selection for heavy-tailed linear regression using finite mixtures

In this paper, we present a novel methodology to perform Bayesian model selection in linear models with heavy-tailed distributions. We consider a finite mixture of distributions to model a latent variable where each component of the mixture corresponds to one possible model within the symmetrical class of normal independent distributions. Naturally, the Gaussian model is one of the possibilities. This allows for a simultaneous analysis based on the posterior probability of each model. Inference is performed via Markov chain Monte Carlo—a Gibbs sampler with Metropolis–Hastings steps for a class of parameters. Simulated examples highlight the advantages of this approach compared to a segregated analysis based on arbitrarily chosen model selection criteria. Examples with real data are presented and an extension to censored linear regression is introduced and discussed.

Environmental correlates of circannual breeding periodicity in a multi-species assemblage of Amazonian electric fishes

We quantified patterns of circannual reproductive breeding periodicity in a multi-species assemblage of the gymnotiform knifefish genus Brachyhypopomus from Amazonian floodplain and terra firme stream habitats. We applied linear mixed-effect models and model selection to a dataset of 3205 fish from the five most abundant species to identify environmental variables predictive of gonadosomatic index (GSI) – including rainfall, river level, water temperature, conductivity, dissolved oxygen (DO), and pH. A single common floodplain species exhibited a well-demarcated ca. 6-month breeding season corresponding to the rising-water and high-water period. In this species the strongest predictors of GSI in both sexes were river level and pH. Conductivity, temperature, and DO all exhibited substantial circannual variation in the floodplain but were uncorrelated to GSI. Four common terra firme stream-dwelling species exhibited longer (ca. 7–9-month) and less distinct breeding seasons that were approximately coincident with the late dry/early rainy season. The stream species exhibited a congruent species-wide response in which GSI was predicted in both sexes by rainfall, but not by physico-chemical water properties (all of which exhibited relatively minor circannual variation). Five of the most abundant species in this study have a semelparous one-year life history, but one (B. beebei) has an iteroparous two-year life history. Consistent with a pattern of terminal reproductive investment, we observed a more pronounced elevation of GSI (especially in females) in B. beebei’s final second breeding season than in its first.

Graph Structured Sparse Subset Selection

We propose a new method for variable subset selection and regression coefficient estimation in linear regression models that incorporates a graph structure of the predictor variables. The proposed method is based on the cardinality constraint that controls the number of selected variables and the graph structured subset constraint that encourages the predictor variables adjacent in the graph to be simultaneously selected or eliminated from the model. Moreover, we develop an efficient discrete projected gradient descent method to handle the NP-hardness of the problem originating from the discrete constraints. Numerical experiments on simulated and real-world data are conducted to demonstrate the usefulness and applicability of the proposed method by comparing it with existing graph regularization methods in terms of the predictive accuracy and variable selection performance. The results confirm that the proposed method outperforms the existing methods.

BeSS: An R Package for Best Subset Selection in Linear, Logistic and Cox Proportional Hazards Models

We introduce a new R package, BeSS, for solving the best subset selection problem in linear, logistic and Cox's proportional hazard (CoxPH) models. It utilizes a highly efficient active set algorithm based on primal and dual variables, and supports sequential and golden search strategies for best subset selection. We provide a C++ implementation of the algorithm using an Rcpp interface. We demonstrate through numerical experiments based on enormous simulation and real datasets that the new BeSS package has competitive performance compared to other R packages for best subset selection purposes.

Impacts of Environmental Changes on Well-Being in Indigenous Communities in Eastern Canada.

Climate change and natural resource exploitation can affect Indigenous people’s well-being by reducing access to ecosystem services, in turn impeding transmission of traditional knowledge and causing mental health problems. We used a questionnaire based on the Environmental Distress Scale (EDS) and the Connor–Davidson Resilience Scale (CD-RISC-10) to examine the impacts of environmental changes on 251 members of four Indigenous communities in the eastern Canadian boreal forest. We also considered the potential mitigating effects of sociodemographic characteristics (i.e., age, gender, parenthood, and time spent on the land) and protective factors (i.e., health, quality of life, resilience, life on the land, life in the community, and support from family and friends). Using linear regression, model selection, and multi-model inference, we show that the felt impacts of environmental changes increased with age but were lower for participants with higher quality of life. The effect of resilience was opposite to expectations: more resilient participants felt more impacts. This could be because less resilient individuals ceased to go on the land when environmental changes exceeded a given threshold; thus, only the most resilient participants could testify to the impacts of acute changes. Further research will be needed to test this hypothesis.

Differential Response Trajectories to Acute Exercise in Blood and Muscle

Human physical performance, a critical determinant of human survival over thousands of years, is among the most potent medical interventions. However, the molecular mechanisms that drive exercise adaptation are incompletely understood. We created a computational framework comprising data from 49 studies, including 1,986 individuals before and after acute exercise and long-term training from skeletal muscle (n=1,260) and blood (n=726). This is the largest framework of its kind. Using a linear mixed effects meta-regression model selection strategy, we detected specific time patterns and novel regulatory modulators of the acute exercise response. We found that acute and long-term responses to exercise are transcriptionally distinct. Exercise induced a more pronounced inflammatory response in skeletal muscle of older individuals and our models revealed multiple sex-associated responses to training. These results deepen our understanding of the transcriptional responses to exercise and provide a powerful resource for future research efforts in exercise physiology and medicine.