Number Of Candidate Predictors Research Articles

LASSO–penalized clusterwise linear regression modelling: a two–step approach

In clusterwise regression analysis, the goal is to predict a response variable based on a set of explanatory variables, each with cluster-specific effects. In many real–life problems, the number of candidate predictors is typically large, with perhaps only a few of them meaningfully contributing to the prediction. A well–known method to perform variable selection is the LASSO, with calibration done by minimizing the Bayesian Information Criterion (BIC). However, existing LASSO-penalized estimators are problematic for several reasons. First, only certain types of penalties are considered. Second, the computations may sometimes involve approximate schemes. Third, variable selection is usually time consuming, due to a complex calibration of the penalty term, possibly requiring several multiple evaluations of an estimator for each plausible value of the tuning parameter(s). We introduce a two–step approach to fill these gaps. In step 1, we fit LASSO clusterwise linear regressions with some pre–specified level of penalization (Fit step). In step 2 (Selection step), we perform covariate selection locally, i.e. on the weighted data, with weights corresponding to the posterior probabilities from the previous step. This is done by using a generalization of the Least Angle Regression (LARS) algorithm, which permits covariate selection with a single evaluation of the estimator. In addition, both Fit and Selection steps leverage on an Expectation Maximization (EM) algorithm, fully in closed forms, designed with a very general version of the LASSO penalty. The advantages of our proposal, in terms of computation time reduction, and accuracy of model estimation and selection, are shown by means of a simulation study, and illustrated with a real data application.

Comparison of likelihood penalization and variance decomposition approaches for clinical prediction models: A simulation study.

Logistic regression is one of the most commonly used approaches to develop clinical risk prediction models. Developers of such models often rely on approaches that aim to minimize the risk of overfitting and improve predictive performance of the logistic model, such as through likelihood penalization and variance decomposition techniques. We present an extensive simulation study that compares the out-of-sample predictive performance of risk prediction models derived using the elastic net, with Lasso and ridge as special cases, and variance decomposition techniques, namely, incomplete principal component regression and incomplete partial least squares regression. We varied the expected events per variable, event fraction, number of candidate predictors, presence of noise predictors, and the presence of sparse predictors in a full-factorial design. Predictive performance was compared on measures of discrimination, calibration, and prediction error. Simulation metamodels were derived to explain the performance differences within model derivation approaches. Our results indicate that, on average, prediction models developed using penalization and variance decomposition approaches outperform models developed using ordinary maximum likelihood estimation, with penalization approaches being consistently superior over the variance decomposition approaches. Differences in performance were most pronounced on the calibration of the model. Performance differences regarding prediction error and concordance statistic outcomes were often small between approaches. The use of likelihood penalization and variance decomposition techniques methods was illustrated in the context of peripheral arterialdisease.

Re-evaluation of the comparative effectiveness of bootstrap-based optimism correction methods in the development of multivariable clinical prediction models

BackgroundMultivariable prediction models are important statistical tools for providing synthetic diagnosis and prognostic algorithms based on patients’ multiple characteristics. Their apparent measures for predictive accuracy usually have overestimation biases (known as ‘optimism’) relative to the actual performances for external populations. Existing statistical evidence and guidelines suggest that three bootstrap-based bias correction methods are preferable in practice, namely Harrell’s bias correction and the .632 and .632+ estimators. Although Harrell’s method has been widely adopted in clinical studies, simulation-based evidence indicates that the .632+ estimator may perform better than the other two methods. However, these methods’ actual comparative effectiveness is still unclear due to limited numerical evidence.MethodsWe conducted extensive simulation studies to compare the effectiveness of these three bootstrapping methods, particularly using various model building strategies: conventional logistic regression, stepwise variable selections, Firth’s penalized likelihood method, ridge, lasso, and elastic-net regression. We generated the simulation data based on the Global Utilization of Streptokinase and Tissue plasminogen activator for Occluded coronary arteries (GUSTO-I) trial Western dataset and considered how event per variable, event fraction, number of candidate predictors, and the regression coefficients of the predictors impacted the performances. The internal validity of C-statistics was evaluated.ResultsUnder relatively large sample settings (roughly, events per variable ≥ 10), the three bootstrap-based methods were comparable and performed well. However, all three methods had biases under small sample settings, and the directions and sizes of biases were inconsistent. In general, Harrell’s and .632 methods had overestimation biases when event fraction become lager. Besides, .632+ method had a slight underestimation bias when event fraction was very small. Although the bias of the .632+ estimator was relatively small, its root mean squared error (RMSE) was comparable or sometimes larger than those of the other two methods, especially for the regularized estimation methods.ConclusionsIn general, the three bootstrap estimators were comparable, but the .632+ estimator performed relatively well under small sample settings, except when the regularized estimation methods are adopted.

A divide-and-conquer method for sparse risk prediction and evaluation.

Divide-and-conquer (DAC) is a commonly used strategy to overcome the challenges of extraordinarily large data, by first breaking the dataset into series of data blocks, then combining results from individual data blocks to obtain a final estimation. Various DAC algorithms have been proposed to fit a sparse predictive regression model in the $L_1$ regularization setting. However, many existing DAC algorithms remain computationally intensive when sample size and number of candidate predictors are both large. In addition, no existing DAC procedures provide inference for quantifying the accuracy of risk prediction models. In this article, we propose a screening and one-step linearization infused DAC (SOLID) algorithm to fit sparse logistic regression to massive datasets, by integrating the DAC strategy with a screening step and sequences of linearization. This enables us to maximize the likelihood with only selected covariates and perform penalized estimation via a fast approximation to the likelihood. To assess the accuracy of a predictive regression model, we develop a modified cross-validation (MCV) that utilizes the side products of the SOLID, substantially reducing the computational burden. Compared with existing DAC methods, the MCV procedure is the first to make inference on accuracy. Extensive simulation studies suggest that the proposed SOLID and MCV procedures substantially outperform the existing methods with respect to computational speed and achieve similar statistical efficiency as the full sample-based estimator. We also demonstrate that the proposed inference procedure provides valid interval estimators. We apply the proposed SOLID procedure to develop and validate a classification model for disease diagnosis using narrative clinical notes based on electronic medical record data from Partners HealthCare.

The Quality of Life in Bipolar Disorder (QoL.BD) questionnaire a decade on – A systematic review of the measurement of condition-specific aspects of quality of life in bipolar-disorder

BackgroundCondition-specific quality of life (QoL) instruments are more representative of the priorities of people with lived experience. As such, the development of the first and only bipolar disorder (BD) specific measurement instrument, the Quality of Life in Bipolar Disorder (QoL.BD) questionnaire, marked an important step forward for the literature. The present systematic review aims to characterise applications of the QoL.BD in the BD literature and review empirical findings obtained from studies using this measure. MethodsA systematic search identified 37 peer-reviewed publications which reported original empirical data using the QoL.BD in a BD population. No restrictions were placed on language/study type. ResultsAdaptations to the QoL.BD displayed appropriate psychometric properties. Although clinical trials were typically underpowered, promising effect sizes for a number of treatment modalities were reported. QoL.BD scores were moderately correlated with depressive symptoms; a number of candidate predictors were identified. LimitationsDue to resource limitations, the present review used one database (Google Scholar), and a single author reviewed articles for eligibility. On balance the risks of missing relevant studies were deemed minimal. ConclusionA sizeable, international body of evidence now exists regarding the measurement, presentation, and treatment of condition-specific aspects of QoL in BD. Key avenues for future research include large scale, randomized control clinical trials using the QoL.BD as a primary outcome, and granular exploration of potential correlates of QoL.BD domain scores. Finally, longer follow-up periods are required to inform understanding of the dynamic relationship between clinical variables and condition-specific aspects of QoL in BD.

Predictors of pain in general ageing populations: results from a multi-country analysis based on ATHLOS harmonized database

BackgroundPain is a common symptom, often associated with neurological and musculoskeletal conditions, and experienced especially by females and by older people, and with increasing trends in general populations. Different risk factors for pain have been identified, but generally from studies with limited samples and a limited number of candidate predictors. The aim of this study is to evaluate the predictors of pain from a large set of variables and respondents.MethodsWe used part of the harmonized dataset of ATHLOS project, selecting studies and waves with a longitudinal course, and in which pain was absent at baseline and with no missing at follow-up. Predictors were selected based on missing distribution and univariable association with pain, and were selected from the following domains: Socio-demographic and economic characteristics, Lifestyle and health behaviours, Health status and functional limitations, Diseases, Physical measures, Cognition, personality and other psychological measures, and Social environment. Hierarchical logistic regression models were then applied to identify significant predictors.ResultsA total of 13,545 subjects were included of whom 5348 (39.5%) developed pain between baseline and the average 5.2 years’ follow-up. Baseline risk factors for pain were female gender (OR 1.34), engaging in vigorous exercise (OR 2.51), being obese (OR 1.36) and suffering from the loss of a close person (OR 1.88) whereas follow-up risk factors were low energy levels/fatigue (1.93), difficulties with walking (1.69), self-rated health referred as poor (OR 2.20) or average to moderate (OR 1.57) and presence of sleep problems (1.80).ConclusionsOur results showed that 39.5% of respondents developed pain over a five-year follow-up period, that there are proximal and distal risk factors for pain, and that part of them are directly modifiable. Actions aimed at improving sleep, reducing weight among obese people and treating fatigue would positively impact on pain onset, and avoiding vigorous exercise should be advised to people aged 60 or over, in particular if female or obese.

On the overestimation of random forest's out-of-bag error.

The ensemble method random forests has become a popular classification tool in bioinformatics and related fields. The out-of-bag error is an error estimation technique often used to evaluate the accuracy of a random forest and to select appropriate values for tuning parameters, such as the number of candidate predictors that are randomly drawn for a split, referred to as mtry. However, for binary classification problems with metric predictors it has been shown that the out-of-bag error can overestimate the true prediction error depending on the choices of random forests parameters. Based on simulated and real data this paper aims to identify settings for which this overestimation is likely. It is, moreover, questionable whether the out-of-bag error can be used in classification tasks for selecting tuning parameters like mtry, because the overestimation is seen to depend on the parameter mtry. The simulation-based and real-data based studies with metric predictor variables performed in this paper show that the overestimation is largest in balanced settings and in settings with few observations, a large number of predictor variables, small correlations between predictors and weak effects. There was hardly any impact of the overestimation on tuning parameter selection. However, although the prediction performance of random forests was not substantially affected when using the out-of-bag error for tuning parameter selection in the present studies, one cannot be sure that this applies to all future data. For settings with metric predictor variables it is therefore strongly recommended to use stratified subsampling with sampling fractions that are proportional to the class sizes for both tuning parameter selection and error estimation in random forests. This yielded less biased estimates of the true prediction error. In unbalanced settings, in which there is a strong interest in predicting observations from the smaller classes well, sampling the same number of observations from each class is a promising alternative.

Factors Affecting the Number of Visitors in National Parks in the Czech Republic, Germany and Austria

In the context of national-level strategies, the importance of tourism in national parks is on the rise. The objective of this study is to investigate the relationship between the number of visitors to national parks and five variables: area, number of employees, budget, average employee salary and number of researchers in 12 national parks in the Czech Republic, Germany and Austria. Analysis of factors influencing the number of visitors to national parks uses the method of retrospective analysis of the data contained in internal documents and questionnaires among managers of national parks. The number of candidate predictors is relatively high when compared with the number of observations. Due to this fact, the Gilmour method for statistical analysis is used. Statistical results represented by the parameter β2 for number of employees is −33,016 (95% CI, −50,592–−15,441) and by the parameter β3 for budget is 0.586 (95% CI, 0.295–0.878), showing that the number of visitors increases with budget, while it decreases with the number of employees. The results of this study are a useful starting point for managers in their efforts to focus on developing key areas in an appropriate way. In conclusion, results show that increasing the economic benefits accruing from national parks regional policy could aim at a qualitative upgrading of tourist services, increased marketing of the unique national park label and the promotion of a diverse regional supply base.

Augmented estimation for t-year survival with censored regression models.

Reliable and accurate risk prediction is fundamental for successful management of clinical conditions. Estimating comprehensive risk prediction models precisely, however, is a difficult task, especially when the outcome of interest is time to a rare event and the number of candidate predictors, p, is not very small. Another challenge in developing accurate risk models arises from potential model misspecification. Time-specific generalized linear models estimated with inverse censoring probability weighting are robust to model misspecification, but may be inefficient in the rare event setting. To improve the efficiency of such robust estimation procedures, various augmentation methods have been proposed in the literature. These procedures can also leverage auxiliary variables such as intermediate outcomes that are predictive of event risk. However, most existing methods do not perform well in the rare event setting, especially when p is not small. In this article, we propose a two-step, imputation-based augmentation procedure that can improve estimation efficiency and that is robust to model misspecification. We also develop regularized augmentation procedures for settings where p is not small, along with procedures to improve the estimation of individualized treatment effect in risk reduction. Numerical studies suggest that our proposed methods substantially outperform existing methods in efficiency gains. The proposed methods are applied to an AIDS clinical trial for treating HIV-infected patients.

An enhanced dead reckoning algorithm with hybrid extrapolation models (AisaSim 2016)

The traditional Dead Reckoning algorithm predicts the future motion state based on a determined polynomial predictor, and the forecasting performance would vary with different types of motion entities. This paper proposes an enhanced dead reckoning algorithm based on hybrid extrapolation models, which can be used to reduce the communication in a distributed interactive simulation. The proposed algorithm perform extrapolation using a number of candidate predictors. Its idea is based on the assumption that a complex trajectory can be decomposed into several simple trajectories. The experimental evaluations show that the enhanced Dead Reckoning algorithm provides better performance in correction data reduction and accurate estimation.

Visualization of evidence in regression with the QR decomposition

Graphical ANOVA is a simple and effective tool for visualizing evidence of differences between treatment means for data coming from factorial experiments. The purpose of the present article is to propose an analogous method for the visualization of the significance of regression, using the QR decomposition. Two graphical tests are proposed and compared with the classical $$F$$F test, by simulation. It is found that when the number of candidate predictors is small relative to the sample size, the classical test has slightly higher power than the graphical tests. When the number of predictors is large, the graphical tests remain powerful. while the classical $$F$$F test exhibits poor power properties.

Bayes Variable Selection in Semiparametric Linear Models

There is a rich literature on Bayesian variable selection for parametric models. Our focus is on generalizing methods and asymptotic theory established for mixtures of g-priors to semiparametric linear regression models having unknown residual densities. Using a Dirichlet process location mixture for the residual density, we propose a semiparametric g-prior which incorporates an unknown matrix of cluster allocation indicators. For this class of priors, posterior computation can proceed via a straightforward stochastic search variable selection algorithm. In addition, Bayes’ factor and variable selection consistency is shown to result under a class of proper priors on g even when the number of candidate predictors p is allowed to increase much faster than sample size n, while making sparsity assumptions on the true model size.

Neighborhood search heuristics for selecting hierarchically well-formulated subsets in polynomial regression

The importance of subset selection in multiple regression has been recognized for more than 40 years and, not surprisingly, a variety of exact and heuristic procedures have been proposed for choosing subsets of variables. In the case of polynomial regression, the subset selection problem is complicated by two issues: (1) the substantial growth in the number of candidate predictors, and (2) the desire to obtain hierarchically well-formulated subsets that facilitate proper interpretation of the regression parameter estimates. The first of these issues creates the need for heuristic methods that can provide solutions in reasonable computation time; whereas the second requires innovative neighborhood search approaches that accommodate the hierarchical constraints. We developed tabu search and variable neighborhood search heuristics for subset selection in polynomial regression. These heuristics are applied to a classic data set from the literature and, subsequently, evaluated in a simulation study using synthetic data sets. © 2009 Wiley Periodicals, Inc. Naval Research Logistics, 2010

Variable Selection in Linear Regression With Many Predictors

With advanced capability in data collection, applications of linear regression analysis now often involve a large number of predictors. Variable selection thus has become an increasingly important issue in building a linear regression model. For a given selection criterion, variable selection is essentially an optimization problem that seeks the optimal solution over 2m possible linear regression models, where m is the total number of candidate predictors. When m is large, exhaustive search becomes practically impossible. Simple suboptimal procedures such as forward addition, backward elimination, and backward-forward stepwise procedure are fast but can easily be trapped in a local solution. In this article we propose a relatively simple algorithm for selecting explanatory variables in a linear regression for a given variable selection criterion. Although the algorithm is still a suboptimal algorithm, it has been shown to perform well in extensive empirical study. The main idea of the procedure is to partition the candidate predictors into a small number of groups. Working with various combinations of the groups and iterating the search through random regrouping, the search space is substantially reduced, hence increasing the probability of finding the global optimum. By identifying and collecting “important” variables throughout the iterations, the algorithm finds increasingly better models until convergence. The proposed algorithm performs well in simulation studies with 60 to 300 predictors. As a by-product of the proposed procedure, we are able to study the behavior of variable selection criteria when the number of predictors is large. Such a study has not been possible with traditional search algorithms.This article has supplementary material online.

Development and Evaluation of Therapeutically Relevant Predictive Classifiers Using Gene Expression Profiling

Gene expression profi les offer the possibility of improving risk predication and optimizing treatment selection for individual patients. Two articles in this issue of the Journal describe clinical studies of gene expression profi ling — one a developmental study and the other a validation study. Asgharzadeh et al. ( 1 ) address the development of a prognostic classifi er for patients with metastatic neuroblastoma lacking amplifi cation of the MYCN gene. Buyse et al. ( 2 ) report the validation of a gene expression – based prognostic classifi er for patients with early breast cancer. Asgharzadeh et al. ( 1 ) developed their classifi er based on the expression of 55 genes that appears to predict risk of disease progression more accurately than does patient age, histologic type, or other currently used risk features. The claims of improved risk prediction are based on an internal estimate of prediction accuracy computed by Asgharzadeh et al. The approach taken by the authors allowed them to avoid one of the major pitfalls of developmental studies, which is that they often provide highly biased estimates of accuracy. The fundamental principle is that the same data should not be used for developing a predictive classifi er and for evaluating the accuracy of that classifi er. This principle is especially important for microarray-based studies because the number of candidate predictors (genes) is generally orders of magnitude greater than the number of cases. In this setting, the bias of using the same data for developing and evaluating a classifi er is overwhelming ( 3 ) . Some developmental studies avoid this bias by separating the data into a training set used for model development and a test set used for evaluating the predictive accuracy of the model. Although this split-sample approach is useful, it represents an ineffi cient use of the data in that the training set may be too small to develop an accurate classifi cation model and the test set may be too small to provide an accurate estimate of prediction accuracy. Molinaro et al. ( 4 ) showed that various cross-validation approaches can provide better estimates of predictive accuracy. Such methods are based on repeatedly partitioning the sample into a relatively large portion that is used for classifi er development and a small portion that is used for classifi er evaluation and then averaging the results over the multiple partitions. In this case, the estimate of prediction accuracy pertains to the model that was developed using the full dataset, which is the model that will be used in future studies. The cross-validation procedure develops multiple classifi ers based on reduced training sets only as steps in computing an estimate of prediction accuracy for the classifi er developed using the full dataset. In developing a classifi er on a reduced data training set, the model development algorithm must be applied from scratch, without using any information based on data not part of that reduced data training set. This point is frequently overlooked by biomedical scientists, statisticians, and computer scientists. Asgharzadeh et al. were careful to use cross-validation methods

Tests of Significance in Forward Selection Regression With anF-to-Enter Stopping Rule

A Monte Carlo simulation is used to estimate the upper percentage points of the null distribution of the sample squared multiple correlation coefftcient (R 2) when the number of predictors selected is determined by a stopping rule. In the study, the sample size n and the number of candidate predictors m satisfy 2 ≤ m ≤ 20 and 10 ≤ n – m – 1 ≤ 200, while the F threshold ranges from two to four. Tables of the upper five percent and upper one percent sample R* values are presented and an example is given to illustrate the use of the tables.