Variable Subset Selection Research Articles

Graph Structured Sparse Subset Selection

We propose a new method for variable subset selection and regression coefficient estimation in linear regression models that incorporates a graph structure of the predictor variables. The proposed method is based on the cardinality constraint that controls the number of selected variables and the graph structured subset constraint that encourages the predictor variables adjacent in the graph to be simultaneously selected or eliminated from the model. Moreover, we develop an efficient discrete projected gradient descent method to handle the NP-hardness of the problem originating from the discrete constraints. Numerical experiments on simulated and real-world data are conducted to demonstrate the usefulness and applicability of the proposed method by comparing it with existing graph regularization methods in terms of the predictive accuracy and variable selection performance. The results confirm that the proposed method outperforms the existing methods.

Modeling and Prediction of Gas Chromatography Relative Retention Times of Volatile Organic Compounds

The theme of this paper is to foresee relative retention time of 122 volatile organic compounds. QSRR analysis was accomplished on a serial of 122 VOCs. Multiple Linear Regression (MLR) and support vector machine (SVM) methods were used to build linear and nonlinear (QSRR) models, respectively, which correlate the (RTT) values of these chemical substance to their structural descriptors. At first, the data set was separated using Kennard and Stone algorithm into a training set (92 chemicals) and a test set (30 chemicals) for statistical external validation. The five-dimensional models were developed using as independent variables the theoretical descriptors derived from the DRAGON software during the application of the procedure GA (genetic algorithm) - VSS (Variable Subset Selection). The robustness and the predictive performance of the MLR model have been demonstrated by inside and outer statistical validation. Non-linear technique leads to the best QSRR model with good internal and external predictive abilities. It is based on support vector machines using the RBF function for the optimal parameters values. The values of and were 17, 0.2, 0.2, respectively.

Modeling and Prediction of Gas Chromatography Relative Retention Times of Volatile Organic Compounds

The theme of this paper is to foresee relative retention time of 122 volatile organic compounds. QSRR analysis was accomplished on a serial of 122 VOCs. Multiple Linear Regression (MLR) and support vector machine (SVM) methods were used to build linear and nonlinear (QSRR) models, respectively, which correlate the (RTT) values of these chemical substance to their structural descriptors. At first, the data set was separated using Kennard and Stone algorithm into a training set (92 chemicals) and a test set (30 chemicals) for statistical external validation. The five-dimensional models were developed using as independent variables the theoretical descriptors derived from the DRAGON software during the application of the procedure GA (genetic algorithm) - VSS (Variable Subset Selection). The robustness and the predictive performance of the MLR model have been demonstrated by inside and outer statistical validation. Non-linear technique leads to the best QSRR model with good internal and external predictive abilities. It is based on support vector machines using the RBF function for the optimal parameters values. The values of and were 17, 0.2, 0.2, respectively.

QSAR study of the octanol/water partition coefficient of organophosphorous compounds: The hybrid GA/MLR and GA/ANN approaches

This study aimed at predicting the n-octanol/water partition coefficient (Kow) of 43 organophosphorous insecticides. Quantitative structure?property relationship analysis was performed on the series of 43 insecticides using two different methods, linear (multiple linear regression, MLR) and non-linear (artificial neural network, ANN), which Kow values of these chemicals to their structural descriptors. First, the data set was separated with a duplex algorithm into a training set (28 chemicals) and a test set (15 chemicals) for statistical external validation. A model with four descriptors was developed using as independent variables theoretical descriptors derived from Dragon software when applying genetic algorithm (GA)?variable subset selection (VSS) procedure. The values of statistical parameters, R2, Q2 ext, SDEPext and SDEC for the MLR (94.09 %, 92.43 %, 0.533 and 0.471, respectively) and ANN model (97.24 %, 92.17 %, 0.466 and 0.332, respectively) obtained for the three approaches are very similar, which confirmed that the employed four parameters model is stable, robust and significant.

A non-conformational QSAR study for plant-derived larvicides against Zika Aedes aegypti L. vector

A set of 263 plant-derived compounds with larvicidal activity against Aedes aegypti L. (Diptera: Culicidae) vector is collected from the literature, and is studied by means of a non-conformational quantitative structure-activity relationships (QSAR) approach. The balanced subsets method (BSM) is employed to split the complete dataset into training, validation and test sets. From 26,775 freely available molecular descriptors, the most relevant structural features of compounds affecting the bioactivity are taken. The molecular descriptors are calculated through four different freewares, such as PaDEL, Mold2, EPI Suite and QuBiLs-MAS. The replacement method (RM) variable subset selection technique leads to the best linear regression models. A successful QSAR equation involves 7-conformation-independent molecular descriptors, fulfiling the evaluated internal (loo, l30%o, VIF and Y-randomization) and external (test set with Ntest = 65 compounds) validation criteria. The practical application of this QSAR model reveals promising predicted values for some natural compounds with unknown experimental larvicidal activity. Therefore, the present model constitutes the first one based on a large molecular set, being a useful computational tool for identifying and guiding the synthesis of new active molecules inspired by natural products.

Variable selection methods applied to the mathematics scores of Indonesian students based on convex penalized likelihood

Variable selection is an important topic in linear regression analysis. In practice, a large number of predictors usually are introduced at the initial stage of modeling to attenuate possible modeling biases. stepwise deletion and subset selection are usually used which can be computationally expensive and ignore stochastic errors in the variable selection process. In addition, the best subset selection of variables suffers from several disadvantages, the most severe of which is its a lack of stability. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Some of penalty functions are used to produce sparse solutions. Based on the RMSE and Generalized Information Criterion (GIC) criteria, it was found that the factors affecting Indonesian mathematics scores, where LASSO produces 11 important variables for the model while SCAD has 6 variables which mean that the LASSO model is more complex than SCAD. The MCP produces a simpler model with 5 important variables but has excessive biassed. The results also showed that the SCAD penalty function had the best performance compared to LASSO, Ridge and MCP. Ridge penalty has a worst performance based on all criteria.

An ex vivo tissue model of cartilage degradation suggests that cartilage state can be determined from secreted key protein patterns.

The pathophysiology of osteoarthritis (OA) involves dysregulation of anabolic and catabolic processes associated with a broad panel of proteins that ultimately lead to cartilage degradation. An increased understanding about these protein interactions with systematic in vitro analyses may give new ideas regarding candidates for treatment of OA related cartilage degradation. Therefore, an ex vivo tissue model of cartilage degradation was established by culturing tissue explants with bacterial collagenase II. Responses of healthy and degrading cartilage were analyzed through protein abundance in tissue supernatant with a 26-multiplex protein profiling assay, after exposing the samples to a panel of 55 protein stimulations present in synovial joints of OA patients. Multivariate data analysis including exhaustive pairwise variable subset selection identified the most outstanding changes in measured protein secretions. MMP9 response to stimulation was outstandingly low in degrading cartilage and there were several protein pairs like IFNG and MMP9 that can be used for successful discrimination between degrading and healthy samples. The discovered changes in protein responses seem promising for accurate detection of degrading cartilage. The ex vivo model seems interesting for drug discovery projects related to cartilage degradation, for example when trying to uncover the unknown interactions between secreted proteins in healthy and degrading tissues.

QSPR Models for the prediction of octanol/water partition coefficient of organophosphorous insecticides

This study aims to predict the octanol/water partition coefficient (Kow) of 43 organophosphorous insecticides. Quantitative structure- property relationship analysis was performed on a series of 43 insecticides using Multiple Linear Regression (MLR) and Support Vector Machines (SVM) methods, which correlate octanol- water partition coefficient (Kow) values of these chemicals to their structural descriptors. At first, the data set was separated with duplex algorithm into a training set (22 chemicals) and a test set (21 chemicals) for statistical external validation. The IX'XI ratio for the two data sets was 0.9839 indicating that the volumes of the regions covered by the two data sets were approximately the same. Model with four descriptors was developed using as independent variables theoretical descriptors derived from DRAGON software when applying GA (Genetic Algorithm)- VSS (Variable Subset Selection) procedure . The values of statistical parameters R2, Q2ext, SDEPext and SDEC for MLR and SVM model were: (93.57%; 92.73%; 0.493; 0.463), (98.60%; 96.30%; 0.504; 0.316); obtained for the two approaches are very similar, which confirm that our four parameters model is stable, robust and significant.

Foodinformatics: Quantitative Structure-Property Relationship Modeling of Volatile Organic Compounds in Peppers.

The aim of this work was the foodinformatic (chemoinformatic) modeling of volatile organic compounds (VOCs) of different samples of peppers based on a quantitative structure-property relationship (QSPR) for the retention indices of 273 identified compounds. The experimental retention indices were measured by means of comprehensive two-dimensional gas chromatography combined with quadrupole-mass spectrometry (GC × GC/qMS) using the BPX5 and BP20 column coupled system. All the VOCs were represented by means of both conformation-independent molecular descriptors and molecular fingerprints calculated in the Dragon and PaDEL-Descriptor software. The dataset was divided into training, validation and test sets of molecules according to the Balanced Subsets Method (BSM). Subsequently, the V-WSP unsupervised variable reduction method was used to reduce the presence of multicollinearity, redundancy, and noise in the initial pool of 4,336 molecular descriptors and fingerprints. Using this method, a reduced pool of 1,664 was submitted to the supervised selection by means of the replacement method (RM) variable subset selection in order to define a four-descriptor model. The quality of the model was measured by means of the coefficient of determination and the root-mean-square deviation in fitting ( and ), validation ( and ), and prediction ( and ). The negligible differences among the parameters in the three sets indicate a stable and predictive QSPR model. This quantitative structure-activity relationship was developed keeping in mind the five principles defined by the Organization for Economic Co-operation and Development (OECD) to make it applicable. PRACTICAL APPLICATION: This predictive mathematical model developed from the retention indices of 273 volatile organic compounds (VOCs) detected in pepper samples could be useful for chromatographers working on the identification of other common VOCs in peppers or other foods by means of comprehensive two-dimensional gas chromatography combined with quadrupole-mass spectrometry (GC × GC/qMS) using a bi-dimensional stationary phase coupled system (BPX5 and BP20).

Quantitative structure-property relationship studies for the prediction of the vapor pressure of volatile organic compounds

A theoretical model (QSPR) using multiple linear regression analysis for predicting the vapor pressure (pv) of volatile organic compounds (VOCs) has been developed. A series of 51 compounds were analyzed by multiple linear regression analysis. First, the data set was separated arbitrarily into a training set (39 chemicals) and a test set (12 chemicals) for statistical external validation. A four-dimensional model was developed using as independent variables theoretical descriptors derived from Dragon software when applying the GA (genetic algorithm)?VSS (variable subset selection) procedure. The obtained model was used to predict the vapor pressure of the test set compounds, and an agreement between experimental and predicted values was verified. This model, with high statistical significance (R2 = 0.9090, Q2 LOO = 0.8748, Q2 ext = 0.8307, s = 0.24), could be used adequately for the prediction and description of the log pv value of other VOCs. The applicability domain of MLR model was investigated using a William?s plot to detect outliers and outsides compounds.

A Sub-Model Theorem for Ordinary Least Squares

Variable selection or subset selection is an important step in the process of model fitting. There are many ways to select the best subset of variables including Forward selection, Backward elimination, etcetera. Ordinary least squares (OLS) is one of the most commonly used methods of fitting the final model. Final sub-model can perform poorly if the variable selection process failed to choose the right number of variables. This paper gives a new theorem and a mathematical proof to illustrate the reason for the poor performances, when using the least squares method after variable selection.

New structure-based models for the prediction of flash point of multi-component organic mixtures

In this study, new models based on quantitative structure–property relationship (QSPR) have been developed to estimate the flash point of multi-component mixtures. For the model development, a large data set including 921 experimental data on the flash point of 93 different binary mixtures including different chemical families was applied. Enhanced replacement method (ERM) was employed for subset variable selection. Based on the ERM selected molecular descriptors of mixture, two new models including linear model, and artificial neural network (ANN) based model have been proposed. The prediction capability of the developed models was evaluated using different statistical criteria. Besides, the estimation capability of the developed models was evaluated by estimating the flash point of 4 different ternary mixtures containing 221 experimental data as an extra external validation data set based on “mixture out” external validation strategy. All statistical analysis confirmed that the proposed models are reliable and accurate.

Utility Metrics for Assessment and Subset Selection of Input Variables for Linear Estimation [Tips & Tricks

This tutorial article introduces the utility metric and its generalizations, which allow for a quick-and-dirty quantitative assessment of the relative importance of the different input variables in a linear estimation model. In particular, we show how these metrics can be cheaply calculated, thereby making them very attractive for model interpretation, online signal quality assessment, or greedy variable selection. The main goal of this article is to provide a transparent and consistent framework that consolidates, unifies, and extends the existing results in this area. In particular, we (1) introduce the basic utility metric and show how it can be calculated at virtually no cost, (2) generalize it toward group-utility and noise-impact metrics, and (3) further extend it to cope with linearly dependent inputs and minimum norm requirements.

An integrated dominance analysis and dynamic programing approach for measuring predictor importance for customer satisfaction

Dominance analysis is a procedure for measuring the importance of predictors in multiple regression analysis. We show that dominance analysis can be enhanced using a dynamic programing approach for the rank-ordering of predictors. Using customer satisfaction data from a call center operation, we demonstrate how the integration of dominance analysis with dynamic programing can provide a better understanding of predictor importance. As a cautionary note, we recommend careful reflection on the relationship between predictor importance and variable subset selection. We observed that slight changes in the selected predictor subset can have an impact on the importance rankings produced by a dominance analysis.

Developing intelligent classification models for rock burst prediction after recognizing significant predictor variables, Section 1: Literature review and data preprocessing procedure

Rock bursts constitute serious hazards in underground mining and excavating. Up to now, numerous researches in the form of empirical, experimental, analytical, intelligent and numerical methods with their own specific scope, characteristics, strengths and weaknesses, have been conducted for rock burst prediction. The weaknesses and limitations of the mentioned prediction methods, especially the intelligent studies, indicate the need for continuing the researches in this field. In this research, a rock burst database, consisting of 188 datasets, was considered. Each dataset corresponds to a series of predictor variables and one of defined classes for the dependent variable “rock burst intensity”. To design classification models, describing important characteristics of datasets and predicting future trends, a data preprocessing procedure was conducted. The procedure consisted of a statistical analysis strategy, a metaheuristic technique for feature (variable) subset selection and some feature extraction techniques. The statistical analysis led to conclude that by considering the available datasets, some predictor variables have statistically insignificant contributions for rock burst prediction. By contrast, the other predictor variables have considerable ordinal contributions. These statistical inferences were completely in accordance with the results of the feature subset selection technique. Besides, the application of this technique revealed specific combinations of significant predictor variables having the highest priorities for modelling the dependent variable. The application of feature extraction techniques to construct derived components from initial datasets did not lead to representative results. Therefore, a high rank combination of significant predictor variables can be adopted to design and develop new classification models based on the considered datasets.

Quantifying the Link between Employee Engagement and Customer Satisfaction in the Car Rental Industry

In the retail service industry, employee engagement may play an important role in customer relationship management, as employees often interact directly with customers. This paper investigates the link between employee engagement and customer satisfaction. We analyze this potential link using a unique data set from a large car rental company. Our analysis accounts for the possibility that employee engagement is endogenously determined. To address potential endogeneity of employee engagement, we make use of instrumental variables based on location and time varying employee composition. Our analysis confirms that there is a positive effect of employee engagement on customer satisfaction, which can in turn increase customer loyalty. The causal link between employee engagement and customer satisfaction is especially important, as our analysis using LASSO (for variable subset selection) highlights that customer satisfaction is among the most important predictors of customer loyalty. Ultimately, our findings have implications not just for customer targeting but also firm resource allocation decisions regarding employee motivation.

Impact of Antithrombotic Regimen on Mortality, Ischemic, and Bleeding Outcomes after Transcatheter Aortic Valve Replacement.

IntroductionOptimal antithrombotic therapy after transcatheter aortic valve replacement (TAVR) remains unclear. We evaluated the association between antithrombotic regimens and outcomes in TAVR patients.MethodsWe retrospectively analyzed consecutive patients who underwent TAVR at a single academic center from April 2009 to March 2014. Antithrombotic regimens were classified as single or dual antiplatelet therapy (AP), single antiplatelet plus anticoagulant (SAC), or triple therapy (TT). The primary endpoint was a composite of death, myocardial infarction (MI), stroke, and major bleeding. Adjusted hazard ratios (HRs) were obtained with best subset variable selection methods using bootstrap resampling.ResultsOf 246 patients who underwent TAVR, 241 were eligible for analysis with 133, 88, and 20 patients in the AP, SAC, and TT groups, respectively. During a median 2.1-year follow-up, 53.5% had at least one endpoint—the most common was death (68%), followed by major bleeding (23%), stroke (6%), and MI (3%). At 2 years, the composite outcome occurred in 70% of TT, 42% of SAC, and 31% of AP patients. Compared to AP, adjusted HRs for the composite outcome were 2.88 [95% Confidence intervals (CI) (1.61–5.16); p = 0.0004] and 1.66 (95% CI [1.13-2.42]; p = 0.009) in the TT and SAC groups, respectively. Mortality rates at 2 years were 61% in the TT, 32% in the SAC, and 26% in the AP groups (p = 0.005).ConclusionsThe risk of the composite outcome of death, MI, stroke, or major bleeding at 2-year follow-up was significantly higher in TAVR patients treated with TT or SAC versus AP, even after multivariate adjustment.

Quantitative structure-activity relationship (QSAR) analysis of plant-derived compounds with larvicidal activity against Zika Aedes aegypti (Diptera: Culicidae) vector using freely available descriptors.

We have developed a quantitative structure-activity relationship (QSAR) model for predicting the larvicidal activity of 60 plant-derived molecules against Aedes aegypti L. (Diptera: Culicidae), a vector of several diseases such as dengue, yellow fever, chikungunya and Zika. The balanced subsets method (BSM) based on k-means cluster analysis (k-MCA) was employed to split the data set. The replacement method (RM) variable subset selection technique coupled with multivariable linear regression (MLR) proved to be successful for exploring 18 326 molecular descriptors and fingerprints calculated with PaDEL, Mold2 and EPI Suite open-source softwares. A robust QSAR model (Rtrain2=0.84, Strain = 0.20 and Rtest2=0.92, Stest = 0.23) involving five non-conformational descriptors was established. The model was validated and tested through the use of an external test set of compounds, the leave-one-out (LOO) and leave-more-out (LMO) cross-validation methods, Y-randomization and applicability domain (AD) analysis. The QSAR model surpasses previously published models based on geometrical descriptors, thereby representing a suitable tool for predicting larvicidal activity against the vector A. aegypti using a conformation-independent approach. © 2018 Society of Chemical Industry.

Linear Regression QSAR Models for Polo-Like Kinase-1 Inhibitors.

A structurally diverse dataset of 530 polo-like kinase-1 (PLK1) inhibitors is compiled from the ChEMBL database and studied by means of a conformation-independent quantitative structure-activity relationship (QSAR) approach. A large number (26,761) of molecular descriptors are explored with the main intention of capturing the most relevant structural characteristics affecting the bioactivity. The structural descriptors are derived with different freeware, such as PaDEL, Mold2, and QuBiLs-MAS; such descriptor software complements each other and improves the QSAR results. The best multivariable linear regression models are found with the replacement method variable subset selection technique. The balanced subsets method partitions the dataset into training, validation, and test sets. It is found that the proposed linear QSAR model improves previously reported models by leading to a simpler alternative structure-activity relationship.

Tool for osteoarthritis risk prediction (TOARP) over 8 years using baseline clinical data, X-ray, and MRI: Data from the osteoarthritis initiative.

Osteoarthritis (OA), a multifactorial disease causing joint degeneration, often leads to severe disability. The rising rates of disability highlight the need for implementing preventative measures at early stages of the disease, which would especially benefit subjects at high risk for OA development. To develop a risk prediction tool for moderate-severe OA (TOARP) over 8 years based on subject characteristics, knee radiographs, and MRI data at baseline using data from the Osteoarthritis Initiative (OAI). Retrospective. 641 subjects with no/mild radiographic OA (Kellgren-Lawrence [KL] 0-2) and no clinically significant symptoms (Western Ontario and McMaster Universities Arthritis Index [WOMAC] 0-1) were selected from the OAI. MR images were obtained using 3.0T. Compartment-specific cartilage and meniscus morphology and cartilage T2 were assessed. Baseline subject demographics, risk factors, KL score, cartilage WORMS score, presence of meniscus tear, and cartilage T2 were used to predict the development of moderate/severe OA (KL = 3-4 or WOMAC pain ≥5 or total knee replacement [TKR]) over 8 years. Best subsets variable selection followed by cross-validation were used to assess which combinations of variables best predict moderate/severe OA. Model 1 included KL score, previous knee injury in the last 12 months, age, gender, and BMI. Model 2 included all variables in Model 1 plus presence of cartilage defects in the lateral femur and patella, and presence of a meniscal tear. Model 3 included all variables in Models 1 and 2, plus cartilage T2 in the medial tibia and medial femur. Compared to Model 1 (cross-validated AUC = 0.67), Model 3 performed significantly better (AUC = 0.72, P = 0.04), while Model 2 showed a statistical trend (AUC = 0.71, P = 0.08). We established a risk calculator for the development of moderate/severe knee OA over 8 years that includes radiographic and MRI data. The inclusion of MRI-based morphological abnormalities and cartilage T2 significantly improved model performance. 2 Technical Efficacy: Stage 3 J. Magn. Reson. Imaging 2018;47:1517-1526.