Variable Selection Strategy Research Articles

In-line monitoring of the transesterification reactions for biodiesel production using NIR spectroscopy

In this work, methods for in-line monitoring of the transesterification reactions of soybean oil with methanol using NIR spectroscopy are developed. Gas chromatography was employed for determining the contents of methyl ester (ME), monoglycerides (MG), diglycerides (DG) and triglycerides (DG) during the transesterification reaction and used as reference to build partial least squares regression (PLS) and multiple linear regression (MLR) calibration models employing NIR spectra. Some pre-processing and variable selection strategies of the spectral data were evaluated. Satisfactory RMSEP values were obtained for the external prediction subset. In addition, a multivariate control chart based on latent variables to monitor the progress of the reactions is also presented.

TU‐G‐108‐05: Assessment of Different Machine Learning Techniques for Multivariate Radiation Pneumonitis Modeling

Purpose: We intended to perform an independent verification of different machine learning methods for inferring radiation pneumonitis (RP) risk from patient‐specific biological and dosimetric informationMethods: 29 NSCLC patients who received chemoradiation were recruited from two institutions (22 from WUSTL, 7 from McGill). Blood samples were collected from each patient before and during radiotherapy (RT). From each sample the concentration of the five following candidate biomarkers were measured: alpha‐2‐macroglobulin (α2M), angiotensin converting enzyme (ACE), transforming growth factor β (TGF‐β), interleukin‐6 (IL‐6), and osteopontin (OPN). Dimensionality of the raw biomarker data was reduced by a semi‐supervised variable selection scheme. The reduced biomarker variables along with three known dosimetric RP predictors (mean lung dose, V20, tumor position in superior‐inferior direction of the lung) were used as features for classifying high RP risk (CTCAE grade 2 or higher) patients. Four different machine learning methods (Bayesian Network, Logistic Regression, Naive Bayes, Support Vector Machine) were used for training classifiers on the WUSTL subset and were tested with respect to classification performance on the McGill subset. Results: The following 4 biomarker variables were chosen by the variable filtering: (1) pre‐RT concentration level of α2M, (2) ratio of pre‐to intra‐RT levels of α2M, (3) intra‐RT ACE level, (4) intra‐RT TGFβ level. The best performance was achieved by Support Vector Machine in terms of classification accuracy (85.71%) and the area under the ROC curve (AUC) (0.9167). Bayesian Network recorded the same accuracy and slightly lower AUC (0.8750). Less accurate models were Naive Bayes and logistic regression in which overfitting was predominant with the largest difference in classification performance between the training and the testing dataset. Conclusion: Preliminary results from this ongoing study suggests the need of a machine learning approach capable of modeling inter‐relation between physical and biological variables where commonly used multivariate logistic regression would be inappropriate.

A comparison of multivariate analysis techniques and variable selection strategies in a laser-induced breakdown spectroscopy bacterial classification

Laser-induced breakdown spectroscopy has been used to obtain spectral fingerprints from live bacterial specimens from thirteen distinct taxonomic bacterial classes representative of five bacterial genera. By taking sums, ratios, and complex ratios of measured atomic emission line intensities three unique sets of independent variables (models) were constructed to determine which choice of independent variables provided optimal genus-level classification of unknown specimens utilizing a discriminant function analysis. A model composed of 80 independent variables constructed from simple and complex ratios of the measured emission line intensities was found to provide the greatest sensitivity and specificity. This model was then used in a partial least squares discriminant analysis to compare the performance of this multivariate technique with a discriminant function analysis. The partial least squares discriminant analysis possessed a higher true positive rate, possessed a higher false positive rate, and was more effective at distinguishing between highly similar spectra from closely related bacterial genera. This suggests it may be the preferred multivariate technique in future species-level or strain-level classifications.

Level of information about the relationship between diabetes mellitus and periodontitis - results from a nationwide diabetes information program

BackgroundA comprehensive knowledge about the mutual influence between diabetes and periodontitis is decisive for the successful treatment of both diseases. The present investigation aimed at assessing the diabetic and periodontal conditions and, in particular, the degree of knowledge about the relationship between diabetes and periodontitis.MethodsDuring a diabetes information program, 111 nondiabetics (ND), 101 type 1 diabetics (T1D), and 236 type 2 diabetics (T2D) were subject to a medical and dental examination and completed a self-administered questionnaire. Medical examination included measurements of glycated hemoglobin (HbA1c), blood glucose (BG), and body mass index (BMI). Full-mouth examination consisted of the assessment of the decayed, missing, filled teeth index (DMFT) and the periodontal screening index (PSI). Chi-square test, ANOVA, t test of independent samples, univariate and multivariate logistic regression models with variable selection strategies were used for statistical analyses. Due to the exploratory character of the investigation a value of P ≤0.05 was considered to be statistically substantial.ResultsT2D had a significantly higher PSI when compared to T1D and ND (t test: P <0.001; P = 0.005). Approximately 90% of T2D suffered from periodontitis. In addition, diabetics with periodontitis showed a significantly higher BMI when compared to diabetics without periodontitis (multivariate logistic regression: P = 0.002). Almost 60% of all investigated subjects were not informed about the mutual influence between diabetes and periodontitis. T2D had almost as little information about the increased risk for periodontitis as ND.ConclusionsThe data of the present investigation suggest that there is a strong association between type 2 diabetes and chronic periodontitis. The lack of awareness of the mutual influence between diabetes and periodontitis, especially in T2D, demonstrates that this topic is still neglected in dental and diabetic treatment.

Improved Sparse Multi-Class SVM and Its Application for Gene Selection in Cancer Classification

BackgroundMicroarray techniques provide promising tools for cancer diagnosis using gene expression profiles. However, molecular diagnosis based on high-throughput platforms presents great challenges due to the overwhelming number of variables versus the small sample size and the complex nature of multi-type tumors. Support vector machines (SVMs) have shown superior performance in cancer classification due to their ability to handle high dimensional low sample size data. The multi-class SVM algorithm of Crammer and Singer provides a natural framework for multi-class learning. Despite its effective performance, the procedure utilizes all variables without selection. In this paper, we propose to improve the procedure by imposing shrinkage penalties in learning to enforce solution sparsity.ResultsThe original multi-class SVM of Crammer and Singer is effective for multi-class classification but does not conduct variable selection. We improved the method by introducing soft-thresholding type penalties to incorporate variable selection into multi-class classification for high dimensional data. The new methods were applied to simulated data and two cancer gene expression data sets. The results demonstrate that the new methods can select a small number of genes for building accurate multi-class classification rules. Furthermore, the important genes selected by the methods overlap significantly, suggesting general agreement among different variable selection schemes.ConclusionsHigh accuracy and sparsity make the new methods attractive for cancer diagnostics with gene expression data and defining targets of therapeutic intervention.Availability: The source MATLAB code are available from http://math.arizona.edu/~hzhang/software.html.

A two-stage variable selection strategy for supersaturated designs with multiple responses

A supersaturated design (SSD), whose run size is not enough for estimating all the main effects, is commonly used in screening experiments. It offers a potential useful tool to investigate a large number of factors with only a few experimental runs. The associated analysis methods have been proposed by many authors to identify active effects in situations where only one response is considered. However, there are often situations where two or more responses are observed simultaneously in one screening experiment, and the analysis of SSDs with multiple responses is thus needed. In this paper, we propose a two-stage variable selection strategy, called the multivariate partial least squares-stepwise regression (MPLS-SR) method, which uses the multivariate partial least squares regression in conjunction with the stepwise regression procedure to select true active effects in SSDs with multiple responses. Simulation studies show that the MPLS-SR method performs pretty good and is easy to understand and implement.

Variable selection strategies for nearest neighbor imputation methods used in remote sensing based forest inventory

We examined the problem of selecting predictor variables for Nearest Neighbor (NN) imputation in remote sensing based forest inventory. Eighty-three variables were calculated from Airborne Laser Scanning data and aerial images, with responses being either dominant height or a set of five common stand attributes. Three different approaches were compared with select predictor variables. Analyses were repeated with three different NN imputation methods using a varying number of predictor variables. Results indicated that variable selection is justified, but it must be done properly. The most accurate method to select predictors was to minimize error using Simulated Annealing. For a single response, the most accurate imputation method was Random Forest proximity matrix-based imputation, whereas Most Similar Neighbor was the most accurate for five responses. An optimization-based distance metric also worked well. We also examined the degree to which different imputation methods are prone to overfitting as well as how to properly do cross-validation in NN imputation.

Application of modified particle swarm optimization as an efficient variable selection strategy in QSAR/QSPR studies

In this work, we present a novel method combining modified particle swarm optimization (MPSO) and partial least squares (PLS) for selecting a subset of relevant descriptors and building the optimal linear model for quantitative structure–activity relationship (QSAR)/quantitative structure–property relationship (QSPR) studies. The proposed method is applied to the QSPR modeling of the entropy of formation for 1133 different organic compounds. By using the MPSO–PLS approach, three descriptors are selected from a large set of molecular descriptors, and a linear model is developed. Analysis of the results indicates the squared correlation coefficient of 0.9691 for MPSO–PLS model. Thus, MPSO–PLS is a promising technique, which is helpful for developing the QSAR/QSPR models with high accuracy. Copyright © 2012 John Wiley &amp; Sons, Ltd.

Variable selection for propensity score models when estimating treatment effects on multiple outcomes: a simulation study

It is often preferable to simplify the estimation of treatment effects on multiple outcomes by using a single propensity score (PS) model. Variable selection in PS models impacts the efficiency and validity of treatment effects. However, the impact of different variable selection strategies on the estimated treatment effects in settings involving multiple outcomes is not well understood. The authors use simulations to evaluate the impact of different variable selection strategies on the bias and precision of effect estimates to provide insight into the performance of various PS models in settings with multiple outcomes. Simulated studies consisted of dichotomous treatment, two Poisson outcomes, and eight standard-normal covariates. Covariates were selected for the PS models based on their effects on treatment, a specific outcome, or both outcomes. The PSs were implemented using stratification, matching, and weighting (inverse probability treatment weighting). PS models including only covariates affecting a specific outcome (outcome-specific models) resulted in the most efficient effect estimates. The PS model that only included covariates affecting either outcome (generic-outcome model) performed best among the models that simultaneously controlled measured confounding for both outcomes. Similar patterns were observed over the range of parameter values assessed and all PS implementation methods. A single, generic-outcome model performed well compared with separate outcome-specific models in most scenarios considered. The results emphasize the benefit of using prior knowledge to identify covariates that affect the outcome when constructing PS models and support the potential to use a single, generic-outcome PS model when multiple outcomes are being examined.

Multiresolution local binary pattern texture analysis combined with variable selection for application to false-positive reduction in computer-aided detection of breast masses on mammograms

In this paper, a new and novel approach is designed for extracting local binary pattern (LBP) texture features from the computer-identified mass regions, aiming to reduce false-positive (FP) detection in a computerized mass detection framework. The proposed texture feature, the so-called multiresolution LBP feature, is well able to characterize the regional texture patterns of core and margin regions of a mass, as well as to preserve the spatial structure information of the mass. In addition, to maximize a complementary effect on improving classification accuracy, multiresolution texture analysis has been incorporated into the extraction of LBP features. Further, SVM-RFE-based variable selection strategy is applied for selecting an optimal subset of variables of multiresolution LBP texture features to maximize the separation between breast masses and normal tissues. Extensive and comparative experiments have been conducted to evaluate the proposed method on two public benchmark mammogram databases (DBs). Experimental results show that the proposed multiresolution LBP features (extracted from automatically segmented mass boundaries) outperform other state-of-the-art texture features developed for FP reduction. Our results also indicate that combining our multiresolution LBP features with variable selection strategy is an effective solution for reducing FP signals in computer-aided detection (CAD) of mammographic masses.

Location and allocation based branch and bound algorithms for the capacitated multi-facility Weber problem

Given the locations of J customers, their demands and I capacitated facilities, the Capacitated Multi-facility Weber Problem (CMWP) is concerned with locating I facilities in the plane to satisfy the demand of J customers with the minimum total transportation cost which is proportional to the distance between them. We propose two types of branch and bound algorithms for the lr distance CMWP with 1≤r<∞. One of them is an allocation space based branch and bound algorithm for which a new branching variable selection strategy and new lower bounding procedures have been proposed. The other one is new and partitions the location space. Based on extensive computational experiments we can say that the proposed algorithms are promising and perform better than the existing ones.

Planning as satisfiability: Heuristics

Reduction to SAT is a very successful approach to solving hard combinatorial problems in Artificial Intelligence and computer science in general. Most commonly, problem instances reduced to SAT are solved with a general-purpose SAT solver. Although there is the obvious possibility of improving the SAT solving process with application-specific heuristics, this has rarely been done successfully.In this work we propose a planning-specific variable selection strategy for SAT solving. The strategy is based on generic principles about properties of plans, and its performance with standard planning benchmarks often substantially improves on generic variable selection heuristics, such as VSIDS, and often lifts it to the same level with other search methods such as explicit state-space search with heuristic search algorithms.

VARIABLE SELECTION IN LINEAR MIXED EFFECTS MODELS.

This paper is concerned with the selection and estimation of fixed and random effects in linear mixed effects models. We propose a class of nonconcave penalized profile likelihood methods for selecting and estimating important fixed effects. To overcome the difficulty of unknown covariance matrix of random effects, we propose to use a proxy matrix in the penalized profile likelihood. We establish conditions on the choice of the proxy matrix and show that the proposed procedure enjoys the model selection consistency where the number of fixed effects is allowed to grow exponentially with the sample size. We further propose a group variable selection strategy to simultaneously select and estimate important random effects, where the unknown covariance matrix of random effects is replaced with a proxy matrix. We prove that, with the proxy matrix appropriately chosen, the proposed procedure can identify all true random effects with asymptotic probability one, where the dimension of random effects vector is allowed to increase exponentially with the sample size. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. We further illustrate the proposed procedures via a real data example.

Variable selection for functional density trees

In this paper, the exhaustive search principle used in functional trees for classifying densities is shown to select variables with more split points. A new variable selection scheme is proposed to correct this bias. The Pearson chi-squared tests for associated two-way contingency tables are used to select the variables. Through simulation, we show that the new method can control bias and is more powerful in selecting split variable.

Characterization of data analysis methods for information recovery from metabolic 1H NMR spectra using artificial complex mixtures

The assessment of data analysis methods in 1H NMR based metabolic profiling is hampered owing to a lack of knowledge of the exact sample composition. In this study, an artificial complex mixture design comprising two artificially defined groups designated normal and disease, each containing 30 samples, was implemented using 21 metabolites at concentrations typically found in human urine and having a realistic distribution of inter-metabolite correlations. These artificial mixtures were profiled by 1H NMR spectroscopy and used to assess data analytical methods in the task of differentiating the two conditions. When metabolites were individually quantified, volcano plots provided an excellent method to track the effect size and significance of the change between conditions. Interestingly, the Welch t test detected a similar set of metabolites changing between classes in both quantified and spectral data, suggesting that differential analysis of 1H NMR spectra using a false discovery rate correction, taking into account fold changes, is a reliable approach to detect differential metabolites in complex mixture studies. Various multivariate regression methods based on partial least squares (PLS) were applied in discriminant analysis mode. The most reliable methods in quantified and spectral 1H NMR data were PLS and RPLS linear and logistic regression respectively. A jackknife based strategy for variable selection was assessed on both quantified and spectral data and results indicate that it may be possible to improve on the conventional Orthogonal-PLS methodology in terms of accuracy and sensitivity. A key improvement of our approach consists of objective criteria to select significant signals associated with a condition that provides a confidence level on the discoveries made, which can be implemented in metabolic profiling studies.

Detecting disease-associated genes with confounding variable adjustment and the impact on genomic meta-analysis: With application to major depressive disorder

BackgroundDetecting candidate markers in transcriptomic studies often encounters difficulties in complex diseases, particularly when overall signals are weak and sample size is small. Covariates including demographic, clinical and technical variables are often confounded with the underlying disease effects, which further hampers accurate biomarker detection. Our motivating example came from an analysis of five microarray studies in major depressive disorder (MDD), a heterogeneous psychiatric illness with mostly uncharacterized genetic mechanisms.ResultsWe applied a random intercept model to account for confounding variables and case-control paired design. A variable selection scheme was developed to determine the effective confounders in each gene. Meta-analysis methods were used to integrate information from five studies and post hoc analyses enhanced biological interpretations. Simulations and application results showed that the adjustment for confounding variables and meta-analysis improved detection of biomarkers and associated pathways.ConclusionsThe proposed framework simultaneously considers correction for confounding variables, selection of effective confounders, random effects from paired design and integration by meta-analysis. The approach improved disease-related biomarker and pathway detection, which greatly enhanced understanding of MDD neurobiology. The statistical framework can be applied to similar experimental design encountered in other complex and heterogeneous diseases.

QSPR-based estimation of the half-lives for polychlorinated biphenyl congeners

In this study, the depuration half-lives of 62 polychlorinated biphenyl (PCB) congeners in juvenile rainbow trout (Oncorhynchus mykiss) were estimated from their structural features based on QSPR methodology. A genetic algorithm (GA) was applied as a variable subset selection strategy and QSPR models established upon multiple linear regression (MLR), multilayer perceptron neural network (MLP NN) and support vector regression (SVR) procedures. Robustness and predictive stability of the constructed models were evaluated through internal and external validation methods. The high numerical values of and , and low RMSE in the case of the MLP NN model, confirm the supremacy of this model as well as nonlinear dependency of molecular structural features to the PCB congeners half-lives. In the best developed QSPR model the following four descriptors are used; lopping centric index (Lop), mean topological charge index of order 1 (JGI1), Geary autocorrelation lag-8/weighted by atomic Sanderson electronegativities (GATS8e) and highest eigenvalue of Burden matrix/weighted by atomic masses (BEHm3). Analysis of the descriptors involved in these models revealed that 2D molecular structural features, compactness and electronegativities are the main factors contributing to the half-lives of PCBs. The structural information presented in this work can be used for further evaluation of half-lives of PCBs and other similar structural compounds in the environment.

Fine tuning genomic evaluations in dairy cattle through SNP pre-selection with the Elastic-Net algorithm

For genomic selection methods, the statistical challenge is to estimate the effect of each of the available single-nucleotide polymorphism (SNP). In a context where the number of SNPs (p) is much higher than the number of bulls (n), this task may lead to a poor estimation of these SNP effects if, as for genomic BLUP (gBLUP), all SNPs have a non-null effect. An alternative is to use approaches that have been developed specifically to solve the 'p >> n' problem. This is the case of variable selection methods and among them, we focus on the Elastic-Net (EN) algorithm that is a penalized regression approach. Performances of EN, gBLUP and pedigree-based BLUP were compared with data from three French dairy cattle breeds, giving very encouraging results for EN. We tried to push further the idea of improving SNP effect estimates by considering fewer of them. This variable selection strategy was considered both in the case of gBLUP and EN by adding an SNP pre-selection step based on quantitative trait locus (QTL) detection. Similar results were observed with or without a pre-selection step, in terms of correlations between direct genomic value (DGV) and observed daughter yield deviation in a validation data set. However, when applied to the EN algorithm, this strategy led to a substantial reduction of the number of SNPs included in the prediction equation. In a context where the number of genotyped animals and the number of SNPs gets larger and larger, SNP pre-selection strongly alleviates computing requirements and ensures that national evaluations can be completed within a reasonable time frame.

English

Enumeration strategies (that is, selection of a variable and a value of its domain) are crucial components of Constraint Programming: they significantly influence the performances of the solving process, sometimes of several orders of magnitude. In this paper, we propose to use Local Search in order to help and guide enumeration: we extend the usual variable selection strategies of constraint programming and we perform the value selection with respect to the results of some local search. The experimental results obtained are rather promising.

1H NMR variable selection approaches for classification. A case study: The determination of adulterated foodstuffs

Whenever dealing with large amount of data as is the case of a NMR spectrum, carrying out a variable selection before applying a multivariate technique is necessary. This work applies various variable selection techniques to extract relevant information from 1H NMR spectral data. Three approaches have been chosen, because each is based on very different foundations. The first method, called Xdiff, is based on calculating the normalized differences between the mean spectrum of a class considered to be the reference and the spectra of each sample. The second approach is the interval Partial Least Squares method (iPLS), which investigates the influential zones of the spectra that contains the most discriminating predictors calculating local PLS-DA models on narrow intervals. The last one is Genetic Algorithms (GAs) which finds the optimal variables from a random initial subset of variables by means of an iterative process. The performance of each variable selection strategy is determined by the classification results obtained when multiclass Partial Least Squares-Discriminant Analysis is applied. This study has been applied to NMR spectra of culinary spices that might be adulterated with banned dyes such as Sudan dyes (I–IV). The three techniques give neither the same number nor the same selected variables, but they do select a common zone from the spectra containing the most discriminating variables. All three techniques give satisfactory classification and prediction results, being higher than 95% with iPLS and GA and around 89% with Xdiff, therefore the three variable selection techniques are suitable to be used with NMR data in the determination of food adulteration with Sudan dyes as well as the specific type of adulterant used (I–IV).